Retry tracker registration when initial connect fails.

Start background re-registration when the tracker is unreachable at startup so nodes do not stay permanently unregistered.

Co-authored-by: Cursor <cursoragent@cursor.com>

.claude/memory/project-status.md

@@ -35,3 +35,14 @@ Historical handoff note: `/mnt/c/Users/popov/Downloads/neuron-tai-alpha-handoff-
 - Run: `meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct`
 - Known: tracker registration fails with `http://` — must use `https://`
 - pynvml deprecation warning is harmless (use nvidia-ml-py to silence it)
+
+## Model cache layout (2026-07-07)
+- Node downloads now cache files directly under `<download_dir>/<model>/`, not `<model>/layers_<start>-<end>/`, so a wider cached layer assignment can satisfy a later narrower assignment without duplicate shard folders.
+- Downloader checks tracker-advertised `files` + `file_sizes` before peer/HF download; complete local files return immediately and preserve any extra files already in the model folder.
+- Verification: downloader/startup targeted subset passes (`pytest tests/test_node_startup.py -k "download_shard or same_shard"`). Full `tests/test_node_startup.py` has 46 passed and 4 unrelated Windows chmod/path separator failures.
+- Live Windows confirmation: `meshnet-node start --tracker http://192.168.0.179:8080 --model Qwen3.6-35B-A3B` reuses `F:\_STORAGE\models\qwen3.6-35b-a3b`, prints `Cached at`, registers, and reaches ready as node `5gMLrmyB-26b1f8a4204a`.
+- Follow-up fix: preset-model startup now starts the heartbeat thread after registration; without this, the node appeared briefly on the dashboard and was purged on first inference/route after heartbeat expiry. Tracker dashboard now has a "Console output" panel backed by `/v1/console` for node register/expiry, routing failures, and proxy events.
+- Qwen3.6-35B-A3B reserve-based split is expected: an 79 GB CPU node may be assigned layers 0-36, and a second node fills 37-39. Do not "fix" this by bypassing the 20% assignment reserve unless the shard-planning policy changes.
+- Route hardening: tracker chat proxy and `/v1/route` diagnostics now use alias-aware preset node matching for split Qwen3.6 routes; dashboard derives grouped inference history from proxy route/complete console events and shows observed TPS after completion.
+- Live proxy hardening: model lookup trims outer whitespace before alias matching (`qwen3.6-35b-a3b ` resolves), and tracker route logs/dashboard queue depth combine heartbeat queue with tracker-local proxy in-flight counts so Postman-style bursts no longer show every selected route as queue `0`.
+- Split-shard streaming hardening: Qwen3.6-style distributed generation now emits SSE chunks token-by-token from the head node instead of buffering all generated text until completion. Tracker direct/relay stream proxy logs `proxy progress` with live tokens/TPS, dashboard Inference history shows currently processing requests with live TPS/tokens/queue, and relay stream completion no longer references an undefined `session_id`.

.env.testnet

@@ -5,3 +5,9 @@ MESHNET_CONTRACT_ADAPTER=solana-testnet
 MESHNET_REGISTRY_PROGRAM_ID=
 MESHNET_PAYMENT_PROGRAM_ID=
 MESHNET_SETTLEMENT_PROGRAM_ID=
+
+# MESHNET_DOWNLOAD_DIR=
+#  HF_TOKEN=
+# URL_TRACKER=
+# DEFAULT_MODEL=
+

.gitignore

@@ -11,3 +11,12 @@ dist/
 
 # Ralph local runtime state
 .ralph-tui/
+
+
+.env
+.env.*
+!.env.example
+!.env.testnet
+.rocm-local/*
+billing.sqlite
+.pytest-tmp/*

.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md

@@ -0,0 +1,177 @@
+# ADR-0020: Distributed GGUF/llama.cpp Runtime With Per-Shard Local KV
+
+Status: Proposed
+
+## Context
+
+The project currently uses PyTorch/Transformers for real model shards. That decision was captured in ADR-0001 because llama.cpp RPC at the time required the primary node to load the full model and distribute weights to workers, which conflicted with the desired model where nodes independently hold shards.
+
+We now want to serve very large open models, including GLM-5.2 and Ornith-class MoE models, over a torrent-like inference marketplace. CPU and mixed consumer hardware matter. LM Studio and llama.cpp demonstrate much better CPU/GGUF performance than our current PyTorch CPU path. The user also has a personal relationship with Georgi Gerganov, making upstream collaboration plausible.
+
+The current distributed PyTorch path is not yet production-grade: it recomputes the full growing sequence for every output token and disables KV cache inside manual layer calls. It sends hidden activations across seams, not KV, but those activations currently cover the full sequence every decode step.
+
+## Decision
+
+Adopt a distributed GGUF/llama.cpp runtime track while keeping PyTorch as the reference and fast-architecture backend.
+
+The runtime model is:
+
+- GGUF/model artifacts are distributed through torrent/content-addressed storage.
+- Nodes independently acquire and verify artifacts; no root node streams model weights to workers at session start.
+- Tracker chooses a sticky route covering all layers.
+- Each node owns hot KV/state for the layers it executes.
+- Prefill sends chunked activations through the route and builds local per-shard KV.
+- Decode sends one-step activations through the route and appends local KV at every shard.
+- Cache/CDN servers store cold artifacts and optional prefix/session snapshots, not hot per-token KV.
+- Context is capped at 128K for the first serious product path.
+
+## Technical Framework
+
+The design separates five planes:
+
+- **Control plane**: tracker registry, coverage map, route selection, session lifecycle, telemetry, billing, and audit.
+- **Artifact plane**: Shard Swarms, GGUF/safetensors/tokenizer files, manifests, hashes, and local node storage.
+- **Execution plane**: active Inference Route, chunked prefill, one-step decode, and hidden-state movement across activation seams.
+- **Session state plane**: per-shard Hot KV State on route nodes, plus optional Prefix Snapshots outside the hot loop.
+- **Economics/trust plane**: reward accounting, validation events, slash proofs, public/private route policy.
+
+Hard invariants:
+
+1. Public-network Shards are contiguous layer ranges.
+2. Hot KV State is local to the node serving that Shard in that Route Session.
+3. Artifact distribution and route execution are separate systems.
+4. Decode seam payload must be `O(hidden_size)`.
+5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
+6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
+7. Model/backend-specific cache internals stay behind backend capability reports.
+8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
+9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
+
+The full challenge register is in [technical-challenges.md](./technical-challenges.md). The open decision gates are in [decision-framework.md](./decision-framework.md).
+
+Resolved gate:
+
+- Public-network Shards are layer ranges. Tensor-parallel/ring execution belongs inside a trusted node, colocated pod, or future composite node abstraction, not as the v1 public routing primitive.
+- Hot KV State is local to each route node for the Shard it serves. Cache servers may store Prefix Snapshots, but they are not part of the per-token decode path.
+- Distributed Route Session and Hot KV State semantics will be proven in the PyTorch route before llama.cpp/GGUF is extended for layer-boundary execution.
+- Streaming responses are preferred when feasible. Realtime Generation Telemetry is required so clients can see phase, generated token count, and tokens/sec even during prefill or non-streaming fallback paths.
+- llama.cpp/GGUF work targets upstreamable `libllama`/ggml hooks. A prototype fork is acceptable for exploration, but a permanent fork is not the plan.
+- Model targeting is two-tiered: use a small llama.cpp-supported GGUF model for the first protocol smoke test, then use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. GLM-5.2 and Ornith remain later support audits.
+- Alpha fails Route Sessions on route-node loss instead of attempting automatic route repair. Repair requires compatible Prefix Snapshots and is a later capability.
+- v1 activation transfer stays on binary HTTP as defined by ADR-0008. QUIC/WebRTC/custom transport can be introduced later behind the same activation protocol.
+
+## Non-Goals
+
+- Do not put remote cache servers in the per-token hot KV path.
+- Do not require every node to hold the full model.
+- Do not fork llama.cpp long-term if upstream APIs can support the needed layer-boundary hooks.
+- Do not target GLM-5.2 or Ornith first; prove the route/KV protocol on a simpler well-supported GGUF model, then target DeepSeek-V4-Flash as the first serious large model.
+
+## Options Considered
+
+### A. Keep PyTorch-only distributed inference
+
+Pros:
+
+- Easy access to new Hugging Face architectures.
+- Transformers has mature single-process KV semantics.
+- Existing code already loads shards.
+
+Cons:
+
+- CPU inference is much slower than llama.cpp/GGUF.
+- Current distributed path bypasses `generate()` and disables cache.
+- Quantized GGUF ecosystem and LM Studio users are outside the runtime.
+
+### B. Use llama.cpp only as a full local model backend
+
+Pros:
+
+- Quick performance win for nodes with enough RAM/VRAM.
+- Minimal coordination with distributed protocol.
+
+Cons:
+
+- Does not unlock 397B/753B-class models for ordinary nodes.
+- Does not solve marketplace layer routing.
+
+### C. Distributed GGUF with per-shard local KV (chosen)
+
+Pros:
+
+- Aligns with torrent artifact distribution.
+- Avoids root streaming weights to workers.
+- Uses llama.cpp/GGUF performance where supported.
+- Compatible with public node rewards by layer/work contribution.
+- Scales KV memory by layer range.
+
+Cons:
+
+- Requires new runtime APIs around layer-boundary hidden states and per-session KV.
+- Requires model-specific cache metadata for DSA/MLA/hybrid attention.
+- Harder to debug than single-process `generate()`.
+
+### D. Centralized KV cache servers
+
+Pros:
+
+- Easier apparent session failover.
+- Central accounting of active cache.
+
+Cons:
+
+- Puts remote storage in the per-token hot path.
+- Adds bandwidth and latency at the worst possible point.
+- Creates consistency and privacy problems.
+
+Rejected for hot decode. Accepted only for cold prefix snapshots and failover checkpoints.
+
+## Consequences
+
+- ADR-0001 should eventually be amended: PyTorch remains valid, but llama.cpp/GGUF becomes a first-class backend.
+- The activation protocol must split prefill and decode explicitly.
+- Session IDs must be stable across the full request. The current fresh UUID-per-hop-call behavior must change.
+- Backends must report cache budget and cache compatibility.
+- Tracker route selection must include disk, memory pressure, cache warmth, and network latency.
+- Billing can be based on layer work, prefill tokens, decode tokens, and observed route participation.
+- Client UX should stream token deltas when feasible and must include route-session progress telemetry even when token deltas are not streamed.
+
+## Required Runtime Capabilities
+
+PyTorch path:
+
+- manual layer calls with `past_key_values` / model-specific cache object
+- per-shard session cache store
+- prefill chunk append
+- decode step append
+- stable session lifecycle endpoints
+
+llama.cpp/GGUF path:
+
+- full local GGUF serving
+- layer/tensor map extraction from GGUF
+- optional partial layer loading or mmap-backed selected execution
+- inbound hidden-state execution from arbitrary start layer
+- outbound hidden-state return at stop layer
+- per-session KV ownership for loaded layers
+- cache budget/compatibility introspection
+- GLM-5.2 DSA support when upstream/runtime supports it
+
+## Implementation Plan
+
+1. Add full-model `LlamaCppBackend` using `llama-server` or `libllama`.
+2. Implement distributed KV in the PyTorch path to prove semantics.
+3. Add session lifecycle and prefill/decode wire protocol.
+4. Add model artifact manifest and torrent seeding metadata.
+5. Prototype localhost two-process llama.cpp layer boundary execution.
+6. Generalize to network route.
+7. Bring in GLM-5.2/Ornith once backend support and cache accounting are verified.
+
+## Acceptance Criteria
+
+- A two-node localhost route can prefill once and decode N tokens without recomputing the full prompt.
+- Seam payload during decode is `O(hidden_size)`, not `O(sequence_length * hidden_size)`.
+- Per-node KV memory grows with owned layer count and context length.
+- Route loss during alpha fails cleanly with explicit reason.
+- Full local GGUF backend outperforms PyTorch CPU on a supported model.
+- Artifact manifest can identify exactly which files/chunks a node must seed for its advertised layer range.

.scratch/distributed-gguf-runtime/PRD.md

@@ -0,0 +1,83 @@
+# PRD: Distributed GGUF Runtime
+
+## Summary
+
+Build a distributed inference runtime that can serve large, quality-first open models by combining torrent-style model artifact distribution with sticky multi-node Inference Routes and per-shard local Hot KV State.
+
+The first runtime proof uses the existing PyTorch route because it exposes model internals and cache semantics more directly. GGUF/llama.cpp becomes the performance path after the route-session contract is proven.
+
+## Goals
+
+- Eliminate full-prompt recompute in distributed decode.
+- Keep decode activation seams proportional to `hidden_size`, not `context_length * hidden_size`.
+- Keep Hot KV State local to the node serving the relevant Shard.
+- Stream token deltas when feasible and always expose Generation Telemetry.
+- Add a local full-model GGUF backend for immediate CPU performance wins.
+- Define Model Artifact manifests so nodes can verify, seed, and advertise artifacts without depending on Hugging Face at request time.
+- Prototype an upstreamable llama.cpp/libllama layer-boundary API.
+- Use DeepSeek-V4-Flash as the first serious large-model target after smaller protocol smoke tests.
+
+## Non-Goals
+
+- No centralized hot KV cache in the per-token decode path.
+- No automatic route repair in alpha.
+- No permanent llama.cpp fork as the intended architecture.
+- No GLM-5.2 or Ornith first; they remain follow-up support audits.
+- No transport rewrite to QUIC/WebRTC before route/session semantics are proven.
+
+## Resolved Decisions
+
+- Public-network Shards are contiguous transformer layer ranges.
+- Tensor/ring parallelism belongs inside one trusted node, one colocated pod, or a future composite node abstraction.
+- Hot KV State is local to route nodes; Prefix Snapshots are optional cold recovery/reuse artifacts.
+- PyTorch distributed KV/session semantics are proven before llama.cpp distributed execution.
+- Streaming responses are preferred; Generation Telemetry is mandatory.
+- llama.cpp/GGUF work targets upstreamable `libllama`/ggml hooks.
+- Alpha fails Route Sessions on route-node loss.
+- v1 activation transfer stays on binary HTTP.
+
+## Target User Experience
+
+A client sends an OpenAI-compatible request. The Gateway or Tracker Node accepts the request, creates a Route Session, and streams token deltas when supported. The client receives live Generation Telemetry for route phase, prefill progress, generated token count, rolling tokens/sec, route health, and failure reason.
+
+If a route node drops in alpha, the request fails clearly. A retry starts a new Route Session from scratch.
+
+## Runtime Shape
+
+```text
+client request
+  -> Gateway / Tracker Node creates Route Session
+  -> Tracker selects sticky Inference Route
+  -> prefill:
+       prompt chunks move through Shards
+       each node appends local Hot KV State
+  -> decode:
+       one-step activation moves through Shards
+       each node reads/appends local Hot KV State
+       tail returns token/logits
+  -> client receives streamed token deltas where possible
+  -> Generation Telemetry continues until complete or failed
+```
+
+## Milestones
+
+| Milestone | Outcome | Issues |
+|---|---|---|
+| M1 — Session protocol proof | Stub route has stable Route Sessions, prefill/decode split, telemetry, and streaming contract | 01, 02, 03 |
+| M2 — PyTorch reference route | Distributed PyTorch decode uses local per-shard cache and stops full-prompt recompute | 04 |
+| M3 — Local GGUF performance path | Single-node GGUF backend serves through the node API and reports backend metadata | 05 |
+| M4 — Artifact plane | Model Artifact manifest supports verification, layer mapping, and node advertisement | 06 |
+| M5 — llama.cpp collaboration proof | Localhost layer-boundary prototype identifies upstreamable llama.cpp/libllama API | 07 |
+| M6 — Networked GGUF route | Multi-node GGUF route uses the resolved protocol and fails cleanly on node loss | 08 |
+| M7 — First large model | DeepSeek-V4-Flash support path is audited and converted into follow-up runtime tasks | 09 |
+
+## Acceptance Criteria
+
+- A two-node route can prefill once and decode without resending full prompt activations.
+- Decode seam payload is one token/hidden-state step after prefill.
+- Route Session telemetry is visible before first token and during decode.
+- Streaming token deltas work where the backend supports them.
+- Route-node loss produces a structured alpha failure and does not attempt unsafe repair.
+- A local GGUF model can serve via the node API.
+- A Model Artifact manifest can prove which Shards a node can serve.
+- DeepSeek-V4-Flash has a written support recommendation: PyTorch, vLLM/SGLang, llama.cpp/GGUF, or blocked.

.scratch/distributed-gguf-runtime/README.md

@@ -0,0 +1,63 @@
+# Distributed GGUF runtime — planning index
+
+Status: draft scratch package.
+
+Goal: make the node network capable of serving large, high-quality open models by distributing GGUF/model artifacts over a torrent-style swarm while executing inference over a sticky multi-node route with per-shard local KV cache.
+
+This scratch supersedes the old assumption in [ADR-0001](../../docs/adr/0001-pytorch-over-llama-cpp.md) that llama.cpp is only a single-node leaf backend. That assumption was correct for the original llama.cpp RPC shape, but the target is now different: torrent-distributed GGUF artifacts plus an explicit route/KV protocol owned by this platform, ideally developed in collaboration with upstream llama.cpp.
+
+## Artifacts
+
+| Path | Purpose |
+|---|---|
+| [architecture.md](./architecture.md) | Proposed runtime architecture, data flow, session state, and failure model |
+| [technical-challenges.md](./technical-challenges.md) | Detailed challenge/solution register with acceptance tests |
+| [decision-framework.md](./decision-framework.md) | Grilling framework for open decisions and recommended answers |
+| [research-prior-art.md](./research-prior-art.md) | Prior-art notes for Petals, exo, Distributed Llama, prima.cpp, llama.cpp, DeepSeek-V4-Flash, GLM-5.2, and Ornith |
+| [ADR-0020-distributed-gguf-runtime.md](./ADR-0020-distributed-gguf-runtime.md) | Draft decision record for the GGUF/llama.cpp distributed runtime |
+| [PRD.md](./PRD.md) | Product/runtime requirements and acceptance criteria |
+| [milestones.md](./milestones.md) | Dependency-ordered implementation milestones |
+| [issues/](./issues/) | Implementation-ready tracer-bullet issue briefs |
+
+## Decision Summary
+
+Adopt a hybrid runtime:
+
+- **Weights and artifacts**: distributed by torrent / content-addressed storage / optional CDN.
+- **Hot KV cache**: local to the node that owns the corresponding layer range.
+- **Prefix snapshots**: optionally persisted to cache servers for reuse, retry, and failover.
+- **Active route**: sticky for one request/session.
+- **Context cap**: 128K hard product limit for large models unless explicitly revised.
+- **Backends**: keep PyTorch for fast model-architecture coverage and validation; add llama.cpp/GGUF as the performance path for supported models.
+- **Client feedback**: stream token deltas when feasible; always expose Generation Telemetry.
+- **First serious target model**: DeepSeek-V4-Flash after a smaller GGUF protocol smoke test.
+
+## What We Learned
+
+- Our current full-model PyTorch path uses Transformers `generate()` and gets local KV cache.
+- Our current distributed PyTorch path disables cache and recomputes the full growing sequence per token.
+- The seam today carries hidden activations, not KV cache; at 128K this becomes impossible for serious models if repeated every decode token.
+- The missing capability is not "send KV across the network"; it is **stable per-session local KV cache per shard**.
+- GGUF distribution is solved enough at the artifact layer, but GGUF/llama.cpp needs explicit layer-boundary execution APIs for our route model.
+
+## Recommended Order
+
+See [milestones.md](./milestones.md) for the full dependency map.
+
+1. [01 — Route Session lifecycle](./issues/01-route-session-lifecycle.md)
+2. [02 — Prefill/decode binary HTTP protocol](./issues/02-prefill-decode-binary-http.md)
+3. [03 — Generation Telemetry and streaming response contract](./issues/03-generation-telemetry-and-streaming.md)
+4. [04 — PyTorch distributed KV reference route](./issues/04-pytorch-distributed-kv-reference.md)
+5. [05 — Local llama.cpp/GGUF backend](./issues/05-local-llamacpp-gguf-backend.md)
+6. [06 — Model Artifact manifest and Shard advertisement](./issues/06-model-artifact-manifest.md)
+7. [07 — llama.cpp layer-boundary prototype](./issues/07-llamacpp-layer-boundary-prototype.md)
+8. [08 — Networked distributed GGUF route](./issues/08-networked-distributed-gguf-route.md)
+9. [09 — DeepSeek-V4-Flash support audit](./issues/09-deepseek-v4-flash-support-audit.md)
+10. [10 — GLM-5.2 and Ornith follow-up support audit](./issues/10-glm52-ornith-followup-audit.md)
+
+## Open Questions
+
+- Does upstream llama.cpp already expose enough internal API for arbitrary layer-range execution and hidden-state boundary I/O, or do we need an extension?
+- Can GGUF split metadata be made layer/tensor semantic enough for torrent placement and partial loading?
+- What is the minimum protocol needed for compressed KV formats such as GLM-5.2 DSA/MLA without exposing model-specific internals to the tracker?
+- How much reliability do we need in alpha: fail request on route loss, or support route repair with KV snapshots?

.scratch/distributed-gguf-runtime/architecture.md

@@ -0,0 +1,274 @@
+# Distributed GGUF Runtime Architecture
+
+## Product Stance
+
+The platform optimizes for access to high-quality models, not lowest latency. Latency is acceptable if the user can run models that are otherwise unavailable to them. The hard context limit for the first serious distributed runtime should be **128K tokens**. Longer context usually means the product is compensating for missing task decomposition, retrieval, or workspace summarization.
+
+## Current State
+
+The current node has two materially different inference paths:
+
+- **Full local PyTorch model**: calls Hugging Face `model.generate()`, so Transformers owns autoregressive decode and local KV cache.
+- **Distributed PyTorch route**: bypasses `model.generate()`, calls individual layers with `use_cache=False`, and recomputes the full growing sequence for every generated token.
+
+Current distributed data flow:
+
+```text
+client request
+  -> head node formats prompt
+  -> for each output token:
+       head tokenizes full current text
+       head runs early layers over all tokens
+       head sends full activation [batch, sequence, hidden] to next node
+       middle nodes run their layers over all tokens
+       tail returns one decoded token string
+       head appends token to text
+```
+
+This is correct for small demos but not viable for large models. For GLM-5.2, a single 128K seam activation is roughly:
+
+```text
+128K tokens * hidden_size 6144 * 2 bytes ~= 1.5 GiB per hop
+```
+
+Sending that every output token is the bottleneck.
+
+## Target State
+
+Target distributed data flow:
+
+```text
+client request
+  -> tracker selects route and pins session
+  -> head node creates session_id
+  -> prefill:
+       prompt is chunked
+       each shard computes its layer range
+       each shard appends local KV/state for its own layers
+       activations cross only layer seams
+  -> decode loop:
+       head sends one new token / one-step hidden state
+       each shard reads local KV/state for session_id
+       each shard appends one step to local KV/state
+       only one-step activation crosses seams
+       tail returns logits/token
+```
+
+The KV cache remains local to the node that computed it. It is not sent to the next node and not read from a remote cache server during every decode step.
+
+## Client Feedback
+
+Streaming responses are desirable when the backend and client transport support them. The product should stream token deltas when possible, and it must always provide realtime Generation Telemetry while the route is working.
+
+The fallback behavior is a non-streaming final answer plus live telemetry. That fallback is acceptable for early route proofs or models/backends that cannot expose clean token deltas yet, but the preferred client experience is streamed output plus telemetry.
+
+Minimum client-visible telemetry:
+
+- route/session accepted
+- selected model and quantization
+- prefill phase started/completed
+- decode phase started
+- generated token count
+- rolling tokens per second
+- route health or retry/failure reason
+- estimated billing units when available
+
+Implementation options:
+
+- Server-Sent Events or WebSocket for realtime progress
+- polling endpoint for simple clients
+- OpenAI-compatible streaming for clients that require token deltas
+
+This means "no token streaming" is acceptable only as a fallback. "Silent wait for minutes" is not acceptable.
+
+## Artifact Plane
+
+Artifact distribution is separate from execution.
+
+```text
+model publisher
+  -> produces model manifest
+  -> creates GGUF / safetensors / tokenizer artifacts
+  -> content-addresses every file/chunk
+  -> publishes torrent/magnet + HTTP fallback metadata
+
+node
+  -> chooses model/layer range
+  -> downloads needed files/chunks
+  -> verifies hash
+  -> advertises availability to tracker
+```
+
+Required manifest fields:
+
+- model id and version
+- upstream source repo and revision
+- license
+- architecture name
+- tokenizer files and hashes
+- quantization
+- tensor-to-layer map
+- file/chunk hashes
+- optional GGUF split files
+- supported runtime backends
+- context cap
+- KV/cache format descriptor
+
+## Execution Plane
+
+The tracker selects routes using layer coverage and observed performance:
+
+```text
+route = [
+  head node: embeddings + layers 0..k
+  middle nodes: contiguous layer ranges
+  tail node: final layers + norm + lm_head
+]
+```
+
+Route selection inputs:
+
+- model id/version/quantization
+- layer coverage
+- node hardware
+- measured prefill throughput
+- measured decode throughput
+- queue depth
+- latency to neighboring nodes
+- cache warmth for the requested prefix/session
+- reliability/reputation
+
+The route is sticky for the request/session. A new route means either a fresh prefill or restoring compatible KV snapshots.
+
+## KV Cache Ownership
+
+KV/state ownership is by layer range:
+
+```text
+session_id = request scoped id
+node A owns layers 0..15 KV for session_id
+node B owns layers 16..31 KV for session_id
+node C owns layers 32..77 KV for session_id
+```
+
+The tracker does not own hot KV. It may know which nodes hold active KV for session accounting and failure handling.
+
+Cache servers may store:
+
+- prompt-prefix snapshots
+- session checkpoints for retry
+- cold reusable context blocks
+- audit samples
+
+Cache servers must not be in the per-token hot loop unless colocated with the compute node.
+
+## 128K KV Budget
+
+GLM-5.2 compressed DSA/MLA-style estimate from config:
+
+```text
+layers = 78
+kv_lora_rank = 512
+qk_rope_head_dim = 64
+dtype = bf16 = 2 bytes
+context = 128K
+
+per_token ~= 78 * (512 + 64) * 2 = 89,856 bytes ~= 87.75 KiB
+128K total ~= 10.7 GiB
+per layer ~= 137 MiB
+```
+
+This is feasible when sharded:
+
+| Layer count | Approx active KV at 128K |
+|---:|---:|
+| 1 | 137 MiB |
+| 10 | 1.37 GiB |
+| 20 | 2.75 GiB |
+| 78 | 10.7 GiB |
+
+The exact runtime value depends on implementation and cache quantization, but the order of magnitude is acceptable.
+
+## Protocol Sketch
+
+### Prefill
+
+```http
+POST /v1/sessions/{session_id}/prefill
+Content-Type: application/octet-stream
+X-Meshnet-Model: zai-org/GLM-5.2
+X-Meshnet-Route-Id: ...
+X-Meshnet-Token-Range: 0-2047
+X-Meshnet-Shape: 1,2048,6144
+X-Meshnet-Dtype: bfloat16
+
+<activation bytes>
+```
+
+The receiver:
+
+- validates route/session
+- runs assigned layer range for that chunk
+- appends local KV/state
+- forwards resulting activation to next hop
+
+### Decode
+
+```http
+POST /v1/sessions/{session_id}/decode-step
+Content-Type: application/octet-stream
+X-Meshnet-Model: zai-org/GLM-5.2
+X-Meshnet-Position: 131072
+X-Meshnet-Shape: 1,1,6144
+X-Meshnet-Dtype: bfloat16
+
+<one-step activation bytes>
+```
+
+The receiver:
+
+- loads local KV/state by `session_id`
+- runs one decode step for assigned layers
+- appends one token position to local KV/state
+- forwards one-step activation
+
+## GGUF / llama.cpp Integration
+
+The target llama.cpp integration needs more than `llama-server`.
+
+Required capabilities:
+
+- load full GGUF locally for immediate single-node performance
+- optionally load only selected tensors/layers
+- execute a layer range against inbound hidden states
+- expose outbound hidden states at a boundary
+- own per-session KV/state for only the loaded layer range
+- support prefill chunks and decode-step calls
+- expose model-specific cache metadata for DSA/MLA without requiring the tracker to understand tensor internals
+
+If llama.cpp cannot expose these as stable APIs today, the collaboration target is an upstream extension rather than a long-lived fork.
+
+## Failure Model
+
+Alpha behavior:
+
+- Route node drops during prefill: fail request and retry from scratch.
+- Route node drops during decode: fail request unless a recent KV snapshot exists.
+- Tracker restart: active sessions may be lost; completed billing records persist.
+- Node restart: local hot KV is lost.
+
+Later behavior:
+
+- periodic KV snapshots for long sessions
+- prefix cache reuse across requests
+- route repair when a semantically equivalent node has the same model/layer range and compatible cache snapshot
+
+## Security And Trust
+
+Activation/KV data can reveal user prompts. Public volunteer routes are not private. For sensitive workloads:
+
+- use private swarms
+- allow paid trusted nodes
+- encrypt transport
+- avoid storing hot KV on untrusted shared cache servers
+- sample outputs for fraud/audit as already planned in alpha hardening

.scratch/distributed-gguf-runtime/decision-framework.md

@@ -0,0 +1,268 @@
+# Distributed GGUF Decision Framework
+
+This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
+
+## Core Vocabulary
+
+Use the existing domain terms this way:
+
+- **Shard**: contiguous transformer layer range. This is the compute, routing, cache, and reward unit.
+- **Shard Swarm**: storage/download group for artifacts needed by a shard.
+- **Inference Route**: ordered node sequence that covers all layers for one request.
+- **Route Session**: one active request bound to one inference route and stable session id.
+- **Hot KV State**: live per-shard cache held by the route node during a route session.
+- **Prefix Snapshot**: persisted route-session state used for reuse or failover, not the hot decode path.
+- **Artifact Manifest**: canonical mapping from model artifacts to semantic model parts and runtime support.
+- **Generation Telemetry**: realtime progress for a route session, including phase and tokens/sec, independent of whether token deltas are streamed.
+
+## The Five Planes
+
+### 1. Control Plane
+
+Owner: Tracker.
+
+Responsibilities:
+
+- node registry
+- coverage map
+- route selection
+- rebalance directives
+- route-session creation
+- health and telemetry
+- client-visible Generation Telemetry
+- billing/audit records
+
+Must not do:
+
+- serve hot KV during every token
+- become the only place model artifacts can be fetched
+
+### 2. Artifact Plane
+
+Owner: Shard Swarms, local node storage, optional CDN/bootstrap mirrors.
+
+Responsibilities:
+
+- GGUF/safetensors/tokenizer download
+- content-addressed verification
+- local artifact inventory
+- artifact-to-layer mapping
+- cache eviction
+
+Must not do:
+
+- define execution order by file split alone
+- imply that a downloaded file chunk equals a Shard
+
+### 3. Execution Plane
+
+Owner: active Inference Route.
+
+Responsibilities:
+
+- chunked prefill
+- one-step decode
+- hidden-state transfer across activation seams
+- start-layer handling for overlapping shards
+- backpressure
+
+Must not do:
+
+- resend full context activations during decode
+- require cross-node tensor parallel all-reduce for public v1
+
+### 4. Session State Plane
+
+Owner: route nodes for hot KV; cache servers only for snapshots.
+
+Responsibilities:
+
+- per-shard local KV ownership
+- cache allocation and eviction
+- cache ABI compatibility
+- session close/release
+- optional prefix snapshots
+
+Must not do:
+
+- centralize hot KV in a remote service
+- let a replacement node continue from incompatible state
+
+### 5. Economics And Trust Plane
+
+Owner: tracker plus settlement/validation components.
+
+Responsibilities:
+
+- distinguish storage/seeding work from inference work
+- account for prefill and decode separately
+- record route participation
+- sample validation events
+- slash proven fraud
+
+Must not do:
+
+- pay a node for merely holding files as if it generated tokens
+- hide public-swarm privacy limits from clients
+
+## Hard Invariants
+
+These are the framework rules unless we deliberately write a new ADR:
+
+1. Public-network Shards are contiguous layer ranges.
+2. Hot KV State is local to the node serving that Shard in that Route Session.
+3. Artifact distribution and route execution are separate systems.
+4. Decode seam payload must be `O(hidden_size)`.
+5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
+6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
+7. Model/backend-specific cache internals stay behind backend capability reports.
+8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
+9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
+
+## Resolved Gates
+
+### Gate 1: Public Shard Semantics
+
+Decision: public-network Shards are contiguous transformer layer ranges. Tensor-parallel or ring-style execution is allowed only inside one trusted node, one colocated pod, or a future composite node abstraction.
+
+Rationale:
+
+- Layer ranges match the existing `Shard`, `Coverage Map`, `Inference Route`, billing, and fraud vocabulary.
+- Public volunteer nodes should not require cross-node all-reduce or tight per-layer synchronization in v1.
+- Existing projects such as prima.cpp and Distributed Llama can still inform local-cluster/backend execution without becoming the public routing primitive.
+
+Consequences:
+
+- Artifact Manifests must map files/tensors to semantic layer ranges.
+- Route selection remains ordered layer coverage.
+- Rewards can be attributed to layer-range work.
+- Hot KV State is naturally owned by the node serving that layer range for the Route Session.
+
+### Gate 2: Hot KV Strategy
+
+Decision: v1 rejects centralized hot KV. Hot KV State is local to the node serving the relevant Shard in the active Route Session. Cache servers may store Prefix Snapshots for reuse, retry, or failover, but they are not in the per-token decode path.
+
+Rationale:
+
+- Decode is the tight loop; adding remote cache I/O there makes latency and bandwidth worse at the worst point.
+- Local KV naturally follows layer-range Shard ownership.
+- Centralized hot KV increases privacy exposure and creates consistency problems.
+- Prefix Snapshots preserve the useful part of central storage without making it mandatory for every generated token.
+
+Consequences:
+
+- Route Session must be sticky.
+- Failover is limited in alpha unless a compatible Prefix Snapshot exists.
+- Cache servers are optimization infrastructure, not required runtime infrastructure.
+- Route repair requires compatible model revision, layer range, backend cache ABI, and snapshot position.
+
+### Gate 3: First Runtime Proof
+
+Decision: prove distributed Route Session and Hot KV State semantics in the existing PyTorch route before modifying llama.cpp/GGUF.
+
+Rationale:
+
+- PyTorch exposes model internals and cache objects more directly, so it is the fastest way to validate the distributed protocol.
+- The current distributed PyTorch route already has the right high-level shape but disables cache and recomputes full prompts.
+- Fixing that path gives us a reference implementation for correctness tests, telemetry, session lifecycle, and wire protocol behavior.
+- llama.cpp/GGUF should receive a clear target ABI rather than becoming both the protocol experiment and the performance backend at once.
+
+Consequences:
+
+- Issue 02 precedes issue 05.
+- llama.cpp collaboration has a concrete target ABI.
+- The PyTorch route remains the architecture-coverage/reference backend even after GGUF becomes the preferred performance path.
+- The first success metric is eliminating full-prompt recompute in distributed decode.
+
+### Gate 3A: Client Feedback During Latency
+
+Decision: streaming responses are preferred when feasible, and realtime Generation Telemetry is required regardless of streaming support.
+
+Rationale:
+
+- The product optimizes for access to large capable models, so some latency is acceptable.
+- Users still need confidence that the route is alive and roughly how fast it is generating.
+- Streaming token deltas give the best user experience when the backend exposes them cleanly.
+- Tokens/sec remains useful during prefill, queueing, and any backend that cannot stream token deltas.
+
+Consequences:
+
+- The gateway should stream token deltas through an OpenAI-compatible response when possible.
+- The gateway must expose progress through SSE, WebSocket, or polling.
+- The final answer can be delivered after completion only as a fallback.
+- Telemetry must include route phase, generated token count, and rolling tokens/sec.
+- Non-streaming clients still need realtime telemetry.
+
+### Gate 4: llama.cpp Collaboration Shape
+
+Decision: target upstreamable `libllama`/ggml hooks instead of planning around a permanent fork.
+
+Rationale:
+
+- llama.cpp changes quickly across model support, quantization, kernels, and hardware backends.
+- A permanent fork would become expensive to maintain and would lag upstream improvements.
+- A short-lived prototype branch is acceptable if it proves the API and makes upstream collaboration concrete.
+- Keeping tracker/routing logic outside llama.cpp makes the upstream ask smaller and cleaner.
+
+Consequences:
+
+- Need a minimal reproducible localhost demo before asking upstream to carry the design.
+- Need to separate "what llama.cpp should expose" from "what our tracker does".
+- Desired upstream surface is layer-range execution, hidden-state boundary I/O, partial loading/introspection, and per-session KV ownership.
+- If upstream rejects the shape, we revisit whether to carry a narrow adapter fork or keep GGUF distributed execution as experimental.
+
+### Gate 5: First Model Target
+
+Decision: use a two-tier model target. Use a small, boring, llama.cpp-supported GGUF model for the first protocol smoke test. Use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. Keep GLM-5.2 and Ornith as later support audits.
+
+Rationale:
+
+- The first protocol proof should isolate route/session/KV bugs from model-architecture bugs.
+- DeepSeek-V4-Flash is a strong first serious target because it is much smaller than 1.6T-class models while still being large enough to validate the product thesis.
+- DeepSeek-V4-Flash still has architecture-specific risks, so it should not be the first smoke test.
+- GLM-5.2 and Ornith remain valuable targets, but they add DSA/MLA/hybrid attention uncertainty.
+
+Consequences:
+
+- 128K cache accounting can be modeled now.
+- The first "real" target-model audit is DeepSeek-V4-Flash support in PyTorch, vLLM/SGLang, and any available GGUF/llama.cpp quantization path.
+- Production support waits for backend capability reports and exact cache ABI support.
+
+### Gate 6: Failure Semantics
+
+Decision: alpha fails Route Sessions on route-node loss instead of attempting automatic route repair.
+
+Rationale:
+
+- Route repair requires compatible Prefix Snapshots, cache ABI checks, replacement-node selection, billing correction, and client stream/error recovery.
+- Local Hot KV State means a replacement node cannot continue unless it has compatible state at the same position.
+- Fail-fast keeps the first implementation correct while the session/KV protocol is still being proven.
+
+Consequences:
+
+- Better observability and explicit errors are required.
+- Snapshotting becomes a later feature, not a blocker for first inference.
+- Generation Telemetry must report the last known phase and failure reason.
+- Client or gateway retry starts a new Route Session from scratch.
+
+### Gate 7: Transport
+
+Decision: keep binary HTTP for v1 activation transfer instead of jumping immediately to QUIC, WebRTC, or a custom transport.
+
+Rationale:
+
+- ADR-0008 already defines binary activation bodies with HTTP headers.
+- HTTP keeps the first implementation debuggable with the existing server stack and tooling.
+- The core risk is route/session/KV correctness, not transport optimization.
+- QUIC/WebRTC can be introduced later behind the same activation protocol once semantics are proven.
+
+Consequences:
+
+- Focus benchmark work on payload shape, chunking, and cache behavior first.
+- QUIC/WebRTC can be introduced as an optimization behind the same activation protocol.
+- v1 implementation can reuse the current HTTP routing, relay, and observability infrastructure.
+- Transport abstraction should be kept narrow enough that HTTP can be replaced later without changing backend cache semantics.
+
+## Grilling Progress
+
+Gates 1, 2, 3, 3A, 4, 5, 6, and 7 are resolved. The remaining work is to convert the resolved framework into implementation-ready issue briefs and prototype milestones.

.scratch/distributed-gguf-runtime/issues/01-route-session-lifecycle.md

@@ -0,0 +1,19 @@
+# 01 — Route Session lifecycle
+
+Status: ready-for-agent
+
+## What to build
+
+Add the narrowest end-to-end Route Session lifecycle that can be used by distributed inference routes: create a session, bind it to a selected Inference Route, expose status, and close it cleanly. This slice does not need real model cache yet; it proves stable session identity across the control plane and activation plane.
+
+## Acceptance criteria
+
+- [ ] A request can create a Route Session with a stable `session_id`, `route_id`, model preset, backend id, and route membership.
+- [ ] Every downstream activation request carries the same session identity and fails clearly if the session or route id does not match.
+- [ ] Session status reports phase, route nodes, model preset, backend id, created time, and last activity time.
+- [ ] Closing a session releases all registered per-session state.
+- [ ] Tests cover create, status, close, stale-session rejection, and wrong-route rejection.
+
+## Blocked by
+
+None - can start immediately.

.scratch/distributed-gguf-runtime/issues/02-prefill-decode-binary-http.md

@@ -0,0 +1,20 @@
+# 02 — Prefill/decode binary HTTP protocol
+
+Status: ready-for-agent
+
+## What to build
+
+Split the activation protocol into explicit prefill and decode-step calls using the existing binary HTTP direction from ADR-0008. The completed slice should work against a stub backend so payload shape, route/session headers, relay preservation, and failure behavior are testable before real KV cache work begins.
+
+## Acceptance criteria
+
+- [ ] Prefill accepts chunked binary activations with route/session metadata and forwards them through the selected route.
+- [ ] Decode-step accepts a one-step binary activation and forwards a one-step activation through the selected route.
+- [ ] Decode-step payload size is independent of prompt length in protocol tests.
+- [ ] Relay forwarding preserves route/session headers, shape, dtype, position, and wire version.
+- [ ] Legacy `/forward` either remains as a compatibility wrapper or fails with a clear wire-version error.
+- [ ] Tests cover prefill chunking, decode-step shape validation, relay preservation, and malformed header rejection.
+
+## Blocked by
+
+- 01 — Route Session lifecycle.

.scratch/distributed-gguf-runtime/issues/03-generation-telemetry-and-streaming.md

@@ -0,0 +1,21 @@
+# 03 — Generation Telemetry and streaming response contract
+
+Status: ready-for-agent
+
+## What to build
+
+Expose realtime Generation Telemetry for active Route Sessions and stream token deltas when the serving path can produce them. This slice should make long distributed requests observable before real large-model work begins.
+
+## Acceptance criteria
+
+- [ ] A client can observe route-session phase changes: queued, loading, prefill, decode, finalizing, completed, failed.
+- [ ] Telemetry includes prefill progress, generated token count, rolling tokens/sec, average tokens/sec, active route nodes, and failure reason.
+- [ ] Telemetry is available before the first output token.
+- [ ] A streaming response can include token deltas while telemetry remains available.
+- [ ] A non-streaming fallback still exposes telemetry until final answer or failure.
+- [ ] Route-node failure reports the last known phase and reason.
+- [ ] Tests cover telemetry updates, streaming token deltas, non-streaming fallback, and structured failure closeout.
+
+## Blocked by
+
+- 01 — Route Session lifecycle.

.scratch/distributed-gguf-runtime/issues/04-pytorch-distributed-kv-reference.md

@@ -0,0 +1,23 @@
+# 04 — PyTorch distributed KV reference route
+
+Status: ready-for-agent
+
+## What to build
+
+Fix the existing distributed PyTorch route so it uses the Route Session and prefill/decode protocol to keep Hot KV State local to each Shard node. The visible behavior is that prefill processes the prompt once, and decode no longer recomputes or resends the full growing prompt for every token.
+
+## Acceptance criteria
+
+- [ ] Distributed PyTorch prefill stores per-session cache/state on each Shard node.
+- [ ] Distributed PyTorch decode-step reads and appends local per-shard cache/state.
+- [ ] Decode activation seam payload is one token/hidden-state step after prefill.
+- [ ] The old full-growing-prompt decode loop is not used for models that support the reference cache path.
+- [ ] Unsupported model/cache APIs fail with an explicit backend capability error.
+- [ ] Session close or TTL cleanup releases per-shard cache.
+- [ ] Regression tests prove decode does not call the full prompt encoder for every generated token.
+
+## Blocked by
+
+- 01 — Route Session lifecycle.
+- 02 — Prefill/decode binary HTTP protocol.
+- 03 — Generation Telemetry and streaming response contract.

.scratch/distributed-gguf-runtime/issues/05-local-llamacpp-gguf-backend.md

@@ -0,0 +1,20 @@
+# 05 — Local llama.cpp/GGUF backend
+
+Status: ready-for-agent
+
+## What to build
+
+Add a local full-model GGUF backend so a node that can hold a GGUF model can serve it through the existing node API. This is the immediate CPU-performance path and the baseline for later distributed llama.cpp work.
+
+## Acceptance criteria
+
+- [ ] A node can start with backend `llama.cpp` or `gguf` for a local full-model GGUF artifact.
+- [ ] The node can answer an OpenAI-compatible chat completion through the existing API.
+- [ ] Startup and registration clearly report backend, quantization/artifact metadata, context cap, and local model path.
+- [ ] The PyTorch backend remains unchanged and selectable.
+- [ ] A smoke test or script validates backend wiring with a small GGUF model or a stubbed llama.cpp process.
+- [ ] A benchmark command can compare local PyTorch CPU and local GGUF CPU for the same small supported model when both are available.
+
+## Blocked by
+
+None - can start immediately.

.scratch/distributed-gguf-runtime/issues/06-model-artifact-manifest.md

@@ -0,0 +1,20 @@
+# 06 — Model Artifact manifest and Shard advertisement
+
+Status: ready-for-agent
+
+## What to build
+
+Introduce a Model Artifact manifest that separates storage distribution from route execution. A node should be able to verify local model files, determine which Shards it can serve, and advertise artifact/layer availability to the Tracker without contacting Hugging Face at request time.
+
+## Acceptance criteria
+
+- [ ] Manifest records model preset, upstream revision, license, backend support, quantization, context cap, tokenizer artifacts, file hashes, piece hashes, and tensor/layer mapping where available.
+- [ ] A node can verify local artifacts against the manifest and reject corrupt or incomplete artifacts.
+- [ ] A node can derive advertised Shard ranges from the manifest and local files.
+- [ ] Tracker registration can include artifact id, backend id, Shard range, and verification status.
+- [ ] Tracker coverage can distinguish model-layer coverage from artifact availability.
+- [ ] Tests cover valid manifest registration, corrupt artifact rejection, and missing layer/tensor metadata.
+
+## Blocked by
+
+- 01 — Route Session lifecycle.

.scratch/distributed-gguf-runtime/issues/07-llamacpp-layer-boundary-prototype.md

@@ -0,0 +1,25 @@
+# 07 — llama.cpp layer-boundary prototype
+
+Status: ready-for-human
+
+## What to build
+
+Build a local prototype that proves whether llama.cpp/libllama can support the platform's distributed execution contract: execute a selected layer range, accept inbound hidden states, emit outbound hidden states, and own per-session cache for only the loaded/served range.
+
+This is the collaboration package for upstream llama.cpp. The target is an upstreamable API shape, not a permanent fork.
+
+## Acceptance criteria
+
+- [ ] A small llama.cpp-supported GGUF model can be split into a two-process localhost head/tail prototype.
+- [ ] The head process runs embeddings and early layers, then emits hidden states at an Activation Seam.
+- [ ] The tail process accepts hidden states, runs later layers plus output head, and produces logits/tokens comparable to single-process execution.
+- [ ] Prefill is performed once and decode-step seam payload is one hidden-state step per generated token.
+- [ ] Each process owns only its own per-session cache/state.
+- [ ] The prototype records the minimum upstream API needed for layer-range execution, hidden-state I/O, partial loading/introspection, and per-session KV ownership.
+- [ ] If upstream support is unavailable, the issue ends with a concrete recommendation: upstream proposal, narrow adapter fork, or keep GGUF distribution local-only for now.
+
+## Blocked by
+
+- 02 — Prefill/decode binary HTTP protocol.
+- 05 — Local llama.cpp/GGUF backend.
+- 06 — Model Artifact manifest and Shard advertisement.

.scratch/distributed-gguf-runtime/issues/08-networked-distributed-gguf-route.md

@@ -0,0 +1,24 @@
+# 08 — Networked distributed GGUF route
+
+Status: pending
+
+## What to build
+
+Run a GGUF-backed model over a real multi-node Inference Route using the resolved Route Session, binary HTTP prefill/decode protocol, local Hot KV State, Generation Telemetry, and alpha fail-fast behavior.
+
+## Acceptance criteria
+
+- [ ] Two machines can form one GGUF-backed Inference Route over contiguous Shards.
+- [ ] Prefill builds local per-shard cache/state and decode-step uses one-step seam payloads.
+- [ ] The client receives streamed token deltas when supported by the GGUF path.
+- [ ] The client receives Generation Telemetry for phase, generated tokens, tokens/sec, route health, and failure reason.
+- [ ] Route-node loss fails the Route Session cleanly; no automatic repair is attempted in alpha.
+- [ ] Tracker metrics show prefill tokens/sec, decode tokens/sec, seam latency, queue depth, and cache memory by node.
+- [ ] Billing/audit records identify route membership and layer/token work for the completed or failed session.
+
+## Blocked by
+
+- 03 — Generation Telemetry and streaming response contract.
+- 04 — PyTorch distributed KV reference route.
+- 06 — Model Artifact manifest and Shard advertisement.
+- 07 — llama.cpp layer-boundary prototype.

.scratch/distributed-gguf-runtime/issues/09-deepseek-v4-flash-support-audit.md

@@ -0,0 +1,21 @@
+# 09 — DeepSeek-V4-Flash support audit
+
+Status: ready-for-agent
+
+## What to build
+
+Audit `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target after the small GGUF protocol smoke test. The output is a compatibility matrix and a recommended runtime path, not full production support.
+
+## Acceptance criteria
+
+- [ ] Verify current PyTorch/Transformers load and generation semantics for DeepSeek-V4-Flash from primary model documentation.
+- [ ] Verify vLLM and SGLang support status from primary runtime documentation or release notes.
+- [ ] Verify whether a GGUF/llama.cpp quantization path exists or would need upstream work.
+- [ ] Estimate artifact size, active parameter behavior, and 128K cache memory by Shard range.
+- [ ] Identify required backend capability flags for the Tracker.
+- [ ] Produce a compatibility matrix: PyTorch, vLLM, SGLang, llama.cpp/GGUF.
+- [ ] End with one recommendation: first runtime path, blocked pending upstream, or defer.
+
+## Blocked by
+
+None - can start immediately.

.scratch/distributed-gguf-runtime/issues/10-glm52-ornith-followup-audit.md

@@ -0,0 +1,20 @@
+# 10 — GLM-5.2 and Ornith follow-up support audit
+
+Status: pending
+
+## What to build
+
+Audit GLM-5.2 and Ornith after the smaller protocol smoke path and DeepSeek-V4-Flash audit. The output is a follow-up compatibility matrix focused on architecture/runtime blockers: DSA/MLA, hybrid attention, cache accounting, and GGUF/llama.cpp support.
+
+## Acceptance criteria
+
+- [ ] Verify GLM-5.2 PyTorch/Transformers serving requirements and cache semantics from primary model documentation.
+- [ ] Verify llama.cpp/GGUF support status for `glm_moe_dsa` or equivalent architecture support.
+- [ ] Verify Ornith/Qwen3.5-MoE and hybrid attention support status in the candidate runtimes.
+- [ ] Estimate artifact size and 128K cache memory by Shard range for each model.
+- [ ] Identify smallest quality-preserving quantization worth testing.
+- [ ] Convert each runtime blocker into a follow-up issue or upstream collaboration note.
+
+## Blocked by
+
+- 09 — DeepSeek-V4-Flash support audit.

.scratch/distributed-gguf-runtime/milestones.md

@@ -0,0 +1,32 @@
+# Distributed GGUF Runtime Milestones
+
+## Proposed Breakdown
+
+| Order | Issue | Title | Blocked by | User-visible proof |
+|---:|---|---|---|---|
+| 1 | [01](./issues/01-route-session-lifecycle.md) | Route Session lifecycle | None | Stable route/session status and cleanup |
+| 2 | [02](./issues/02-prefill-decode-binary-http.md) | Prefill/decode binary HTTP protocol | 01 | Stub route proves prefill chunks and one-step decode payloads |
+| 3 | [03](./issues/03-generation-telemetry-and-streaming.md) | Generation Telemetry and streaming response contract | 01 | Client sees route progress and streamed deltas when available |
+| 4 | [04](./issues/04-pytorch-distributed-kv-reference.md) | PyTorch distributed KV reference route | 01, 02, 03 | Distributed PyTorch decode stops full-prompt recompute |
+| 5 | [05](./issues/05-local-llamacpp-gguf-backend.md) | Local llama.cpp/GGUF backend | None | Local GGUF model serves through node API |
+| 6 | [06](./issues/06-model-artifact-manifest.md) | Model Artifact manifest and Shard advertisement | 01 | Node verifies artifacts and advertises serveable Shards |
+| 7 | [07](./issues/07-llamacpp-layer-boundary-prototype.md) | llama.cpp layer-boundary prototype | 02, 05, 06 | Local two-process GGUF route identifies upstream API |
+| 8 | [08](./issues/08-networked-distributed-gguf-route.md) | Networked distributed GGUF route | 03, 04, 06, 07 | Two machines serve one GGUF route with telemetry |
+| 9 | [09](./issues/09-deepseek-v4-flash-support-audit.md) | DeepSeek-V4-Flash support audit | None | Runtime recommendation for first serious large model |
+| 10 | [10](./issues/10-glm52-ornith-followup-audit.md) | GLM-5.2 and Ornith follow-up support audit | 09 | Follow-up compatibility matrix and upstream blockers |
+
+## First Three To Implement
+
+1. **01 — Route Session lifecycle**: makes every later cache, telemetry, and route decision concrete.
+2. **02 — Prefill/decode binary HTTP protocol**: proves the payload shape and route/session headers before model internals.
+3. **03 — Generation Telemetry and streaming response contract**: gives every later long-running route a visible user experience and failure surface.
+
+## Parallel Work
+
+- **05 — Local llama.cpp/GGUF backend** can run in parallel with 01–03 because it is a full-model local backend.
+- **09 — DeepSeek-V4-Flash support audit** can run in parallel because it is research/compatibility work.
+
+## Human-Gated Work
+
+- **07 — llama.cpp layer-boundary prototype** is the collaboration point with Georgi/upstream llama.cpp.
+- **08 — Networked distributed GGUF route** should wait until the PyTorch reference route proves the cache/session contract.

.scratch/distributed-gguf-runtime/research-prior-art.md

@@ -0,0 +1,231 @@
+# Prior Art: Distributed Large-Model Inference
+
+This note captures what existing projects appear to solve and what remains specific to this platform.
+
+## Petals
+
+Source: <https://github.com/bigscience-workshop/petals>
+
+Petals is the closest conceptual match for public volunteer inference. Its README describes running large models "BitTorrent-style", where a user loads a model through a `transformers`-like API and connects to a distributed network that hosts model layers. It explicitly supports seeing hidden states and using PyTorch/Transformers flexibility. The public README also notes privacy limitations: data is processed by other people in the public swarm, and sensitive use should run in a private swarm.
+
+What it solves:
+
+- public swarm of layer-serving peers
+- hidden-state exposure
+- route-like execution over model blocks
+- private swarm option
+- PyTorch/Transformers integration
+
+What it does not directly solve for us:
+
+- GGUF/llama.cpp artifact path
+- torrent artifact storage tied to node rewards
+- our billing/fraud/reputation model
+- our OpenAI-compatible tracker/node route model
+- a production path for GLM-5.2/DSA GGUF
+
+Design import:
+
+- Keep a PyTorch route as a reference implementation and validation harness.
+- Preserve hidden-state seam semantics.
+- Treat privacy as an explicit swarm property.
+
+## exo
+
+Source: <https://github.com/exo-explore/exo>
+
+exo connects local devices into an AI cluster. Its README emphasizes automatic device discovery, topology-aware model splitting, tensor parallelism, MLX support, RDMA over Thunderbolt, and multiple API compatibilities. It is strongest for colocated owned devices, especially Apple Silicon / MLX clusters.
+
+What it solves:
+
+- automatic local cluster discovery
+- topology-aware splitting
+- tensor parallelism
+- OpenAI/Ollama/Claude API compatibility
+- model placement previews
+- cluster dashboard
+
+What it does not directly solve for us:
+
+- untrusted internet volunteer network
+- reward, fraud, and reputation
+- torrent artifact distribution
+- Linux GPU maturity is stated as still under development in the README
+- GGUF/llama.cpp route protocol
+
+Design import:
+
+- Add placement previews before committing a route.
+- Model prefill/decode separately in benchmarks.
+- Use topology-aware routing, not just layer coverage.
+
+## Distributed Llama / dllama
+
+Source: <https://github.com/b4rtaz/distributed-llama>
+
+Distributed Llama connects home devices into a cluster for CPU/GPU inference. Its README describes tensor parallelism, Ethernet synchronization, Linux/macOS/Windows support, ARM and x86 AVX2 optimization, and a root/worker architecture. The root node loads the model and forwards weights/state to workers. Known limitations include only `2^n` nodes and a maximum node count equal to the model's number of KV heads.
+
+What it solves:
+
+- practical cross-platform home-device cluster
+- tensor-parallel synchronization
+- root/worker process model
+- custom model format and conversion path
+
+What it does not directly solve for us:
+
+- arbitrary volunteer joins/leaves
+- independent shard ownership from local/torrent disk
+- layer-range routing with tracker-managed marketplace
+- public network fraud/billing
+- GGUF as the native published artifact
+
+Design import:
+
+- KV-head constraints matter for tensor-parallel designs.
+- A root node that distributes weights is unacceptable for our torrent-first marketplace; nodes must independently acquire artifacts.
+
+## prima.cpp
+
+Sources:
+
+- <https://github.com/Lizonghang/prima.cpp>
+- <https://arxiv.org/abs/2504.08791>
+
+prima.cpp is a distributed llama.cpp implementation for low-resource home clusters. The README highlights mmap-based low memory pressure, piped-ring parallelism with prefetching, heterogeneity-aware workload distribution, automatic weak-device removal, GGUF quantization support, speculative decoding, dynamic batching, and support for Llama/Qwen/DeepSeek-class models. Its commands require each rank to point at the same GGUF file, and the README shows ring communication across ranks.
+
+What it solves:
+
+- llama.cpp-derived GGUF distributed execution
+- heterogeneous device scheduling
+- low memory pressure via mmap/page cache behavior
+- disk prefetch as a first-class performance dimension
+- ring communication for home clusters
+- GGUF quantization support
+
+What it does not directly solve for us:
+
+- public volunteer marketplace
+- torrent artifact discovery and seeding economics
+- tracker-injected route over internet/NAT/relay
+- per-node independent shard selection and rewards
+- GLM-5.2 support is not established from the README
+
+Design import:
+
+- Study mmap and prefetching before inventing partial GGUF loading.
+- Include disk speed and memory pressure in routing.
+- Heterogeneity-aware scheduling is mandatory.
+- Weak nodes should be excluded from a route if they slow the whole decode path.
+
+## llama.cpp / GGUF
+
+Sources:
+
+- <https://github.com/ggml-org/llama.cpp>
+- <https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/gguf-split/README.md>
+- <https://raw.githubusercontent.com/ggml-org/llama.cpp/master/ggml/CMakeLists.txt>
+
+llama.cpp is the performance runtime we want for GGUF. It supports local GGUF loading, many CPU/GPU backends, OpenAI-compatible serving, quantization, and `gguf-split` can split or merge GGUF files by max size or tensor count. The ggml build options include many hardware backends and RPC support.
+
+What it solves:
+
+- mature CPU/GPU local inference
+- GGUF ecosystem
+- quantized weights
+- local OpenAI-compatible server
+- split/merge tooling for artifact distribution
+
+What it does not solve by itself:
+
+- torrent distribution and reward model
+- per-session distributed route over arbitrary nodes
+- public-node trust/fraud model
+- stable API for arbitrary layer-boundary hidden-state I/O, if not already exposed
+
+Design import:
+
+- Use llama.cpp locally before attempting distributed GGUF.
+- Collaborate upstream on layer-range execution and KV ownership APIs.
+- Keep GGUF split for artifacts, not as the only execution-shard definition.
+
+## GLM-5.2
+
+Sources:
+
+- <https://huggingface.co/zai-org/GLM-5.2>
+- <https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json>
+
+GLM-5.2 is MIT licensed, 753B parameters, and advertises a 1M-token context. The config uses `glm_moe_dsa`, 78 layers, `hidden_size=6144`, `kv_lora_rank=512`, `qk_head_dim=256`, `qk_nope_head_dim=192`, `qk_rope_head_dim=64`, `v_head_dim=256`, and `max_position_embeddings=1048576`. The model card states IndexShare reduces per-token FLOPs at 1M context.
+
+Design import:
+
+- DSA/MLA-style compressed KV makes 128K feasible.
+- Tracker should not need to understand DSA internals; backend should expose cache budget and compatibility metadata.
+- GLM-5.2 is a later target after generic distributed KV works.
+
+## DeepSeek-V4-Flash
+
+Sources:
+
+- <https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash>
+- <https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/main/config.json>
+
+DeepSeek-V4-Flash is MIT licensed and published as `deepseek-ai/DeepSeek-V4-Flash` on Hugging Face. The model card describes DeepSeek-V4-Flash as a 284B-parameter MoE model with 13B activated parameters and a 1M-token context. Hugging Face tags it as `deepseek_v4`, Transformers, Safetensors, and FP8. The repository lists 46 safetensor shards and around 160 GB total size.
+
+Config highlights:
+
+- `model_type=deepseek_v4`
+- `hidden_size=4096`
+- `num_hidden_layers=43`
+- `num_attention_heads=64`
+- `num_key_value_heads=1`
+- `n_routed_experts=256`
+- `num_experts_per_tok=6`
+- `q_lora_rank=1024`
+- `o_lora_rank=1024`
+- `qk_rope_head_dim=64`
+- `sliding_window=128`
+- `max_position_embeddings=1048576`
+- `expert_dtype=fp4`
+- FP8 quantization metadata
+
+Design import:
+
+- Good first serious large-model target after the protocol smoke test because it is much smaller than 1.6T-class models while still validating MoE, compressed attention/cache behavior, and large-context routing.
+- Not the first protocol smoke model. Use a smaller, boring, llama.cpp-supported GGUF model first so route/session/KV bugs are isolated from DeepSeek-specific architecture support.
+- The support audit must verify the available local runtime path: PyTorch/Transformers, vLLM/SGLang, and any GGUF/llama.cpp quantization route.
+
+## Ornith-1.0-397B
+
+Sources:
+
+- <https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B>
+- <https://huggingface.co/inferencerlabs/Ornith-1.0-397B-MLX-Q9>
+
+Ornith-1.0-397B is MIT licensed, Qwen3.5-MoE based, with 397B MoE scale. Its base config shows 60 layers and a hybrid pattern where full attention appears every fourth layer, with other layers using linear attention. The MLX Q9 quantized variant is around 447 GB and reports high-quality Q9 behavior in its model card.
+
+Design import:
+
+- Hybrid attention can make large models more tractable than dense full-attention assumptions.
+- Model-specific cache accounting is required; "params" alone is not enough to route.
+
+## Synthesis
+
+The prior art strongly supports the direction, but no project exactly matches the target product:
+
+- Petals proves volunteer layer-serving is useful.
+- exo proves UX/topology-aware local clusters matter.
+- Distributed Llama proves CPU home clusters can cooperate but also shows root/worker constraints.
+- prima.cpp proves llama.cpp/GGUF distribution across low-resource devices is plausible and that disk/mmap scheduling matters.
+- llama.cpp/GGUF is the ecosystem to collaborate with for runtime performance.
+- DeepSeek-V4-Flash is a plausible first serious large-model target after a small protocol smoke model.
+
+The platform-specific work remains:
+
+- torrent/content-addressed model artifact marketplace
+- tracker-owned route selection and billing
+- per-shard local KV sessions
+- relay/NAT support
+- fraud/reputation/audit
+- OpenAI-compatible public gateway

.scratch/distributed-gguf-runtime/technical-challenges.md

@@ -0,0 +1,412 @@
+# Distributed GGUF Technical Challenge Register
+
+This document focuses on the engineering problems that decide whether the distributed GGUF path is viable. The important distinction is:
+
+- **Model artifacts move like torrents.**
+- **Inference state moves like a pipeline.**
+- **Hot KV state does not move unless we are explicitly checkpointing or repairing a route.**
+
+## Current Constraint
+
+The existing full local PyTorch path lets Transformers own generation and local KV cache.
+
+The existing distributed PyTorch path does not. It manually calls shard layers with cache disabled and recomputes the whole growing prompt for every generated token. It passes hidden activations across shard boundaries, not KV cache, but those activations currently include the full sequence on every decode step.
+
+For a 128K context and `hidden_size=6144`, one bfloat16 activation crossing one shard boundary is roughly:
+
+```text
+131072 tokens * 6144 hidden * 2 bytes = 1.5 GiB
+```
+
+That is acceptable once during chunked prefill only if chunked and streamed. It is not acceptable once per generated token.
+
+## Challenge 1: Decode Must Be O(1) Per Token Across Each Seam
+
+Problem:
+
+During decode, sending `[batch, sequence, hidden]` over the network scales with context length. At 128K, the network dominates everything.
+
+Solution:
+
+Split execution into explicit **prefill** and **decode-step** phases.
+
+- Prefill accepts prompt chunks and builds local cache on every shard.
+- Decode-step accepts exactly one new token or one-step activation.
+- Every shard reads its own hot KV state, appends one position, and forwards a one-step activation.
+
+Acceptance test:
+
+- A two-node route prefills a 4K prompt once.
+- The next 100 generated tokens do not resend a 4K activation.
+- Decode seam payload is proportional to `hidden_size`, not `context_length * hidden_size`.
+
+## Challenge 2: Stable Route Session State
+
+Problem:
+
+KV cache only works if every hop agrees that multiple calls belong to the same route session. A fresh request id per hop or per token destroys cache locality.
+
+Solution:
+
+Introduce a route-session lifecycle.
+
+```text
+create route session
+  -> tracker pins inference route
+  -> head node assigns session_id and route_id
+  -> every hop allocates local cache for its layer range
+  -> prefill chunks append cache
+  -> decode steps append cache
+  -> close session releases cache
+```
+
+Minimum state key:
+
+```text
+session_id
+route_id
+model_preset
+model_revision
+backend_id
+cache_abi
+layer_start
+layer_end
+position
+```
+
+Acceptance test:
+
+- A node can report active sessions and cache bytes by session.
+- Closing a session frees the per-shard cache.
+- Replaying a decode-step with the wrong route/session fails before model execution.
+
+## Challenge 3: KV Cache Ownership
+
+Problem:
+
+A centralized KV cache sounds attractive for failover, but it puts remote storage in the tightest loop of generation. It also creates privacy and consistency problems.
+
+Solution:
+
+Hot KV state is owned by the node that owns the shard for that route session.
+
+```text
+Node A: layers 0..15 hot KV for session S
+Node B: layers 16..31 hot KV for session S
+Node C: layers 32..77 hot KV for session S
+```
+
+The tracker may know where active KV lives, but it does not serve it during decode.
+
+Cache servers may store:
+
+- prefix snapshots
+- failover checkpoints
+- audit samples
+- cold reusable context blocks
+
+They must not be required for every generated token.
+
+Acceptance test:
+
+- Killing a cache server does not affect an active decode route.
+- Killing a route node fails the route in alpha unless a compatible snapshot exists.
+
+## Challenge 4: Prefill Is Still Large
+
+Problem:
+
+Even with correct decode, prefill can move a lot of data. A 128K prompt cannot be sent as one activation blob through many shard boundaries.
+
+Solution:
+
+Use the existing binary activation direction from ADR-0008:
+
+- bfloat16 activation body
+- shape/dtype/session metadata in headers
+- zstd level 1 optional compression
+- chunked prefill
+- backpressure between hops
+
+For large contexts, prefill should stream in chunks such as 128, 256, or 512 tokens. The right chunk size is a benchmark output, not a constant baked into the domain model.
+
+Acceptance test:
+
+- Peak per-hop prefill memory is bounded by chunk size.
+- A slow downstream node applies backpressure instead of letting upstream buffer the whole prompt.
+
+## Challenge 5: GGUF Artifact Splits Are Not Execution Shards
+
+Problem:
+
+GGUF split files can divide model data by size or tensor count. That is useful for storage and transfer, but it is not the same as a network Shard. A Shard in this project is a contiguous layer range with reward, route, and cache meaning.
+
+Solution:
+
+Define an artifact manifest that maps storage chunks to semantic model parts.
+
+Required concepts:
+
+```text
+artifact_id
+model_preset
+upstream_repo
+upstream_revision
+license
+runtime_backend
+quantization
+context_cap
+file_hashes
+piece_hashes
+tensor_to_layer_map
+layer_to_artifact_map
+tokenizer_artifacts
+cache_descriptor
+```
+
+The Shard Swarm seeds artifacts. The Inference Route executes shards.
+
+Acceptance test:
+
+- Given `layers 16..31`, a node can compute the exact artifact pieces it must download and verify.
+- Given a local artifact directory, the node can prove which layer ranges it can serve.
+
+## Challenge 6: llama.cpp Is Optimized For Whole-Graph Execution
+
+Problem:
+
+`llama-server` is excellent for local inference, but a distributed route needs lower-level capabilities:
+
+- load selected layers/tensors or mmap them without full materialization
+- accept hidden states from a previous shard
+- execute only a layer range
+- emit hidden states at a boundary
+- own KV/state for only that layer range
+- report cache layout and memory requirements
+
+Those are not the normal public serving abstractions.
+
+Solution:
+
+Stage the llama.cpp path instead of jumping directly to internet-scale distributed GGUF.
+
+1. Use llama.cpp as a full local GGUF backend for immediate CPU performance.
+2. Build a localhost layer-boundary prototype on a simple supported GGUF model.
+3. Identify the minimal `libllama`/ggml hooks needed for layer-range execution.
+4. Collaborate upstream on a stable extension rather than carrying a long-lived fork.
+
+Acceptance test:
+
+- Process A runs layers `0..k`, exports hidden states.
+- Process B imports those hidden states, runs `k+1..n`, and produces logits close to full single-process execution.
+- Both processes maintain only their own cache state.
+
+## Challenge 7: Model Architectures Are Not Uniform
+
+Problem:
+
+Dense Llama-style attention, MoE, MLA/DSA, and hybrid linear/full attention do not have the same cache shape, routing cost, or layer cost.
+
+GLM-5.2 uses compressed DSA/MLA-style state. Ornith uses a hybrid attention pattern. Parameter count alone is a poor routing metric.
+
+Solution:
+
+Keep model-specific cache internals inside the backend. The tracker should route based on backend-advertised capabilities and measured telemetry, not on hardcoded tensor formulas.
+
+Backend capability report:
+
+```text
+model_arch
+supported_runtime
+supports_prefill
+supports_decode_step
+supports_layer_range
+supports_partial_artifacts
+cache_abi
+cache_bytes_per_token_estimate
+prefill_tokens_per_second
+decode_tokens_per_second
+active_memory_floor
+```
+
+Acceptance test:
+
+- The tracker can reject a route because one node lacks the required cache ABI.
+- A model support audit can say "artifact available, local full inference works, distributed layer-boundary unsupported" without ambiguity.
+
+## Challenge 8: Tensor Parallelism Is Not The Same Product
+
+Problem:
+
+Projects like Distributed Llama and prima.cpp lean toward local-cluster tensor/ring parallelism. That can work on a trusted LAN, but it usually requires tight synchronization every layer. On a public internet volunteer route, that becomes fragile and hard to reward.
+
+Solution:
+
+For the public network, make a Shard a contiguous layer range. Tensor parallelism can exist inside one node, one trusted colocated pod, or one future "composite node", but not as the first public routing primitive.
+
+Acceptance test:
+
+- A public route can be represented as ordered layer coverage.
+- Billing can attribute work to layer ranges.
+- No cross-node all-reduce is required on every layer for v1.
+
+## Challenge 9: Heterogeneity And Stragglers
+
+Problem:
+
+A route is only as fast as its slowest hop during decode. A weak node holding a bottleneck shard can make a 1.6T model technically available but unusable.
+
+Solution:
+
+Route selection must use measured telemetry, not static declarations.
+
+Metrics:
+
+- prefill throughput
+- decode throughput
+- queue depth
+- disk read rate
+- memory pressure
+- network latency to neighbors
+- route failure rate
+- cache warmth
+
+The tracker should prefer complete routes that avoid weak nodes, and the rebalancer should increase redundancy for bottleneck layer ranges.
+
+Acceptance test:
+
+- A slow node is removed from a candidate route even if it has the needed layer range.
+- The coverage map can show "covered but under-provisioned" separately from "coverage gap".
+
+## Challenge 10: Reliability And Failover
+
+Problem:
+
+If hot KV is local, route repair is not free. A replacement node cannot continue decoding unless it has compatible cache state.
+
+Solution:
+
+Alpha behavior should be simple:
+
+- route failure during prefill: fail and retry from scratch
+- route failure during decode: fail unless compatible snapshot exists
+- tracker restart: active sessions may be lost
+- node restart: local hot KV is lost
+- client-visible telemetry reports the last known phase and failure reason
+
+Later behavior:
+
+- periodic prefix snapshots
+- snapshot generation ids
+- cache ABI compatibility checks
+- route repair only when the replacement node has the same model revision, layer range, backend cache ABI, and snapshot position
+
+Acceptance test:
+
+- Failures produce explicit route-session errors.
+- No node silently continues from missing or incompatible cache state.
+
+## Challenge 11: Privacy, Fraud, And Audit
+
+Problem:
+
+Hidden activations and KV state can leak information. Public volunteer inference is not private by default. Also, a node can return bad activations while still appearing available.
+
+Solution:
+
+Separate product modes:
+
+- public swarm: low privacy, broad access, audited
+- private swarm: trusted nodes, stronger privacy expectation
+- paid trusted route: selected nodes with stronger guarantees
+
+Use existing validation-event and slash-proof concepts for audit, but adapt them to distributed routes:
+
+- record model preset, route, node wallets, prompt metadata, output, and sampling seed
+- sample full-route replays where feasible
+- compare output/logits within model-specific tolerance
+
+Acceptance test:
+
+- A client can choose public or private route policy.
+- A validation event contains enough information to reproduce route membership and observed output.
+
+## Challenge 12: Economics Must Not Reward The Wrong Bottleneck
+
+Problem:
+
+Layer count, parameter count, active MoE experts, cache memory, disk serving, and network transfer are different costs. A naive equal split across nodes will be wrong.
+
+Solution:
+
+Start with simple compute accounting:
+
+```text
+node_reward_weight =
+  owned_layer_work
+  * prefill_tokens
+  + owned_layer_work
+  * decode_tokens
+```
+
+Then refine with:
+
+- measured throughput
+- active MoE cost
+- storage/seeding contribution
+- cache memory reservation
+- reliability
+
+Keep artifact seeding rewards separate from inference rewards until fraud and metering are clear.
+
+Acceptance test:
+
+- A node that only seeds artifacts is not paid as if it executed inference.
+- A node that executes a heavier shard can earn more than a node executing a light shard.
+
+## Challenge 13: Long Requests Need Streaming Or Realtime Feedback
+
+Problem:
+
+Large distributed routes may spend meaningful time in artifact loading, prefill, queueing, or slow decode. The product can tolerate latency, but users should not wait blindly.
+
+Solution:
+
+Streaming token deltas is preferred when the backend and client transport support it. Generation Telemetry is required regardless of whether token deltas are streamed.
+
+Minimum telemetry:
+
+```text
+session_id
+route_id
+model_preset
+phase = queued | loading | prefill | decode | finalizing | failed
+prefill_tokens_done
+prefill_tokens_total
+generated_tokens
+rolling_tokens_per_second
+average_tokens_per_second
+active_route_nodes
+failure_reason
+```
+
+The gateway may expose token deltas and telemetry through Server-Sent Events or WebSocket. Simple clients may use a polling endpoint for telemetry and receive the final answer only when complete.
+
+Acceptance test:
+
+- A client can show live progress before the first output token is available.
+- During decode, the user sees streamed token deltas when supported.
+- During decode, the user sees rolling tokens/sec even if output text is not streamed.
+- A failed route returns a final error and the last known phase/reason.
+
+## Engineering Order
+
+1. Fix distributed PyTorch cache semantics first. This proves the route-session model without llama.cpp internals.
+2. Add local full-model llama.cpp/GGUF serving for immediate CPU improvement.
+3. Add Generation Telemetry for route sessions so long requests are observable.
+4. Preserve binary HTTP activation transfer while splitting prefill/decode and measuring payload sizes.
+5. Add artifact manifest and Shard Swarm metadata.
+6. Prototype llama.cpp layer-boundary execution locally.
+7. Network the GGUF route only after the cache/session protocol works.
+8. Audit DeepSeek-V4-Flash as the first serious large-model target.
+9. Audit GLM-5.2 and Ornith support after simpler GGUF models pass the route test.

CONTEXT.md

@@ -22,10 +22,38 @@ _Avoid_: torrent, cluster, pool
 An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
 _Avoid_: pipeline, chain, path
 
+**Route Session**:
+An active inference request bound to one Inference Route and one stable session id while the request is being served.
+_Avoid_: conversation, job, token stream
+
+**Activation Seam**:
+The boundary between two adjacent shard executions where hidden states pass from one node to the next.
+_Avoid_: handoff, layer gap, boundary hop
+
+**Hot KV State**:
+The live attention/cache state a node holds for its own shard during a Route Session.
+_Avoid_: centralized KV cache, global cache, remote cache
+
+**Prefix Snapshot**:
+A persisted copy of reusable Route Session state for a prompt prefix, used for reuse, retry, or failover.
+_Avoid_: hot cache, CDN cache, active KV
+
+**Model Artifact**:
+A versioned model file or tokenizer file that nodes download, verify, and keep locally to serve a Model Preset.
+_Avoid_: model blob, weights dump, asset
+
+**Artifact Manifest**:
+The canonical record that identifies the Model Artifacts, their integrity checks, and the model parts they support.
+_Avoid_: torrent file, metadata JSON, download list
+
 **Gateway**:
-The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results back.
+The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results and progress to the client when possible.
 _Avoid_: proxy, relay, orchestrator, primary
 
+**Generation Telemetry**:
+Realtime progress information for an active Route Session, including phase, generated token count, and tokens-per-second speed.
+_Avoid_: logs, debug output
+
 ### Tracker
 
 **Tracker**:
@@ -33,7 +61,7 @@ The coordinator service that maintains the node registry, scores nodes by throug
 _Avoid_: coordinator, scheduler, director
 
 **Tracker Node**:
-A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results back. Any node advertising a new model to the network becomes its tracker node.
+A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results and progress when possible. Any node advertising a new model to the network becomes its tracker node.
 _Avoid_: primary node, master node, gateway node
 
 **Coverage Map**:

QUICKSTART.md

@@ -54,6 +54,7 @@ on the host firewall if other machines will join:
 
 ```bash
 .venv/bin/meshnet-tracker start --host 0.0.0.0 --port 8080
+# --starting-credit 1 --devnet-topup 10
 ```
 
 Verify from the tracker host:
@@ -97,6 +98,7 @@ Nodes can then join with either the LAN tracker URL or the public URL:
 ```bash
 .venv/bin/meshnet-node start --tracker http://192.168.0.179:8080 --model Qwen/Qwen2.5-0.5B-Instruct
 .venv/bin/meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
+.venv/bin/meshnet-node start --tracker https://ai.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
 ```
 
 ### Windows / WSL2
@@ -514,6 +516,7 @@ Send inference through the tracker (which picks the head node and injects the ro
 ```bash
 curl -s https://ai.neuron.d-popov.com/v1/chat/completions \
   -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-mesh-<your-key>" \
   -d '{
     "model": "Qwen/Qwen2.5-0.5B-Instruct",
     "messages": [{"role": "user", "content": "What is 7 times 8?"}],
@@ -529,6 +532,47 @@ curl -s http://localhost:7001/v1/chat/completions \
   -d '{"model": "Qwen/Qwen2.5-0.5B-Instruct", "messages": [{"role": "user", "content": "Hi"}]}'
 ```
 
+## Accounts, API keys, and credit (billing-enabled trackers)
+
+Public trackers run with billing on: `/v1/chat/completions` requires a real
+API key from a registered account. Unknown bearer strings get `401`; a key
+with no balance gets `402 insufficient balance`.
+
+**Dashboard flow (easiest):** open `https://<tracker>/dashboard`, register with
+an email + password, then click **+ new key**. The key (`sk-mesh-…`) shows its
+balance next to it. If the tracker was started with `--starting-credit`, your
+first key arrives pre-funded (Caller Credit, once per account). If it was
+started with `--devnet-topup`, every key row has a **+$N (devnet)** button to
+refill during testing.
+
+**Curl flow:**
+
+```bash
+# 1. Register (once)
+curl -s https://<tracker>/v1/auth/register \
+  -H "Content-Type: application/json" \
+  -d '{"email": "you@example.com", "password": "hunter22-or-better"}'
+# → {"session_token": "...", ...}
+
+# 2. Create an API key (session token from step 1)
+curl -s https://<tracker>/v1/account/keys -X POST \
+  -H "Authorization: Bearer <session_token>"
+# → {"api_key": "sk-mesh-...", "caller_credit_granted": true}
+
+# 3. Check balance / usage
+curl -s https://<tracker>/v1/account -H "Authorization: Bearer <session_token>"
+
+# 4. (devnet trackers only) top up a key
+curl -s https://<tracker>/v1/account/topup -X POST \
+  -H "Authorization: Bearer <session_token>" \
+  -H "Content-Type: application/json" \
+  -d '{"api_key": "sk-mesh-..."}'
+```
+
+Operator side: both features default to 1 USDT (`--starting-credit` /
+`--devnet-topup`). Set both to 0 on mainnet deployments — real deposits flow
+through the on-chain USDT treasury watcher instead.
+
 ---
 
 ## Step 1 — Start the tracker (Terminal 1)

deploy/portainer/meshnet-tracker-nobuild-stack.yml

@@ -28,6 +28,9 @@ services:
       PUBLIC_RELAY_URL: ${PUBLIC_RELAY_URL:-}
       HEARTBEAT_TIMEOUT: ${HEARTBEAT_TIMEOUT:-30}
       ENABLE_BILLING_DB: ${ENABLE_BILLING_DB:-1}
+      # Devnet-friendly defaults (US-039/040): set both to 0 on mainnet.
+      STARTING_CREDIT: ${STARTING_CREDIT:-1}
+      DEVNET_TOPUP: ${DEVNET_TOPUP:-1}
       SOLANA_RPC_URL: ${SOLANA_RPC_URL:-}
       USDT_MINT: ${USDT_MINT:-}
       MESHNET_TREASURY_KEYPAIR_B64: ${MESHNET_TREASURY_KEYPAIR_B64:-}
@@ -87,6 +90,8 @@ services:
           --relay-url "$${RELAY_URL}" \
           --stats-db /var/lib/meshnet/tracker-stats.sqlite \
           --accounts-db /var/lib/meshnet/accounts.sqlite \
+          --starting-credit "$${STARTING_CREDIT:-1}" \
+          --devnet-topup "$${DEVNET_TOPUP:-1}" \
           $${BILLING_ARGS} \
           $${TREASURY_ARGS} \
           $${PEER_ARGS}

docs/DEPLOY_PORTAINER.md

@@ -43,13 +43,27 @@ ENABLE_BILLING_DB=1
 ```
 
 For first cloud-only test, use `CLUSTER_PEERS=`. Click **Deploy the stack**.
+(`ai.neuron.d-popov.com` is publicly reachable, so two-tracker sync works over the
+internet — but add it only after the cloud-only friends test proves out; two-peer
+Raft adds moving parts without fault tolerance.)
 
 `ENABLE_BILLING_DB=1` makes billing public behavior active: `/v1/chat/completions`
-requires `Authorization: Bearer <api-key>`. Any key is accepted and starts with
-the configured starter credit; calls return `402` after that balance is
+requires `Authorization: Bearer <sk-mesh-...>` — a real API key created through
+an account (register on `/dashboard`, then "+ new key"); arbitrary bearer
+strings are rejected with `401`. Calls return `402` once the key's balance is
 exhausted. Set `ENABLE_BILLING_DB=0` if existing unauthenticated clients must
 keep working during the first redeploy.
 
+Credit variables (US-039/US-040):
+
+```text
+STARTING_CREDIT=1      # one-time Caller Credit (USDT) on an account's first key
+DEVNET_TOPUP=1         # dashboard "+$N (devnet)" faucet button; 0 disables
+```
+
+Both default to **1** (devnet-friendly alpha). On any deployment holding a
+mainnet treasury set both to 0 — the faucet mints client balance for free.
+
 Optional Solana treasury settlement variables:
 
 ```text
@@ -94,7 +108,9 @@ Custom locations on the same proxy host:
 
 Leave sub-folder forwarding empty.
 
-If WebSockets fail, Advanced:
+**Required:** NPM custom locations do NOT inherit the proxy host's "Websockets
+Support" toggle. Paste this into the **Advanced** box of *each* custom location
+(`/ws` and `/rpc`), or every WebSocket handshake to the relay dies at nginx:
 
 ```nginx
 proxy_http_version 1.1;

docs/issues/36-relay-streamed-chat.md

@@ -0,0 +1,88 @@
+# US-036 — Streamed chat completions over the relay RPC path
+
+Status: planned
+Priority: Critical (blocks public friends-test deployment)
+Stage: Designed
+
+## Context
+
+With the tracker deployed on a public VPS (`cloud.neuron.d-popov.com`), every node is
+behind NAT, so **every** chat request is proxied tracker → relay → head node via
+`_relay_http_request` (`packages/tracker/meshnet_tracker/server.py`). That function does a
+single blocking `ws.recv()` and the node's `RelayHttpBridge._handle_request` does a single
+`resp.read()`. Two consequences for `stream: true` requests:
+
+1. **No live streaming** — the client sees nothing until generation completes, then
+   receives the entire SSE body at once with a `Content-Length` header.
+2. **Zero billing** — the tracker runs `_billable_non_stream_tokens(json.loads(body))`
+   on the buffered body; SSE text is not JSON, the parse fails, and the request is
+   billed/credited as 0 tokens. Off-LAN this silently zeroes out *all* streamed-request
+   accounting.
+
+Decision (grilled 2026-07-06): implement **true multi-frame streaming** over the relay
+RPC WebSocket, scoped to the tracker → head-node leg. Rejected alternatives:
+billing-only SSE parse (fragile heuristic, blank-screen UX stays) and forcing
+`stream:false` over the relay (exact billing but still no live tokens for testers).
+Streaming through the existing SSE accounting loop fixes both symptoms with one
+mechanism. Inter-node `/forward` activation hops stay single-frame (ADR-0014) — they
+are one-tensor-in/one-tensor-out and gain nothing from chunking.
+
+## Protocol
+
+A relayed response becomes a sequence of `relay-http-response` envelopes sharing one
+`request_id`:
+
+```json
+// first frame — status + headers, opens the stream
+{"request_id": "<id>", "status": 200, "headers": {"Content-Type": "text/event-stream"},
+ "stream": true, "chunk": "data: {...}\n\n", "done": false}
+// zero or more continuation frames
+{"request_id": "<id>", "stream": true, "chunk": "data: {...}\n\n", "done": false}
+// terminal frame
+{"request_id": "<id>", "stream": true, "done": true}
+```
+
+**Backward compatibility:** a frame with no `stream` key is a complete single response
+(today's format, still used for `/forward` hops, non-SSE responses, and older nodes).
+The relay and tracker treat it as terminal.
+
+### Per component
+
+- **Node bridge** (`packages/node/meshnet_node/relay_bridge.py`): when the local
+  response `Content-Type` is `text/event-stream`, read line-by-line and emit chunk
+  frames as lines arrive; otherwise keep the existing single-frame path (including
+  `body_base64` for binary). Frame sends go through the bridge's WS send lock
+  (US-037) so frames from concurrent requests interleave whole, never torn.
+- **Relay server** (`packages/relay/meshnet_relay/server.py`): `_handle_rpc` replaces
+  the single `asyncio.Future` in `_pending_rpc` with a per-request `asyncio.Queue`.
+  Frames arriving on the target peer's gossip connection are routed by `request_id`
+  and forwarded to the requester WS until `done` (or a terminal legacy frame).
+  Timeouts: keep the 310 s overall cap; add a 120 s per-frame idle timeout so a dead
+  node doesn't pin the queue.
+- **Tracker** (`packages/tracker/meshnet_tracker/server.py`): `_relay_http_request`
+  grows a streaming mode — loop `ws.recv()`; on the first frame send status/headers to
+  the client; write each `chunk` to the client immediately and feed it through the
+  same SSE token-accounting used by the direct-proxy stream loop
+  (`reported_stream_tokens` / `_record_observed_throughput` / `_bill_completed`), so
+  relayed streams bill identically to direct streams. Non-stream frames keep the
+  current buffered handling.
+
+### Known limitation (accepted for alpha)
+
+If the client disconnects mid-stream, the relay drops undeliverable frames but the
+node keeps generating until completion — wasted compute bounded by one generation.
+Cancellation propagation is future work.
+
+## Acceptance criteria
+
+- `stream: true` chat request via relay delivers SSE chunks to the client
+  incrementally (test observes ≥2 distinct frame arrivals before `[DONE]`)
+- Relayed streamed request records nonzero billed tokens and node work credit
+- Non-streamed relayed requests and `/forward` binary hops behave exactly as before
+  (single frame, `body_base64` round-trip intact)
+- Legacy single-frame response from an old node is accepted as terminal
+- Idle stream (no frame for 120 s) returns 504 to the client and cleans up the
+  relay-side queue
+- Extend `tests/test_gossip_and_relay.py` alongside
+  `test_relay_rpc_round_trips_http_request_to_peer`
+- `python -m pytest` passes from repo root

docs/issues/37-relay-bridge-concurrency.md

@@ -0,0 +1,51 @@
+# US-037 — Concurrent request handling in the node relay bridge
+
+Status: planned
+Priority: Critical (blocks public friends-test deployment)
+Stage: Designed
+
+## Context
+
+`RelayHttpBridge._run` (`packages/node/meshnet_node/relay_bridge.py`) handles
+`relay-http-request` envelopes serially inside its recv loop: `_handle_request` blocks
+on `urllib.request.urlopen(..., timeout=300)` before the next envelope is read.
+
+Off-LAN this is a correctness bug, not just a throughput limit: a node can be the
+**head** of one inference route and a **downstream hop** of another at the same time.
+While the head request occupies the bridge (up to 300 s of generation), the other
+route's per-token `/forward` calls sit unread in the WebSocket buffer — overlapping
+routes through a shared node are effectively broken.
+
+Decision (grilled 2026-07-06): dispatch requests on a **bounded worker pool, default
+8, configurable**. Rejected alternatives: unbounded thread-per-request (a public
+deployment exposes volunteer machines to request stampedes) and reject-when-full
+(bouncing `/forward` hops kills other routes' in-flight sessions; queueing beyond the
+cap is today's behavior, just 8-wide).
+
+## Design
+
+- Recv loop only parses envelopes and submits to a
+  `ThreadPoolExecutor(max_workers=N)`; workers run `_handle_request` and send the
+  response frame(s).
+- All sends on the single relay WebSocket go through a `threading.Lock`. With US-036
+  a response may be multiple frames — the lock is held **per frame**, so streams from
+  concurrent requests interleave frame-atomically (receivers demux by `request_id`).
+- `N` defaults to 8; configurable via `meshnet-node start --relay-concurrency N`
+  (env `MESHNET_RELAY_CONCURRENCY`). Requests beyond `N` queue in the executor.
+- On reconnect (`_run`'s outer loop), in-flight workers from the dead connection may
+  still try to send; the send helper swallows failures on a closed socket, and the
+  relay side times the orphaned request out (US-036 idle timeout / existing 310 s cap).
+- `stop()` shuts the executor down without waiting for stragglers (daemon threads,
+  same as today's bridge thread).
+
+## Acceptance criteria
+
+- While one relayed request is in flight (slow local handler), a second
+  `relay-http-request` to the same node completes without waiting for the first
+- Responses are correctly matched by `request_id` when they complete out of order
+- More than `N` simultaneous requests queue and all eventually complete; thread count
+  never exceeds `N` workers
+- Bridge survives a relay reconnect with workers still in flight (no crash, no
+  deadlock; orphaned responses dropped)
+- Extend `tests/test_gossip_and_relay.py`
+- `python -m pytest` passes from repo root

docs/issues/38-tracker-seed-join.md

@@ -0,0 +1,62 @@
+# US-038 — Tracker cluster join via a single seed peer
+
+Status: planned
+Priority: High (blocks adding trackers without restarting the fleet)
+Stage: Proposed
+
+## Context
+
+Today the tracker cluster is fully static. `RaftNode` and `NodeGossip`
+(`packages/tracker/meshnet_tracker/raft.py`, `gossip.py`) take a peer list at
+construction and never change it; `/v1/raft/status` does not expose membership.
+
+Consequences of starting a new tracker with only one existing peer in
+`--cluster-peers`:
+
+- Existing trackers never learn about the newcomer — they keep heartbeating and
+  replicating to their original list only.
+- Quorum math (`(len(self.peers) + 1) / 2`) differs per tracker, so vote counts
+  and commit decisions disagree — the cluster silently splits rather than erroring.
+
+Required behavior: a joining tracker is configured with **any one** live tracker
+(a seed). It announces itself, the membership change replicates through the Raft
+log, and every tracker — including the newcomer — converges on the same full peer
+list. Removing the need to restart existing trackers when the hive grows.
+
+## Design
+
+1. **Expose membership.** `GET /v1/cluster/peers` returns
+   `{"self": <url>, "peers": [<url>, ...]}` (admin-safe: URLs only).
+2. **Join handshake.** On startup, the joiner calls the seed's
+   `POST /v1/cluster/join` with its public self-url. The request is
+   hive-HMAC-signed (`MESHNET_HIVE_SECRET`, same fail-closed rule as gossip —
+   an unauthenticated join on a public tracker would let anyone enter the hive).
+   A non-leader seed forwards to the current leader (same pattern as
+   `/v1/nodes/register` forwarding).
+3. **Membership through the log.** The leader appends a
+   `cluster-membership` entry (single-server change: add one peer per entry).
+   `RaftNode` gains `apply`-side handling that swaps `self.peers` — quorum is
+   recomputed from the applied membership, never from local config. Gossip peer
+   list updates from the same apply hook.
+4. **Joiner bootstrap.** After a successful join response (which includes the
+   current peer list), the joiner sets its own peers and starts Raft as a
+   follower; it catches up via normal append-entries.
+5. **Persistence.** Applied membership is written to the stats/state sqlite so a
+   restarted tracker rejoins with the last known list even if its seed is down.
+6. **Config semantics.** `--cluster-peers` becomes "seed list": tried in order
+   until one join succeeds. A tracker with no seeds and no persisted membership
+   runs standalone (current single-tracker behavior unchanged).
+
+Out of scope: peer removal/eviction (leave via operator restart for now),
+joint consensus for multi-server changes, automatic seed retry after startup.
+
+## Acceptance criteria
+
+- Start trackers A+B as a cluster; start C with only A as seed → within one
+  election timeout, A, B, and C all report the same 3-peer membership on
+  `GET /v1/cluster/peers`, and a value proposed on C commits on A and B
+- Join without a valid hive signature is rejected with 403; join to a follower
+  is forwarded to the leader transparently
+- Restarting C with its seed offline rejoins from persisted membership
+- Standalone tracker (no seeds) behaves exactly as today
+- `python -m pytest` passes from repo root

docs/issues/39-caller-credit-account-keys.md

@@ -0,0 +1,60 @@
+# US-039 — Caller Credit granted once per account; chat requires account keys
+
+Status: planned
+Priority: Critical (blocks friends-test inference — every request 402s)
+Stage: Designed
+
+## Context
+
+`DEFAULT_STARTING_CREDIT = 0.0` and nothing can raise it: no CLI flag, no admin
+credit endpoint. The only funding path is the on-chain USDT deposit watcher —
+so on a fresh public tracker every inference request returns
+`402 insufficient balance`, including keys created through the working
+account flow (`/v1/auth/register` → `/v1/account/keys`, also in the dashboard).
+
+Separately, the billing gate accepts **any** bearer string (only revoked keys
+are rejected), so wiring starter credit into `BillingLedger.ensure_client`
+would let anyone mint unlimited free credit by inventing keys.
+
+Decision (grilled 2026-07-06): **Caller Credit is granted once per account**,
+when the account creates its first API key; chat requests on an
+accounts-enabled tracker must present a real active `sk-mesh-` key.
+Rejected: credit-any-bearer (free-inference farming against volunteers' GPUs
+on the open internet) and manual-admin-credit-only (operator toil, no
+self-serve testing).
+
+## Design
+
+1. **`BillingLedger.grant_caller_credit(api_key, account_id, amount) -> bool`**
+   Appends a credit event with the deterministic id
+   `caller-credit-{account_id}` and note `caller-credit`. Already-seen id →
+   no-op returning False. The deterministic id makes the once-per-account rule
+   idempotent locally *and* across hive gossip (event dedupe is by id), so a
+   second key, a retried request, or a replicated event can never double-grant.
+2. **`AccountStore.is_active_key(api_key) -> bool`.** The chat billing gate
+   (`_handle_proxy_chat`) rejects keys that are not active account keys with
+   `401 invalid_api_key` — but only when an accounts store is configured, so
+   embedded/test trackers without accounts keep today's any-key behavior.
+3. **`--starting-credit N`** (USDT) on `meshnet-tracker start`. Default is the
+   single constant `DEFAULT_CALLER_CREDIT_USDT = 1.0` in `server.py`
+   (devnet-friendly alpha, revised 2026-07-06; set 0 on mainnet). Threaded
+   through `TrackerServer` → `_TrackerHTTPServer`; registration and
+   `_handle_account_key_create` call `grant_caller_credit` when billing is on
+   and the amount is positive. The ledger's own `_starting_credit` stays 0 —
+   implicit per-key grants in `ensure_client`/`charge_request` remain dead
+   code paths.
+4. **Stack variable `STARTING_CREDIT`** in
+   `deploy/portainer/meshnet-tracker-nobuild-stack.yml` (default 1).
+5. **Docs.** QUICKSTART gains "Get an API key" (register/login/create key via
+   dashboard or curl, what 402 means); DEPLOY_PORTAINER's stale "any key is
+   accepted and starts with the configured starter credit" paragraph replaced.
+
+## Acceptance criteria
+
+- Fresh account → first key → key has `--starting-credit` balance; chat succeeds
+- Second key on the same account → no additional credit
+- Revoke-and-recreate keys → still no additional credit (deterministic event id)
+- Random bearer string on an accounts-enabled tracker → 401, never 402/free work
+- Tracker without accounts store: gate behavior unchanged
+- `--starting-credit 0` disables the grant entirely (mainnet posture)
+- `python -m pytest` passes from repo root

docs/issues/40-devnet-dashboard-topup.md

@@ -0,0 +1,38 @@
+# US-040 — Devnet top-up button on the dashboard
+
+Status: planned
+Priority: High (friends-test convenience — refill without on-chain deposits)
+Stage: Designed
+
+## Context
+
+Once Caller Credit (US-039) is spent, the only refill path is a real USDT
+deposit through the treasury watcher — devnet Solana plumbing friends won't
+have. For test deployments the dashboard needs a "add $10" style button.
+This is a faucet: 0 disables it, and mainnet deployments must set 0.
+
+## Design
+
+1. **`--devnet-topup N`** (USDT per click) on `meshnet-tracker start`; stack
+   variable `DEVNET_TOPUP`. Default is the single constant
+   `DEFAULT_DEVNET_TOPUP_USDT = 1.0` in `server.py` (devnet-friendly alpha,
+   revised 2026-07-06 — the alpha's public trackers are devnet-only).
+2. **`POST /v1/account/topup`** — session-authenticated (same as key
+   management). Body: `{"api_key": "sk-mesh-..."}`. Rules:
+   - 404 when the feature is disabled (flag absent/0)
+   - key must belong to the logged-in account (403 otherwise)
+   - credits exactly the configured amount (client cannot choose it),
+     note `devnet-topup`; returns the new balance
+3. **Dashboard**: `/v1/account` response gains `topup_amount` (0 when
+   disabled); when positive, each key row renders a `+$N (devnet)` button
+   calling the endpoint.
+4. **Abuse bound**: per-click amount is operator-set and requires a registered
+   session; acceptable for a friends-test faucet. Mainnet deployments simply
+   never set the flag. (Rate limiting is deliberately out of scope.)
+
+## Acceptance criteria
+
+- Flag off: endpoint 404s, dashboard shows no top-up button
+- Flag on: logged-in user tops up own key, balance rises by exactly N
+- Topping up another account's key → 403
+- `python -m pytest` passes from repo root

docs/issues/41-account-wallet-keypair.md

@@ -0,0 +1,65 @@
+# US-041 — Account wallet: browser-extension signing, in-browser generation, export-only
+
+Status: planned
+Priority: Medium (not needed for devnet friends test; needed before mainnet)
+Stage: Designed
+
+## Context
+
+Registration accepts an optional `wallet` field (a pubkey string) and the
+dashboard never shows it. Users need a visible wallet address on the account
+panel and a path to real Solana later, without the tracker ever holding user
+private keys.
+
+Today wallets matter in two places: client deposit attribution
+(`bind_wallet` → deposit watcher) and node payout identity (Pending Balance
+keyed by wallet). Accounts themselves hold no keypair — and will stay that way.
+
+## Decision (2026-07-06)
+
+**Non-custodial.** The tracker stores public keys only; private keys never
+leave the browser.
+
+1. **External wallets via browser extension.** The dashboard integrates the
+   standard Solana wallet-adapter flow (Phantom, Solflare, …): connect the
+   extension, prove ownership by signing a nonce, and the verified pubkey is
+   stored as the account wallet. All future transaction authorization
+   (deposits; later, anything requiring a user signature) happens in the
+   extension — the tracker only ever sees signatures and pubkeys.
+2. **New wallets generated in-browser.** For users without an extension the
+   dashboard generates a keypair client-side (dashboard JS, `Ed25519` via
+   WebCrypto or bundled tweetnacl — CSP-safe, no CDN). The pubkey is uploaded;
+   the private key is shown once for export (base58, plaintext acceptable for
+   devnet) with a "save this now" warning, and optionally kept in
+   localStorage for convenience on devnet.
+3. **No private-key import.** Users cannot paste an existing private key into
+   the dashboard — accepting foreign private keys is a liability the project
+   refuses. Users with existing wallets use the extension path (which imports
+   nothing; it signs in place).
+
+## Scope
+
+- Account panel shows the account wallet address (copy button) or the two
+  attach flows (connect extension / generate in browser)
+- `POST /v1/account/wallet` — session-authenticated; body carries pubkey and,
+  for the extension flow, the signed-nonce ownership proof (reuse
+  `wallet_proof.py` if it fits); binds the wallet to the account's keys for
+  deposit attribution (existing `bind_wallet` path)
+- Wallet is per-account; the tracker fans the binding out to the account's
+  API keys internally
+- Mainnet cutover remains config-only (same boundary as Mock USDT, ADR-0015)
+
+Out of scope: password-wrapped keystore export, hardware wallets, changing
+the node/miner wallet flow.
+
+## Acceptance criteria
+
+- Connect-extension flow stores a verified pubkey (rejects unsigned/mismatched
+  nonce proofs)
+- Generate flow: pubkey lands on the account; private key is never sent to the
+  tracker (assert no such field in the request), export works
+- No endpoint or UI accepts a private key
+- Deposits to the shown address credit the account's keys via the existing
+  watcher
+- Address visible on the account panel after either flow
+- `python -m pytest` passes from repo root

docs/issues/42-gguf-llamacpp-node-backend.md

@@ -0,0 +1,55 @@
+# US-042 — GGUF/llama.cpp node backend
+
+Status: planned
+Priority: High (unlocks big MoE models on volunteer hardware — the pool's core value)
+Stage: Draft design
+
+## Context
+
+The node backend is transformers-only (`model_backend.py` →
+`AutoModelForCausalLM`). For DeepSeek-V4-Flash (158B MoE, official weights FP8
+160 GB) the only quantizations that run on consumer hardware are GGUF
+(IQ2 87 GB → Q4_K_M-XL 175 GB) — llama.cpp format. The transformers-compatible
+quants (FP8, NVFP4, GPTQ W4A16) all need datacenter GPUs. Volunteer machines —
+including our own Strix Halo boxes (128 GB and 80 GB unified memory, GPU via
+Vulkan/ROCm, no FP8 support on RDNA3.5) — run these models today only under
+llama.cpp.
+
+## Design directions to evaluate (design-it-twice)
+
+**A. llama.cpp as a per-node shard executor.** Node loads a *layer range* of a
+GGUF via llama-cpp-python; our existing hop protocol (X-Meshnet-Route,
+activations over HTTP/relay) moves hidden states between nodes. Requires
+llama.cpp partial-layer loading and activation import/export — investigate
+feasibility first; this is the riskiest unknown.
+
+**B. llama.cpp RPC mode under tracker orchestration.** llama.cpp ships a
+native RPC backend that splits one model across machines. The tracker would
+provision/route to an llama.cpp RPC cluster rather than our own hop pipeline.
+Less code, but bypasses our billing/telemetry hop instrumentation and relay
+NAT path — needs a story for both.
+
+**C. Whole-model GGUF nodes (no sharding).** A node with enough memory serves
+a full GGUF (e.g. IQ2/IQ3 on a 128 GB box); the tracker routes whole requests
+to it (single-hop route). Smallest step, no cross-node activation work, and
+already useful: Strix Halo 128 GB serves DeepSeek-V4-Flash IQ3_XXS (114 GB)
+via llama.cpp Vulkan today.
+
+Recommended sequencing: C first (small, real value), then A/B investigation.
+
+## Also in scope
+
+- Model catalog: allow GGUF entries with quant selection; feature
+  `DeepSeek-V4-Flash` IQ4_XS/UD-Q4_K_XL as a curated/featured entry once at
+  least direction C works (a featured model nobody can load is an anti-feature)
+- Hardware detection: recognize Strix Halo/unified-memory APUs and Vulkan
+  (`hardware.py` currently reports "CPU mode" on these boxes)
+- `MESHNET_DOWNLOAD_DIR`/`--download-dir` applies to GGUF files as well
+
+## Acceptance criteria (phase C)
+
+- A node with `--gguf <repo-or-path> --quant IQ3_XXS` serves
+  `/v1/chat/completions` via llama.cpp with GPU offload where available
+- Tracker treats it as a full-coverage node (single-hop routes, billing works)
+- Streamed responses work through the tracker proxy and the relay (US-036)
+- `python -m pytest` passes from repo root (llama.cpp behind an optional extra)

docs/issues/43-dashboard-model-search-cards.md

@@ -0,0 +1,38 @@
+# US-043 — Dashboard model search and model cards
+
+Status: planned
+Priority: Medium (post-deploy polish)
+Stage: Idea
+
+## Context
+
+The dashboard shows nodes/routes/billing but nothing model-centric. Operators
+and testers should be able to search for a model and see, per model, a card
+with what the network knows about it.
+
+## Scope
+
+- **Search**: query box hitting a new tracker endpoint that proxies the HF Hub
+  search API (server-side, so the dashboard stays CSP-clean and unauthenticated
+  browsers aren't rate-limited) merged with the tracker's own model presets and
+  currently-served models.
+- **Model card** per result:
+  - name, architecture, params, layer count (reuse `model_metadata_for`,
+    which now handles nested `text_config` — US layer-detection fix)
+  - coverage on the network: which layer ranges are served, by how many nodes,
+    coverage gaps (the Coverage Map already exists on the tracker)
+  - price per 1K tokens, availability (routable now? single-hop or pipeline?)
+  - memory footprint per quantization where known (bf16 / GGUF sizes)
+  - action: "request this model" — flags demand so node operators (or
+    auto-shard assignment) know what to load next
+- Featured models section driven by the curated catalog (`CURATED_MODELS`),
+  including GGUF entries once US-042 lands.
+
+## Acceptance criteria
+
+- Searching a HF repo id or free text returns results without the browser
+  calling HF directly
+- A served model's card shows live coverage and a working "chat now" state
+- An unserved model's card shows the "request" action and estimated memory
+  per quant
+- `python -m pytest` passes from repo root

docs/issues/44-tracker-shard-source-partial-download.md

@@ -0,0 +1,103 @@
+# US-044 — Tracker as model-file source; nodes download only their shard
+
+Status: in progress
+Priority: High (blocks multi-machine big-model serving; pairs with US-042)
+Stage: Designed (grill remaining decisions before build)
+
+## Context
+
+Common deployment: the tracker and the first node share a machine that already
+holds the model files (e.g. `/run/media/popov/DATA/llm/safetensor`). When a
+second node joins with no model selected, the tracker assigns it the uncovered
+layer range — and today that node then downloads the **entire snapshot from
+HuggingFace**, even for a 20-layer shard of a 160 GB model.
+
+What exists already (build on it, don't duplicate):
+
+- Nodes serve their shard dir as a tar at `GET /v1/shards/download` with
+  checksum verification; `download_shard` tries assignment-provided `peers`
+  before HF (`downloader.py`). But it only matches **identical layer ranges**,
+  and the HF fallback runs `snapshot_download` of the whole repo.
+- The torch path (`--model-id`) bypasses `download_shard` entirely:
+  `TorchModelShard` → `from_pretrained` downloads **and loads into RAM** the
+  full model, then executes only the assigned layers. Sharding currently saves
+  compute, not memory or bandwidth.
+
+## Design
+
+1. **Tracker `--models-dir PATH`** (env `MESHNET_MODELS_DIR`). When set, the
+   tracker indexes HF-layout snapshots under it and advertises itself as a
+   model-file source in `/v1/nodes/assign` responses.
+2. **Layer-aware file selection.** For safetensors models, read
+   `model.safetensors.index.json` and map the assigned layer range → the
+   subset of weight files containing those layers, plus the always-needed
+   files (config, tokenizer, index, embeddings/head files for head/tail
+   shards). Serve exactly that subset (tar stream, per-file checksums).
+   GGUF (US-042): single file or naive byte-range — phase 2.
+3. **Node download order**: exact-shard peer (existing) → tracker/peer file
+   subset (new) → HF `snapshot_download` with `allow_patterns` for the same
+   subset (new — stop downloading the whole repo even from HF) → full snapshot
+   (last resort).
+   - The `allow_patterns` subset must not depend on the tracker having a local
+     snapshot: when the tracker has no `--models-dir` match for a repo (or
+     hasn't cached it yet — the common case for a fresh public tracker),
+     `model_sources` comes back empty and `download_shard` falls straight to
+     `_download_huggingface_subset(..., allow_patterns=None)`, i.e. the full
+     repo. Reported 2026-07-06: a CPU node assigned layers 0–2 of
+     `unsloth/Qwen3.6-35B-A3B` (42 safetensor shards) sat downloading the
+     entire model unauthenticated because of this. Fix: fetch
+     `model.safetensors.index.json` + `config.json` directly from HF (a few
+     KB) and compute the same layer-scoped file subset client-side, so the
+     HF-fallback path is filtered even with an empty `model_sources`.
+4. **Partial LOAD (the hard half).** Downloading a subset is wasted unless the
+   node stops instantiating the full model: build the model skeleton on the
+   `meta` device, materialize only assigned layers (+embeddings/norm/head as
+   role requires) from the local files, leave the rest on meta. Without this,
+   an 80 GB machine can never hold a shard of a 160 GB model regardless of
+   how the bytes arrive. This is the acceptance bar for the issue.
+
+## Open questions (grill before building)
+
+- Trust: joining nodes fetch weights from the tracker/peers — checksum against
+  what root of trust? (HF etag/sha vs tracker-signed manifest.)
+- Disk layout: partial snapshots must not corrupt the HF cache dir; probably
+  a meshnet-owned layout keyed by repo+revision.
+- Serving cost: a 100 GB tar stream per joining node on the tracker box —
+  rate-limit/queue? LAN-only heuristic?
+
+## Acceptance criteria
+
+- [x] Tracker can be started with `--models-dir PATH` / `MESHNET_MODELS_DIR`
+  and advertises a local model-file source in assignment responses when it has
+  a matching HF snapshot.
+- [x] Tracker serves a tar stream containing only the safetensors files selected
+  for the assigned layer range plus config/tokenizer/index metadata.
+- [x] Node downloader keeps exact-shard peers first, then races tracker model
+  sources against a HuggingFace `snapshot_download(..., allow_patterns=...)`
+  subset download, using the first successful source.
+- [x] When no tracker model source is available at all, the HuggingFace
+  fallback still computes `allow_patterns` from the repo's own
+  `model.safetensors.index.json` (fetched directly, not via the tracker) —
+  it never silently downloads the full model just because the tracker has
+  nothing cached.
+- [x] Real PyTorch model startup can use tracker `full_url` sources to fetch
+  the full local snapshot over LAN before `from_pretrained`, so local-network
+  testing no longer has to pull from HuggingFace first.
+- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
+  layers 0–k; machine B joins with no model and receives **only** the files
+  for its assigned range from A — nothing fetched from HF
+- [ ] Machine B's resident memory scales with its shard size, not model size
+- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
+- [x] Single-node/full-model flows unchanged
+- [ ] `python -m pytest` passes from repo root
+
+## Implementation notes
+
+- 2026-07-06: Added the tracker/node download path. For immediate Qwen3.6-35B
+  LAN testing, real PyTorch nodes fetch the full snapshot from the tracker via
+  `full_url`; HuggingFace remains fallback-only, and when it is used the node
+  computes `allow_patterns` from the repo's remote SafeTensors index so it
+  stays layer-filtered even without tracker-cached files. Remaining hard half
+  is true partial model materialization: the backend can prefer a downloaded
+  local model directory, but Transformers still needs a `meta`-device load
+  path that materializes only assigned layers.

docs/issues/45-dual-rate-billing.md

@@ -0,0 +1,60 @@
+# US-045 — Dual-rate billing: separate input and output token prices
+
+Status: in progress
+Priority: High (billing correctness before friends test; providers all price this way)
+Stage: Designed
+
+## Context
+
+Today the ledger has one `price_per_1k_tokens` per model, and the two proxy
+paths don't even agree on what they count:
+
+- **Non-streaming** bills `usage.total_tokens` (prompt + completion) at the
+  blended rate (`_billable_non_stream_tokens`).
+- **Streaming** bills `min(observed output deltas, reported total)` — output
+  only in practice (`_billable_stream_tokens`).
+- The HF pricing refresher (issue 23) averages a provider's input/output
+  rates 50/50 (`blended_price_per_1k_tokens`), which misprices asymmetric
+  models — e.g. Qwen3.6-35B-A3B on deepinfra is $0.15/1M in, $0.95/1M out.
+
+Decision (user, 2026-07-06): charge **both** input and output tokens, at
+**two separate rates**, same as other providers.
+
+## Design
+
+1. **`BillingLedger`** stores `{model: (input_per_1k, output_per_1k)}`.
+   - `set_prices(model, input_per_1k, output_per_1k)` (new);
+     `set_price(model, p)` keeps working and sets both.
+   - `prices_for(model) -> (input, output)` (new); `price_for(model)` returns
+     the blended average for back-compat (estimators/logs).
+   - `charge_request(...)` gains keyword `input_tokens`/`output_tokens`; when
+     provided, `cost = in_rate·in/1k + out_rate·out/1k` and the event records
+     the split. Without them, legacy behavior (blended × total) — old events
+     and gossip replicas replay unchanged (`cost` stays the applied field).
+2. **Token counting** (`server.py`):
+   - Non-stream: prefer `usage.prompt_tokens`/`completion_tokens`; fall back
+     to content estimates (`_estimate_prompt_tokens`, observed completion),
+     capped by `max_tokens` bounds as today.
+   - Stream (direct + relay): output = observed deltas as today; input =
+     `usage.prompt_tokens` when a usage chunk appears, else the prompt
+     estimate from the request body. `_stream_line_tokens` returns the parsed
+     usage triple instead of just the total.
+3. **Presets**: `input_price_per_1k_tokens` / `output_price_per_1k_tokens`
+   (dual keys win; `price_per_1k_tokens` alone still means "both rates").
+   Qwen3.6-35B-A3B: input 0.00012, output 0.00076 (80% of deepinfra).
+4. **HF refresher**: applies 80% of each side separately via `set_prices`
+   (all alias keys); change log keeps recording the blended pair for history
+   continuity.
+5. **Spend cap** (`--max-charge-per-request`): estimate =
+   `in_rate·prompt_estimate + out_rate·completion_limit`.
+
+## Acceptance criteria
+
+- Streamed and non-streamed requests for the same exchange bill the same
+  split (input charged in both)
+- A model with asymmetric provider rates bills input and output differently;
+  `usage_for` / billing events expose the split
+- Old persisted billing events replay byte-identically (balances unchanged)
+- HF refresh sets both rates from the marketplace row, not the average
+- Spend cap uses the dual rates
+- `python -m pytest` passes from repo root

docs/issues/46-tracker-env-and-first-node-autojoin.md

@@ -0,0 +1,63 @@
+# US-046 — Tracker .env awareness + first-node auto-join bootstrap
+
+Status: in progress
+Priority: High (blocks the US-044 two-machine test; auto-join dead on fresh trackers)
+
+## Context
+
+Reported 2026-07-06: a node auto-joining a fresh tracker prints
+`(auto-join unavailable: HTTP Error 503)` and then cannot download the model
+from the tracker. Two independent causes:
+
+1. **Bootstrap chicken-and-egg.** `_handle_network_assign` handles an empty
+   registry by falling back to the first *deployable* recommended preset — but
+   deployability is computed from the registered pool only
+   (`_deployment_summary(all_nodes, preset)`), which is empty, so nothing is
+   ever deployable and the first node always gets 503. The caller's own
+   `vram_mb` / `ram_mb` (already sent in the query) are ignored.
+2. **Tracker ignores `.env`.** `meshnet-node` loads `.env` (cwd) and
+   `~/.config/meshnet/secrets.env` at startup; `meshnet-tracker` does not. The
+   repo `.env` sets `MESHNET_DOWNLOAD_DIR=/run/media/popov/DATA/llm/safetensor/models`,
+   but the tracker only reads `--models-dir` / `MESHNET_MODELS_DIR`, so
+   `models_dir` stays unset → `/v1/model-files/download` returns
+   404 "tracker model-file source is not enabled" and assignments carry no
+   `model_sources`.
+3. **Symlink tars (found while verifying).** HF cache snapshots are symlink
+   farms into `blobs/`; both the tracker's `/v1/model-files/download` and the
+   node's `write_shard_archive` tarred the symlinks themselves
+   (`tarfile` default `dereference=False`), so receivers extracted dangling
+   links instead of weights.
+
+## Fix
+
+1. In the empty-registry branch of `_handle_network_assign`, synthesize a
+   candidate `_NodeEntry` from the caller's `vram_mb`/`ram_mb` query params and
+   include it in the pool used for the deployability gate (and the reported
+   `deployment` summary), so a recommended preset that fits *pool + caller*
+   bootstraps the network.
+2. Tracker CLI loads env defaults the same way the node CLI does
+   (`.env` in cwd, then `~/.config/meshnet/secrets.env`, never overriding
+   already-set env vars).
+3. `TrackerServer` models-dir resolution falls back
+   `--models-dir` → `MESHNET_MODELS_DIR` → `MESHNET_DOWNLOAD_DIR` (the node
+   store and tracker source are the same directory on a box running both).
+4. Archive with `dereference=True` in both tar writers so model file contents
+   ship instead of snapshot symlinks.
+
+## Acceptance criteria
+
+- [x] Fresh tracker (empty registry) + caller with enough memory for a
+  recommended preset → `/v1/network/assign` returns 200 with that preset,
+  `gap_found=true`, and `model_sources` populated when the tracker holds a
+  local snapshot. (Verified live: 128 GB caller got qwen3.6-35b-a3b 0–39
+  with tracker `model_sources`.)
+- [x] Fresh tracker + caller too small for any recommended preset → still 503.
+  (Verified live with a 4 GB caller.)
+- [x] `meshnet-tracker start` in a directory with `.env` setting
+  `MESHNET_DOWNLOAD_DIR` serves `/v1/model-files/download` from that dir with
+  no extra flags. (Verified live; tar entries are regular files, not
+  symlinks.)
+- [x] Explicit `--models-dir` and `MESHNET_MODELS_DIR` still take precedence,
+  in that order.
+- [x] `python -m pytest` passes from repo root (two known env-dependent
+  failures occur only while a live meshnet-node holds port 7000).

docs/issues/47-model-source-download-visibility.md

@@ -0,0 +1,115 @@
+# US-047 — Tracker-first model downloads: visibility, sane timeouts, RAM-based sizing
+
+Status: in progress
+Priority: High (follow-up to US-044/US-046; blocks usable LAN downloads)
+
+## Context
+
+Reported 2026-07-06 (Windows CPU node, 79.2 GB RAM, `--tracker
+http://192.168.0.179:8080 --model Qwen3.6-35B-A3B`):
+
+1. Startup prints `(auto-join unavailable: HTTP Error 503)` even though the
+   user explicitly named a model. The auto-join query (`/v1/network/assign`)
+   never sends the requested model, so a fresh tracker + a caller too small
+   for the *recommended* preset 503s (expected per US-046) — but the whole
+   auto-join step is pointless when the user already picked a model: the
+   `/v1/nodes/assign?model=…` call right after it succeeds (assigned layers
+   0–2 with tracker `model_sources`).
+2. The tracker-vs-HuggingFace race then starts, but only HuggingFace shows
+   progress (hf tqdm bars). The tracker tar download prints nothing and
+   swallows every failure (`except Exception: return None`), so the node
+   *appears* to download only from slow HF; the user killed it. Tracker-side
+   log showed the tar stream reset mid-`archive.add` — with no way to tell
+   whether the client timed out or the user aborted.
+3. `_download_model_source` inherits `peer_timeout` (2.0 s) as its urlopen
+   socket timeout. Any 2 s read stall during a multi-GB tar stream silently
+   kills the tracker source and leaves HF as the only contender.
+4. Every client abort spams the tracker console with a full
+   `BrokenPipeError`/`ConnectionResetError` traceback from `socketserver`.
+
+## Fix
+
+1. `startup.py`: skip the network auto-join query entirely when a model was
+   explicitly requested (`model` set and not `"stub-model"`); path 3b
+   (`/v1/nodes/assign?model=…`) is the authoritative one there.
+2. `downloader.py`: model-source downloads get their own timeout constant
+   (30 s socket timeout) instead of the 2 s peer-probe timeout. Peer shard
+   downloads keep 2 s — they run sequentially before the race, and a dead
+   peer must not hang startup for 30 s; the race is concurrent so a slow
+   source costs nothing.
+3. `downloader.py`: progress + failure visibility for the race —
+   `_download_model_source` prints received bytes every 512 MB and prints
+   the exception when a source fails, so "downloads only from HF" can never
+   happen silently again.
+4. Tracker `_handle_model_files_download`: catch
+   `BrokenPipeError`/`ConnectionResetError` around the tar stream and log a
+   single line instead of a traceback.
+
+## Design revision (2026-07-06, after live retest)
+
+The race is gone. User decision: **HuggingFace is used only when the model is
+not available from a tracker/peer source, or when `--tracker-source-disabled`
+is passed.** Sources are tried sequentially with progress + failure output;
+HF (layer-filtered via the source file list, else the remote index) is the
+fallback.
+
+Second live finding: the node was assigned only layers 0–2 of 40 on a 79 GB
+box. Cause: CPU-mode nodes still report the detected-but-unusable GPU's
+`vram_mb` (RTX 4060 → 8192), and shard sizing used VRAM whenever it was > 0
+(8 GB × 0.8 ≈ 6.5 GB ≈ 3 layers). Fixed on both sides: the node now sends
+`assignment_vram_mb` (0 unless CUDA is actually usable) to `/v1/nodes/assign`,
+and the tracker only trusts `vram_mb` when `device=cuda` (all three sizing
+sites), falling back to `ram_mb`.
+
+## Acceptance criteria
+
+- [x] Node started with an explicit `--model` never queries
+  `/v1/network/assign` and never prints `auto-join unavailable`.
+- [x] Tracker/peer model source is preferred outright; HF is contacted only
+  when no source is advertised, every source fails, or
+  `--tracker-source-disabled` is passed (flag on both CLI parsers, plumbed
+  through config and `run_startup`).
+- [x] Tracker-source downloads print progress every 512 MB and print the
+  exception + URL on failure; nothing fails silently.
+- [x] A ≥2 s read stall no longer aborts a tracker model-source download
+  (30 s socket timeout).
+- [x] Client disconnect during `/v1/model-files/download` logs one line on
+  the tracker, no traceback.
+- [x] CPU node with big RAM gets a RAM-sized shard: `/v1/nodes/assign` and
+  both `/v1/network/assign` sizing paths ignore VRAM unless `device=cuda`.
+- [x] `pytest tests/test_node_startup.py tests/test_tracker_routing.py`
+  passes (139/140; the one failure is the pre-existing port-dependent
+  `test_mining_cli` case, present on clean master).
+- [ ] Live two-machine retest: Windows node downloads only from tracker at
+  LAN speed and is assigned a RAM-sized shard.
+
+## Round 3 (2026-07-06, after live retest showed mid-stream RST)
+
+Live retest: RAM sizing worked (layers 0–36) and the failure finally printed —
+`ConnectionResetError(10054)` ~70 s into the tar stream. Local reproduction
+cleared the tracker: it streams the full 72 GB tar at ~900 MB/s, survives a
+3-minute slow reader, and logs aborts in one line. The RST comes from the
+network path (Windows laptop, likely WiFi + firewall/AV) — and a 72 GB
+single-TCP-stream tar is inherently fragile there.
+
+Fix: per-file downloads (design principle: nodes must be able to fetch any
+missing shard or the complete model from the tracker alone — no hard HF
+dependency):
+
+- Tracker: `/v1/model-files/download?...&file=<rel>` streams one file with
+  `Content-Length` (rel must be in the requested shard/full set; traversal
+  rejected). `model_sources` now advertises `full_files` and a `file_sizes`
+  manifest.
+- Node: `_download_source_files` fetches per file into
+  `<shard>.partial/`, retries each file 3×, verifies against
+  `Content-Length`, and reuses already-complete files (hardlink from the
+  existing shard) via the size manifest — so restarts and drops cost at most
+  one file. Tar stream remains the fallback for old trackers
+  (detected via Content-Type) and sources without a file list.
+- `_full_model_sources` passes `full_files` through, so full-snapshot
+  downloads for the torch path get the same robustness.
+
+Verified live against a local tracker: 14.7 GB shard in 7.6 s per-file;
+re-run over a complete shard instant; corrupt + deleted file recovered in
+1.5 s re-fetching only those two. 114 tests pass (node_startup +
+tracker_routing).

packages/node/meshnet_node/cli.py

@@ -3,12 +3,45 @@
 from __future__ import annotations
 
 import argparse
+import os
 import socket
 import sys
 import time
 from pathlib import Path
 
 
+def _load_env_file(path: Path) -> None:
+    """Load simple KEY=VALUE pairs from an env file without overriding env vars."""
+    if not path.exists():
+        return
+    try:
+        lines = path.read_text().splitlines()
+    except OSError:
+        return
+    for line in lines:
+        text = line.strip()
+        if not text or text.startswith("#"):
+            continue
+        if text.startswith("export "):
+            text = text[len("export "):].strip()
+        if "=" not in text:
+            continue
+        key, value = text.split("=", 1)
+        key = key.strip()
+        if not key or key in os.environ:
+            continue
+        value = value.strip()
+        if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
+            value = value[1:-1]
+        os.environ[key] = value
+
+
+def _load_env_defaults() -> None:
+    """Load local and user-level node env defaults before config defaults are imported."""
+    _load_env_file(Path.cwd() / ".env")
+    _load_env_file(Path.home() / ".config" / "meshnet" / "secrets.env")
+
+
 def _run_node(cfg: dict) -> None:
     """Start the node and hand off to the live dashboard. Blocks until Ctrl-C."""
     from .startup import run_startup
@@ -19,11 +52,11 @@ def _run_node(cfg: dict) -> None:
         node = run_startup(
             tracker_url=cfg["tracker_url"],
             port=cfg.get("port", 7000),
-            model=cfg.get("model_name") or "stub-model",
+            model=cfg.get("model_name") or None,
             model_id=cfg.get("model_hf_repo") or None,
             shard_start=cfg.get("shard_start"),
             shard_end=cfg.get("shard_end"),
-            quantization=cfg.get("quantization", "int8").replace("bf16", "bfloat16"),
+            quantization=cfg.get("quantization", "auto").replace("bf16", "bfloat16"),
             wallet_path=Path(cfg["wallet_path"]) if cfg.get("wallet_path") else None,
             cache_dir=Path(cfg["download_dir"]) if cfg.get("download_dir") else None,
             host=cfg.get("host", "0.0.0.0"),
@@ -32,6 +65,9 @@ def _run_node(cfg: dict) -> None:
             vram_mb_override=cfg.get("vram_mb_override"),
             max_loaded_shards=int(cfg.get("max_loaded_shards", 1)),
             debug=bool(cfg.get("debug", False)),
+            tracker_source_disabled=bool(cfg.get("tracker_source_disabled", False)),
+            torch_threads=cfg.get("torch_threads"),
+            torch_interop_threads=cfg.get("torch_interop_threads"),
         )
     except Exception as exc:
         print(f"\nERROR: {exc}", file=sys.stderr, flush=True)
@@ -54,6 +90,19 @@ def _run_node(cfg: dict) -> None:
         )
 
 
+def _resolve_model_flags(
+    model: str | None,
+    model_id: str | None,
+) -> tuple[str | None, str | None]:
+    """Return (model_name, hf_repo_or_none) from --model / --model-id flags."""
+    explicit = model_id or model
+    if not explicit:
+        return None, None
+    if "/" in explicit:
+        return explicit.split("/")[-1], explicit
+    return explicit, None
+
+
 def _first_available_port(host: str, start: int = 7000, attempts: int = 100) -> int:
     """Return the first TCP port bindable on host, starting at start."""
     bind_host = "" if host == "0.0.0.0" else host
@@ -86,9 +135,10 @@ def _cmd_default(args) -> int:
 
     # Apply CLI overrides on top of saved config
     overrides: dict = {}
-    if args.model:
-        overrides["model_hf_repo"] = args.model
-        overrides["model_name"] = args.model.split("/")[-1]
+    model_name, hf_repo = _resolve_model_flags(args.model, getattr(args, "model_id", None))
+    if model_name is not None:
+        overrides["model_name"] = model_name
+        overrides["model_hf_repo"] = hf_repo or ""
     if args.quantization:
         overrides["quantization"] = args.quantization
     if args.download_dir:
@@ -115,6 +165,12 @@ def _cmd_default(args) -> int:
         overrides["max_loaded_shards"] = args.max_shards
     if args.debug:
         overrides["debug"] = True
+    if getattr(args, "tracker_source_disabled", False):
+        overrides["tracker_source_disabled"] = True
+    if getattr(args, "torch_threads", None) is not None:
+        overrides["torch_threads"] = args.torch_threads
+    if getattr(args, "torch_interop_threads", None) is not None:
+        overrides["torch_interop_threads"] = args.torch_interop_threads
 
     if overrides:
         cfg = merge_cli_overrides(cfg, **overrides)
@@ -166,21 +222,22 @@ def _cmd_config(args) -> int:
 
 def _cmd_start(args) -> int:
     """Legacy `start` subcommand — preserves backward compatibility with existing tests."""
-    from .config import load_config, DEFAULTS
+    from .config import DEFAULTS
 
     # Build a transient config from flags (don't write to disk)
     cfg = dict(DEFAULTS)
-    cfg["tracker_url"] = args.tracker
+    if args.tracker:
+        cfg["tracker_url"] = args.tracker
     cfg["port"] = args.port if args.port is not None else _first_available_port(args.host)
-    if args.model_id is None and "/" in args.model:
-        cfg["model_hf_repo"] = args.model
-        cfg["model_name"] = args.model.split("/")[-1]
-    else:
-        cfg["model_name"] = args.model
+    model_name, hf_repo = _resolve_model_flags(
+        args.model or cfg.get("model_hf_repo") or cfg.get("model_name") or None,
+        args.model_id,
+    )
+    if model_name is not None:
+        cfg["model_name"] = model_name
+        cfg["model_hf_repo"] = hf_repo or ""
     cfg["quantization"] = args.quantization
     cfg["host"] = args.host
-    if args.model_id:
-        cfg["model_hf_repo"] = args.model_id
     if args.shard_start is not None:
         cfg["shard_start"] = args.shard_start
     if args.shard_end is not None:
@@ -198,7 +255,7 @@ def _cmd_start(args) -> int:
             tracker_url=cfg["tracker_url"],
             port=cfg["port"],
             model=cfg["model_name"],
-            model_id=cfg.get("model_hf_repo"),
+            model_id=cfg.get("model_hf_repo") or None,
             shard_start=cfg.get("shard_start"),
             shard_end=cfg.get("shard_end"),
             quantization=cfg["quantization"].replace("bf16", "bfloat16"),
@@ -210,6 +267,9 @@ def _cmd_start(args) -> int:
             vram_mb_override=getattr(args, "memory", None),
             max_loaded_shards=getattr(args, "max_shards", 1),
             debug=getattr(args, "debug", False),
+            tracker_source_disabled=getattr(args, "tracker_source_disabled", False),
+            torch_threads=getattr(args, "torch_threads", None),
+            torch_interop_threads=getattr(args, "torch_interop_threads", None),
         )
     except Exception as exc:
         print(f"ERROR: {exc}", file=sys.stderr, flush=True)
@@ -224,6 +284,8 @@ def _cmd_start(args) -> int:
 
 
 def main() -> None:
+    _load_env_defaults()
+
     parser = argparse.ArgumentParser(
         prog="meshnet-node",
         description="Distributed AI Inference — Node Client",
@@ -239,11 +301,14 @@ def main() -> None:
     )
 
     # Flags that apply to the no-subcommand (default) path
-    parser.add_argument("--model", metavar="HF_REPO", help="HuggingFace repo ID to serve")
+    parser.add_argument("--model", metavar="MODEL", help="Model name or HuggingFace repo ID to serve")
+    parser.add_argument("--model-id", metavar="MODEL", help="Alias for --model (catalog name or HuggingFace repo)")
     parser.add_argument("--quantization", "-q", choices=["bf16", "int8", "nf4", "bfloat16"],
                         help="Quantization level")
     parser.add_argument("--download-dir", metavar="PATH", help="Model download directory")
     parser.add_argument("--tracker", metavar="URL", help="Tracker URL")
+    parser.add_argument("--tracker-source-disabled", action="store_true",
+                        help="Skip tracker/peer model-file sources and download from HuggingFace directly")
     parser.add_argument("--wallet", metavar="PATH", help="Wallet file path")
     parser.add_argument("--shard-start", type=int, metavar="N", help="Pin shard start layer")
     parser.add_argument("--shard-end", type=int, metavar="N", help="Pin shard end layer")
@@ -256,6 +321,10 @@ def main() -> None:
                         help="Override autodetected VRAM/RAM budget in MB used for shard assignment")
     parser.add_argument("--max-shards", type=int, metavar="N", default=None,
                         help="Maximum shard slots this node advertises to the tracker (default 1)")
+    parser.add_argument("--torch-threads", type=int, metavar="N",
+                        help="Set PyTorch intra-op CPU worker threads")
+    parser.add_argument("--torch-interop-threads", type=int, metavar="N",
+                        help="Set PyTorch inter-op CPU worker threads")
     parser.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
     parser.add_argument("--no-tui", action="store_true", help="Plain-text output (no rich dashboard)")
     parser.add_argument("--compact", action="store_true", help="Single-line status output")
@@ -272,13 +341,13 @@ def main() -> None:
 
     # start subcommand (legacy / backward-compat)
     start_cmd = subparsers.add_parser("start", help="Start node (legacy flags)")
-    start_cmd.add_argument("--tracker", default="http://localhost:8080")
+    start_cmd.add_argument("--tracker")
     start_cmd.add_argument("--port", type=int)
-    start_cmd.add_argument("--model", default="stub-model")
-    start_cmd.add_argument("--model-id", help="HuggingFace repo ID")
+    start_cmd.add_argument("--model", help="Model name or HuggingFace repo ID")
+    start_cmd.add_argument("--model-id", help="Alias for --model (catalog name or HuggingFace repo)")
     start_cmd.add_argument("--shard-start", type=int)
     start_cmd.add_argument("--shard-end", type=int)
-    start_cmd.add_argument("--quantization", choices=["bfloat16", "int8", "nf4", "bf16"], default="int8")
+    start_cmd.add_argument("--quantization", choices=["auto", "bfloat16", "int8", "nf4", "bf16"], default="auto")
     start_cmd.add_argument("--host", default="0.0.0.0")
     start_cmd.add_argument("--advertise-host")
     start_cmd.add_argument("--tracker-mode", action="store_true")
@@ -291,7 +360,13 @@ def main() -> None:
                            help="Override autodetected VRAM/RAM budget in MB used for shard assignment")
     start_cmd.add_argument("--max-shards", type=int, default=1, metavar="N",
                            help="Maximum shard slots this node advertises to the tracker (default 1)")
+    start_cmd.add_argument("--torch-threads", type=int, metavar="N",
+                           help="Set PyTorch intra-op CPU worker threads")
+    start_cmd.add_argument("--torch-interop-threads", type=int, metavar="N",
+                           help="Set PyTorch inter-op CPU worker threads")
     start_cmd.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
+    start_cmd.add_argument("--tracker-source-disabled", action="store_true",
+                           help="Skip tracker/peer model-file sources and download from HuggingFace directly")
 
     args = parser.parse_args()
 

packages/node/meshnet_node/config.py

@@ -9,14 +9,21 @@ from pathlib import Path
 
 _DEFAULT_CONFIG_DIR = Path.home() / ".config" / "meshnet"
 _DEFAULT_CONFIG_FILE = _DEFAULT_CONFIG_DIR / "config.json"
-_DEFAULT_DOWNLOAD_DIR = Path.home() / ".meshnet" / "models"
-_DEFAULT_TRACKER_URL = "http://localhost:8080"
+# MESHNET_DOWNLOAD_DIR overrides the model store for every node on this
+# machine (all CLI flows fall back to this default; --download-dir still wins).
+_DEFAULT_DOWNLOAD_DIR = Path(
+    os.environ.get("MESHNET_DOWNLOAD_DIR", str(Path.home() / ".meshnet" / "models"))
+)
+_DEFAULT_TRACKER_URL = os.environ.get("MESHNET_TRACKER_URL", "http://localhost:8080")
 _DEFAULT_WALLET_PATH = str(Path.home() / ".config" / "meshnet" / "wallet.json")
-_DEFAULT_QUANTIZATION = "nf4"
+_DEFAULT_QUANTIZATION = "auto"
+_DEFAULT_MODEL = os.environ.get("MESHNET_MODEL_ID") or os.environ.get("MESHNET_MODEL", "")
+_DEFAULT_MODEL_HF_REPO = _DEFAULT_MODEL if "/" in _DEFAULT_MODEL else ""
+_DEFAULT_MODEL_NAME = _DEFAULT_MODEL.split("/")[-1] if "/" in _DEFAULT_MODEL else _DEFAULT_MODEL
 
 DEFAULTS = {
-    "model_hf_repo": "",
-    "model_name": "",
+    "model_hf_repo": _DEFAULT_MODEL_HF_REPO,
+    "model_name": _DEFAULT_MODEL_NAME,
     "quantization": _DEFAULT_QUANTIZATION,
     "download_dir": str(_DEFAULT_DOWNLOAD_DIR),
     "tracker_url": _DEFAULT_TRACKER_URL,

packages/node/meshnet_node/downloader.py

@@ -1,6 +1,6 @@
-"""Shard downloader — fetches model shards from peers or HuggingFace Hub.
+"""Shard downloader — fetches model files from peers, tracker sources, or HuggingFace.
 
-Cache layout: ~/.cache/meshnet/shards/<model>/layers_<start>-<end>/
+Cache layout: ~/.cache/meshnet/shards/<model>/
 
 For "stub-model" (no HF repo), a placeholder JSON file is written so the
 test suite never touches the network.
@@ -8,9 +8,11 @@ test suite never touches the network.
 
 import hashlib
 import json
+import os
 import shutil
 import tarfile
 import tempfile
+import time
 import urllib.parse
 import urllib.request
 from pathlib import Path
@@ -18,6 +20,12 @@ from typing import Any
 
 _DEFAULT_CACHE = Path.home() / ".cache" / "meshnet" / "shards"
 _PEER_TIMEOUT_SECONDS = 2.0
+# Model-source tar streams are multi-GB; a short socket timeout must not kill
+# them on a transient read stall. Peer probes keep the short timeout because
+# they run sequentially before the race and may hit dead endpoints.
+_MODEL_SOURCE_TIMEOUT_SECONDS = 30.0
+_PROGRESS_INTERVAL_BYTES = 512 * 1024 * 1024
+_FILE_RETRY_ATTEMPTS = 3
 
 
 def compute_shard_checksum(shard_dir: Path) -> str:
@@ -36,7 +44,8 @@ def compute_shard_checksum(shard_dir: Path) -> str:
 
 def write_shard_archive(shard_dir: Path, out_file: Any) -> None:
     """Write a tar archive for *shard_dir* to a binary file-like object."""
-    with tarfile.open(fileobj=out_file, mode="w|") as archive:
+    # dereference: HF cache snapshots are symlinks into blobs/ — ship contents.
+    with tarfile.open(fileobj=out_file, mode="w|", dereference=True) as archive:
         for path in sorted(p for p in shard_dir.rglob("*") if p.is_file()):
             archive.add(path, arcname=path.relative_to(shard_dir).as_posix())
 
@@ -97,14 +106,340 @@ def _download_shard_from_peer(
             _safe_extract_shard(archive_path, extract_dir)
             if compute_shard_checksum(extract_dir) != checksum:
                 return False
-            if shard_dir.exists():
-                shutil.rmtree(shard_dir)
-            shutil.move(str(extract_dir), str(shard_dir))
+            _merge_tree(extract_dir, shard_dir)
             return True
         except Exception:
             return False
 
 
+class _TarOnlySource(Exception):
+    """The server ignored ?file= and streamed a tar — no single-file support."""
+
+
+def _fetch_source_file(
+    file_url: str,
+    dest: Path,
+    timeout: float,
+    on_chunk=None,
+) -> int:
+    """Download one file; skip if *dest* already matches the remote size.
+
+    Returns the file's final size. Raises on any failure, including a
+    short read against the server's Content-Length.
+    """
+    with urllib.request.urlopen(file_url, timeout=timeout) as resp:
+        ctype = resp.getheader("Content-Type") if hasattr(resp, "getheader") else None
+        if isinstance(ctype, str) and "x-tar" in ctype:
+            raise _TarOnlySource(file_url)
+        length = resp.getheader("Content-Length") if hasattr(resp, "getheader") else None
+        expected = int(length) if isinstance(length, str) and length.isdigit() else None
+        if expected is not None and dest.exists() and dest.stat().st_size == expected:
+            if on_chunk is not None:
+                on_chunk(expected)
+            return expected  # complete from an earlier attempt/run
+        received = 0
+        with dest.open("wb") as out:
+            while True:
+                chunk = resp.read(1024 * 1024)
+                if not chunk:
+                    break
+                out.write(chunk)
+                received += len(chunk)
+                if on_chunk is not None:
+                    on_chunk(len(chunk))
+        if expected is not None and received != expected:
+            raise OSError(f"short read: got {received} of {expected} bytes")
+        return received
+
+
+class _SourceProgress:
+    """tqdm bar over the whole per-file download (total / speed / ETA),
+    falling back to plain per-file prints when tqdm is unavailable."""
+
+    def __init__(self, enabled: bool, label: str, total_bytes: int | None):
+        self._label = label
+        self._bar = None
+        self._enabled = enabled
+        if enabled and total_bytes:
+            try:
+                from tqdm import tqdm  # type: ignore[import]
+
+                self._bar = tqdm(
+                    total=total_bytes,
+                    unit="B",
+                    unit_scale=True,
+                    unit_divisor=1024,
+                    desc=f"Downloading ({label})",
+                )
+            except Exception:
+                self._bar = None
+
+    def add_bytes(self, n: int) -> None:
+        if self._bar is not None:
+            self._bar.update(n)
+
+    def rewind(self, n: int) -> None:
+        """Roll the bar back after a failed attempt so retries don't double-count."""
+        if self._bar is not None and n:
+            self._bar.update(-n)
+
+    def file_done(self, index: int, count: int, rel: str, size: int, reused: bool) -> None:
+        note = "already complete, " if reused else ""
+        line = f"  {self._label}: [{index}/{count}] {rel} ({note}{size / 1e9:.2f} GB)"
+        if self._bar is not None:
+            self._bar.set_postfix_str(f"{index}/{count} files", refresh=False)
+        elif self._enabled:
+            print(line, flush=True)
+
+    def message(self, text: str) -> None:
+        if self._bar is not None:
+            self._bar.write(text)
+        elif self._enabled:
+            print(text, flush=True)
+
+    def close(self) -> None:
+        if self._bar is not None:
+            self._bar.close()
+
+
+def _reuse_local_file(expected: int, dest: Path, final: Path) -> bool:
+    """Reuse an already-complete copy of a file instead of re-downloading."""
+    if dest.exists() and dest.stat().st_size == expected:
+        return True
+    if final.exists() and final.stat().st_size == expected:
+        try:
+            os.link(final, dest)
+        except OSError:
+            shutil.copy2(final, dest)
+        return True
+    return False
+
+
+def _valid_source_rel_files(source: dict) -> list[str]:
+    return [
+        rel for rel in (source.get("files") or [])
+        if isinstance(rel, str) and rel and not rel.startswith("/") and ".." not in Path(rel).parts
+    ]
+
+
+def _source_files_cached(source: dict, shard_dir: Path) -> bool:
+    rel_files = _valid_source_rel_files(source)
+    if not rel_files:
+        return False
+    sizes = source.get("file_sizes")
+    if not isinstance(sizes, dict):
+        return False
+    for rel in rel_files:
+        expected = sizes.get(rel)
+        path = shard_dir / rel
+        if not isinstance(expected, int) or not path.exists() or path.stat().st_size != expected:
+            return False
+    return True
+
+
+def _merge_tree(src: Path, dest: Path) -> None:
+    dest.mkdir(parents=True, exist_ok=True)
+    for path in sorted(p for p in src.rglob("*") if p.is_file()):
+        target = dest / path.relative_to(src)
+        target.parent.mkdir(parents=True, exist_ok=True)
+        if target.exists():
+            target.unlink()
+        shutil.move(str(path), str(target))
+
+
+def _download_source_files(
+    source: dict,
+    shard_dir: Path,
+    timeout: float,
+    progress: bool,
+    label: str,
+) -> Path | None:
+    """Per-file download from a model source — retries and resumes per file.
+
+    Far more robust than one multi-GB tar stream on flaky links: a dropped
+    connection costs at most one file, and completed files are skipped when
+    the download is retried or the node is restarted.
+    """
+    url = source.get("url")
+    rel_files = _valid_source_rel_files(source)
+    if not isinstance(url, str) or not url or not rel_files:
+        return None
+    sizes = source.get("file_sizes")
+    if not isinstance(sizes, dict):
+        sizes = {}
+    partial_dir = shard_dir.parent / f"{shard_dir.name}.partial"
+    partial_dir.mkdir(parents=True, exist_ok=True)
+    sep = "&" if "?" in url else "?"
+    known_sizes = [sizes.get(rel) for rel in rel_files]
+    total_bytes = sum(s for s in known_sizes if isinstance(s, int)) if all(
+        isinstance(s, int) for s in known_sizes
+    ) else None
+    tracker_bar = _SourceProgress(progress, label, total_bytes)
+    try:
+        for index, rel in enumerate(rel_files, start=1):
+            dest = partial_dir / rel
+            dest.parent.mkdir(parents=True, exist_ok=True)
+            known_size = sizes.get(rel)
+            if isinstance(known_size, int) and _reuse_local_file(known_size, dest, shard_dir / rel):
+                tracker_bar.add_bytes(known_size)
+                tracker_bar.file_done(index, len(rel_files), rel, known_size, reused=True)
+                continue
+            file_url = f"{url}{sep}{urllib.parse.urlencode({'file': rel})}"
+            for attempt in range(1, _FILE_RETRY_ATTEMPTS + 1):
+                counted = 0
+
+                def _on_chunk(n: int) -> None:
+                    nonlocal counted
+                    counted += n
+                    tracker_bar.add_bytes(n)
+
+                try:
+                    size = _fetch_source_file(file_url, dest, timeout, on_chunk=_on_chunk)
+                    tracker_bar.file_done(index, len(rel_files), rel, size, reused=False)
+                    break
+                except _TarOnlySource:
+                    tracker_bar.message(f"  {label}: no single-file support — using tar stream")
+                    return None
+                except Exception as exc:
+                    tracker_bar.rewind(counted)
+                    tracker_bar.message(
+                        f"  {label}: {rel} attempt {attempt}/{_FILE_RETRY_ATTEMPTS} failed: {exc!r}"
+                    )
+                    if attempt == _FILE_RETRY_ATTEMPTS:
+                        return None
+                    time.sleep(1.0 * attempt)
+    finally:
+        tracker_bar.close()
+    _merge_tree(partial_dir, shard_dir)
+    try:
+        partial_dir.rmdir()
+    except OSError:
+        pass
+    return shard_dir
+
+
+def _download_model_source(
+    source: dict,
+    shard_dir: Path,
+    timeout: float,
+    progress: bool = False,
+    label: str = "model-source",
+) -> Path | None:
+    # Prefer per-file transfers whenever the source advertises its file list;
+    # fall through to the tar stream if the server lacks single-file support
+    # or per-file transfers keep failing.
+    if source.get("files"):
+        fetched = _download_source_files(source, shard_dir, timeout, progress, label)
+        if fetched is not None:
+            return fetched
+    url = source.get("url")
+    if not isinstance(url, str) or not url:
+        endpoint = source.get("endpoint")
+        if not isinstance(endpoint, str):
+            return None
+        url = f"{endpoint.rstrip('/')}/v1/model-files/download"
+    shard_dir.parent.mkdir(parents=True, exist_ok=True)
+    with tempfile.TemporaryDirectory(prefix="meshnet-model-source-", dir=shard_dir.parent) as tmp:
+        tmp_root = Path(tmp)
+        archive_path = tmp_root / "model-files.tar"
+        extract_dir = tmp_root / "extract"
+        extract_dir.mkdir()
+        try:
+            received = 0
+            next_report = _PROGRESS_INTERVAL_BYTES
+            with urllib.request.urlopen(url, timeout=timeout) as resp, archive_path.open("wb") as out:
+                while True:
+                    chunk = resp.read(1024 * 1024)
+                    if not chunk:
+                        break
+                    out.write(chunk)
+                    received += len(chunk)
+                    if progress and received >= next_report:
+                        print(f"  {label}: {received / 1e9:.1f} GB received ...", flush=True)
+                        next_report += _PROGRESS_INTERVAL_BYTES
+            if progress:
+                print(f"  {label}: transfer complete ({received / 1e9:.2f} GB), extracting ...", flush=True)
+            _safe_extract_shard(archive_path, extract_dir)
+            _merge_tree(extract_dir, shard_dir)
+            return shard_dir
+        except Exception as exc:
+            if progress:
+                print(f"  {label}: download failed ({url}): {exc!r}", flush=True)
+            return None
+
+
+def _download_huggingface_subset(
+    hf_repo: str,
+    cache_dir: Path,
+    shard_dir: Path,
+    allow_patterns: list[str] | None,
+) -> Path:
+    from huggingface_hub import snapshot_download  # type: ignore[import]
+
+    kwargs = {
+        "repo_id": hf_repo,
+        "cache_dir": str(cache_dir),
+        "local_dir": str(shard_dir),
+    }
+    if allow_patterns:
+        kwargs["allow_patterns"] = allow_patterns
+    try:
+        return Path(snapshot_download(**kwargs))
+    except TypeError:
+        kwargs.pop("allow_patterns", None)
+        return Path(snapshot_download(**kwargs))
+
+
+def _allow_patterns_from_sources(model_sources: list[dict]) -> list[str] | None:
+    patterns: set[str] = set()
+    for source in model_sources:
+        for rel in source.get("files") or []:
+            if isinstance(rel, str) and rel and not rel.startswith("/") and ".." not in Path(rel).parts:
+                patterns.add(rel)
+    return sorted(patterns) if patterns else None
+
+
+def _allow_patterns_from_remote_index(
+    hf_repo: str,
+    cache_dir: Path,
+    shard_start: int,
+    shard_end: int,
+) -> list[str] | None:
+    """Fetch just the SafeTensors index + config (a few KB) from HF and compute
+    which weight files the assigned layer range needs, so a HuggingFace fallback
+    download stays layer-scoped even when the tracker has no model_sources
+    (e.g. it has no local snapshot for this repo cached yet)."""
+    try:
+        from huggingface_hub import hf_hub_download  # type: ignore[import]
+
+        from .safetensors_selection import (
+            INDEX_FILENAME,
+            METADATA_FILENAMES,
+            layers_from_config_dict,
+            select_files_for_layers_from_index,
+        )
+
+        index_path = hf_hub_download(repo_id=hf_repo, filename=INDEX_FILENAME, cache_dir=str(cache_dir))
+        weight_map = json.loads(Path(index_path).read_text(encoding="utf-8")).get("weight_map")
+    except Exception:
+        return None
+    if not isinstance(weight_map, dict):
+        return None
+
+    total_layers: int | None = None
+    try:
+        config_path = hf_hub_download(repo_id=hf_repo, filename="config.json", cache_dir=str(cache_dir))
+        config = json.loads(Path(config_path).read_text(encoding="utf-8"))
+        total_layers = layers_from_config_dict(config)
+    except Exception:
+        pass
+
+    selected = select_files_for_layers_from_index(
+        weight_map, shard_start, shard_end, total_layers=total_layers
+    )
+    return sorted(selected | METADATA_FILENAMES)
+
+
 def download_shard(
     model: str,
     shard_start: int,
@@ -113,6 +448,7 @@ def download_shard(
     hf_repo: str | None = None,
     progress: bool = True,
     peers: list[dict] | None = None,
+    model_sources: list[dict] | None = None,
     peer_timeout: float = _PEER_TIMEOUT_SECONDS,
 ) -> Path:
     """Ensure the shard is present in *cache_dir* and return its local path.
@@ -122,7 +458,16 @@ def download_shard(
     the test suite hermetic while the real download path is exercised by
     passing a non-stub *hf_repo*.
     """
-    shard_dir = cache_dir / model / f"layers_{shard_start}-{shard_end}"
+    shard_dir = cache_dir / model
+    if progress:
+        print(f"  Target location: {shard_dir}", flush=True)
+
+    for source in model_sources or []:
+        label = str(source.get("type") or "model-source")
+        if _source_files_cached(source, shard_dir):
+            if progress:
+                print(f"  [{label}] requested files already cached at {shard_dir}", flush=True)
+            return shard_dir
 
     for peer in peers or []:
         if progress:
@@ -157,18 +502,43 @@ def download_shard(
                 print(f"  [stub] shard already cached at {shard_dir}", flush=True)
         return shard_dir
 
-    from huggingface_hub import snapshot_download  # type: ignore[import]
-
     if progress:
         print(
             f"  Downloading layers {shard_start}-{shard_end} from {hf_repo} ...",
             flush=True,
         )
-        print("  download source: HuggingFace", flush=True)
+    # Tracker (or peer) model sources are preferred outright — usually LAN-fast.
+    # HuggingFace is only the fallback when every advertised source fails.
+    for source in model_sources or []:
+        label = str(source.get("type") or "model-source")
+        if progress:
+            print(f"  Downloading from {label} model source (HuggingFace is the fallback) ...", flush=True)
+        fetched = _download_model_source(
+            source,
+            shard_dir,
+            timeout=max(peer_timeout, _MODEL_SOURCE_TIMEOUT_SECONDS),
+            progress=progress,
+            label=label,
+        )
+        if fetched is not None:
+            if progress:
+                print(f"  download source: {label}", flush=True)
+            return fetched
+    if model_sources and progress:
+        print("  All model sources failed — falling back to HuggingFace ...", flush=True)
 
-    local_dir = snapshot_download(
-        repo_id=hf_repo,
-        cache_dir=str(cache_dir),
-        local_dir=str(shard_dir),
-    )
-    return Path(local_dir)
+    allow_patterns = None
+    if model_sources:
+        allow_patterns = _allow_patterns_from_sources(model_sources)
+    if allow_patterns is None:
+        allow_patterns = _allow_patterns_from_remote_index(hf_repo, cache_dir, shard_start, shard_end)
+    if progress:
+        if allow_patterns:
+            print("  download source: HuggingFace (layer-filtered)", flush=True)
+        else:
+            print(
+                "  download source: HuggingFace (full snapshot — no SafeTensors index found)",
+                flush=True,
+            )
+
+    return _download_huggingface_subset(hf_repo, cache_dir, shard_dir, allow_patterns)

packages/node/meshnet_node/model_backend.py

@@ -4,10 +4,11 @@ from __future__ import annotations
 
 import base64
 from dataclasses import dataclass
+import json
 from pathlib import Path
 from typing import Any, Literal
 
-Quantization = Literal["bfloat16", "int8", "nf4"]
+Quantization = Literal["auto", "bfloat16", "int8", "nf4"]
 
 
 class ModelBackendError(RuntimeError):
@@ -22,6 +23,10 @@ class InsufficientVRAMError(ModelBackendError):
     """Raised when a requested shard cannot fit in available CUDA memory."""
 
 
+class PartialModelLoadUnsupported(ModelBackendError):
+    """Raised when a shard cannot be materialized from a local snapshot subset."""
+
+
 @dataclass(frozen=True)
 class TensorPayload:
     body: bytes
@@ -31,14 +36,14 @@ class TensorPayload:
 
 
 def validate_quantization(value: str) -> Quantization:
-    if value not in {"bfloat16", "int8", "nf4"}:
-        raise ValueError("quantization must be one of: bfloat16, int8, nf4")
+    if value not in {"auto", "bfloat16", "int8", "nf4"}:
+        raise ValueError("quantization must be one of: auto, bfloat16, int8, nf4")
     return value  # type: ignore[return-value]
 
 
 def build_quantization_config(quantization: Quantization) -> Any | None:
     """Return a transformers BitsAndBytesConfig for quantized weights."""
-    if quantization == "bfloat16":
+    if quantization in {"auto", "bfloat16"}:
         return None
     try:
         import torch
@@ -65,7 +70,7 @@ class TorchModelShard:
         model_id: str,
         shard_start: int,
         shard_end: int,
-        quantization: Quantization = "bfloat16",
+        quantization: Quantization = "auto",
         cache_dir: Path | None = None,
     ) -> None:
         if shard_start < 0 or shard_end < 0 or shard_start > shard_end:
@@ -77,7 +82,7 @@ class TorchModelShard:
 
         try:
             import torch
-            from transformers import AutoModelForCausalLM, AutoTokenizer
+            from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
         except ModuleNotFoundError as exc:
             raise MissingModelDependencyError(
                 "real model backend requires torch, transformers, safetensors, accelerate, and bitsandbytes"
@@ -85,18 +90,48 @@ class TorchModelShard:
 
         self.torch = torch
         self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-        quant_config = build_quantization_config(quantization)
+        load_source = str(cache_dir) if cache_dir is not None and (cache_dir / "config.json").exists() else model_id
+        quant_config, dtype, uses_quantized_weights = _model_load_plan(
+            AutoConfig,
+            load_source,
+            quantization,
+            torch,
+            None if load_source != model_id else cache_dir,
+        )
         try:
-            self.model = AutoModelForCausalLM.from_pretrained(
-                model_id,
-                quantization_config=quant_config,
-                device_map="auto" if quant_config is not None else None,
-                dtype=torch.bfloat16,
-                low_cpu_mem_usage=True,
-                cache_dir=str(cache_dir) if cache_dir is not None else None,
-            )
-            if quant_config is None:
-                self.model.to(self.device)
+            total_layers_hint = _total_layers_for_local_snapshot(AutoConfig, load_source)
+            if _should_partial_materialize_shard(
+                load_source,
+                shard_start,
+                shard_end,
+                total_layers_hint=total_layers_hint,
+                uses_quantized_weights=uses_quantized_weights,
+            ):
+                self.model = _load_partial_model_from_snapshot(
+                    AutoConfig,
+                    AutoModelForCausalLM,
+                    torch,
+                    load_source,
+                    shard_start,
+                    shard_end,
+                    dtype,
+                    self.device,
+                )
+            else:
+                load_kwargs = {
+                    "device_map": "auto" if uses_quantized_weights else None,
+                    "dtype": dtype,
+                    "low_cpu_mem_usage": True,
+                    "cache_dir": str(cache_dir) if cache_dir is not None and load_source == model_id else None,
+                }
+                if quant_config is not None:
+                    load_kwargs["quantization_config"] = quant_config
+                self.model = AutoModelForCausalLM.from_pretrained(
+                    load_source,
+                    **load_kwargs,
+                )
+                if not uses_quantized_weights:
+                    self.model.to(self.device)
         except Exception as exc:
             if _looks_like_oom(exc):
                 raise InsufficientVRAMError(
@@ -107,8 +142,8 @@ class TorchModelShard:
 
         self.model.eval()
         self.tokenizer = AutoTokenizer.from_pretrained(
-            model_id,
-            cache_dir=str(cache_dir) if cache_dir is not None else None,
+            load_source,
+            cache_dir=str(cache_dir) if cache_dir is not None and load_source == model_id else None,
         )
         self.layers = _model_layers(self.model)
         self.total_layers = len(self.layers)
@@ -340,12 +375,200 @@ def load_torch_shard(
     model_id: str,
     shard_start: int,
     shard_end: int,
-    quantization: Quantization = "bfloat16",
+    quantization: Quantization = "auto",
     cache_dir: Path | None = None,
 ) -> TorchModelShard:
     return TorchModelShard(model_id, shard_start, shard_end, quantization, cache_dir)
 
 
+def _total_layers_for_local_snapshot(auto_config: Any, load_source: str) -> int | None:
+    snapshot_dir = Path(load_source)
+    if not (snapshot_dir / "config.json").exists():
+        return None
+    from .model_catalog import layers_from_config
+
+    try:
+        cfg = auto_config.from_pretrained(str(snapshot_dir))
+    except Exception:
+        return None
+    return layers_from_config(cfg)
+
+
+def _should_partial_materialize_shard(
+    load_source: str,
+    shard_start: int,
+    shard_end: int,
+    *,
+    total_layers_hint: int | None,
+    uses_quantized_weights: bool,
+) -> bool:
+    if uses_quantized_weights:
+        return False
+    snapshot_dir = Path(load_source)
+    if not snapshot_dir.exists() or not (snapshot_dir / "config.json").exists():
+        return False
+    if not (snapshot_dir / "model.safetensors.index.json").exists():
+        return False
+    if total_layers_hint is None:
+        return False
+    return not (shard_start == 0 and shard_end >= total_layers_hint - 1)
+
+
+def _load_partial_model_from_snapshot(
+    auto_config: Any,
+    auto_model_for_causal_lm: Any,
+    torch: Any,
+    load_source: str,
+    shard_start: int,
+    shard_end: int,
+    dtype: Any,
+    device: Any,
+    *,
+    init_empty_weights_fn: Any | None = None,
+    set_tensor_fn: Any | None = None,
+    safe_open_fn: Any | None = None,
+) -> Any:
+    from .model_catalog import layers_from_config
+    from .safetensors_selection import (
+        INDEX_FILENAME,
+        select_tensor_names_for_layers_from_index,
+    )
+
+    if init_empty_weights_fn is None:
+        from accelerate import init_empty_weights as init_empty_weights_fn
+    if set_tensor_fn is None:
+        from accelerate.utils import set_module_tensor_to_device as set_tensor_fn
+    if safe_open_fn is None:
+        from safetensors import safe_open as safe_open_fn
+
+    snapshot_dir = Path(load_source)
+    cfg = auto_config.from_pretrained(str(snapshot_dir))
+    total_layers = layers_from_config(cfg)
+    if total_layers is None:
+        raise PartialModelLoadUnsupported(
+            f"could not determine num_hidden_layers for local snapshot {snapshot_dir}"
+        )
+    if shard_end >= total_layers:
+        raise ValueError(
+            f"shard_end {shard_end} exceeds last layer index {total_layers - 1}"
+        )
+
+    index_path = snapshot_dir / INDEX_FILENAME
+    try:
+        index = json.loads(index_path.read_text(encoding="utf-8"))
+    except FileNotFoundError as exc:
+        raise PartialModelLoadUnsupported(
+            f"missing SafeTensors index for partial load: {index_path}"
+        ) from exc
+    weight_map = index.get("weight_map")
+    if not isinstance(weight_map, dict):
+        raise PartialModelLoadUnsupported(f"{INDEX_FILENAME} must contain a weight_map object")
+
+    tensor_names = select_tensor_names_for_layers_from_index(
+        weight_map,
+        shard_start,
+        shard_end,
+        total_layers=total_layers,
+    )
+    if not tensor_names:
+        raise PartialModelLoadUnsupported(
+            f"no checkpoint tensors matched layers {shard_start}-{shard_end} in {snapshot_dir}"
+        )
+
+    with init_empty_weights_fn():
+        model = auto_model_for_causal_lm.from_config(cfg, torch_dtype=dtype)
+    tie_weights = getattr(model, "tie_weights", None)
+    if callable(tie_weights):
+        tie_weights()
+
+    tensors_by_file: dict[str, list[str]] = {}
+    for tensor_name in sorted(tensor_names):
+        rel_file = weight_map.get(tensor_name)
+        if not isinstance(rel_file, str):
+            continue
+        tensors_by_file.setdefault(rel_file, []).append(tensor_name)
+
+    for rel_file, names in tensors_by_file.items():
+        checkpoint_file = snapshot_dir / rel_file
+        if not checkpoint_file.exists():
+            raise PartialModelLoadUnsupported(
+                f"checkpoint file advertised in {INDEX_FILENAME} is missing: {checkpoint_file}"
+            )
+        with safe_open_fn(str(checkpoint_file), framework="pt", device="cpu") as handle:
+            for tensor_name in names:
+                set_tensor_fn(
+                    model,
+                    tensor_name,
+                    device,
+                    value=handle.get_tensor(tensor_name),
+                    dtype=dtype,
+                )
+
+    for module in _active_modules_for_shard(model, shard_start, shard_end):
+        if hasattr(module, "to"):
+            module.to(device)
+    return model
+
+
+def _model_load_plan(
+    auto_config: Any,
+    model_id: str,
+    quantization: Quantization,
+    torch: Any,
+    cache_dir: Path | None = None,
+) -> tuple[Any | None, Any, bool]:
+    """Return (explicit quant config, dtype, uses quantized weights)."""
+    if quantization != "auto":
+        quant_config = build_quantization_config(quantization)
+        return quant_config, torch.bfloat16, quant_config is not None
+
+    cfg = auto_config.from_pretrained(
+        model_id,
+        cache_dir=str(cache_dir) if cache_dir is not None else None,
+    )
+    if _native_quantization_config(cfg) is not None:
+        return None, _native_torch_dtype(cfg, torch), True
+    return None, _native_torch_dtype(cfg, torch), False
+
+
+def _config_candidates(cfg: Any) -> list[Any]:
+    candidates = [cfg]
+    get_text_config = getattr(cfg, "get_text_config", None)
+    if callable(get_text_config):
+        try:
+            candidates.append(get_text_config())
+        except Exception:
+            pass
+    text_config = getattr(cfg, "text_config", None)
+    if text_config is not None:
+        candidates.append(text_config)
+    return candidates
+
+
+def _native_quantization_config(cfg: Any) -> Any | None:
+    for candidate in _config_candidates(cfg):
+        quant_config = getattr(candidate, "quantization_config", None)
+        if quant_config:
+            return quant_config
+    return None
+
+
+def _native_torch_dtype(cfg: Any, torch: Any) -> Any:
+    for candidate in _config_candidates(cfg):
+        for attr in ("dtype", "torch_dtype"):
+            dtype = getattr(candidate, attr, None)
+            if dtype is None:
+                continue
+            if isinstance(dtype, str):
+                dtype_name = dtype.removeprefix("torch.")
+                dtype_value = getattr(torch, dtype_name, None)
+                if dtype_value is not None:
+                    return dtype_value
+            else:
+                return dtype
+    return torch.bfloat16
+
+
 def _model_layers(model: Any) -> Any:
     if hasattr(model, "model") and hasattr(model.model, "layers"):
         return model.model.layers
@@ -372,6 +595,37 @@ def _position_embeddings(model: Any) -> Any | None:
     return None
 
 
+def _rotary_embedding_module(model: Any) -> Any | None:
+    if hasattr(model, "model") and hasattr(model.model, "rotary_emb"):
+        return model.model.rotary_emb
+    if hasattr(model, "transformer") and hasattr(model.transformer, "rotary_emb"):
+        return model.transformer.rotary_emb
+    return None
+
+
+def _active_modules_for_shard(model: Any, shard_start: int, shard_end: int) -> list[Any]:
+    active: list[Any] = []
+
+    def add(module: Any | None) -> None:
+        if module is None:
+            return
+        if any(existing is module for existing in active):
+            return
+        active.append(module)
+
+    if shard_start == 0:
+        add(_embed_tokens(model))
+        add(_position_embeddings(model))
+    add(_rotary_embedding_module(model))
+    for layer in _model_layers(model)[shard_start:shard_end + 1]:
+        add(layer)
+    total_layers = len(_model_layers(model))
+    if shard_end >= total_layers - 1:
+        add(_final_norm(model))
+        add(getattr(model, "lm_head", None))
+    return active
+
+
 def _final_norm(model: Any) -> Any | None:
     if hasattr(model, "model") and hasattr(model.model, "norm"):
         return model.model.norm
@@ -415,11 +669,7 @@ def _rotary_position_embeddings(model: Any, hidden_states: Any, position_ids: An
     """Return model-level rotary embeddings required by newer HF decoder layers."""
     if position_ids is None:
         return None
-    rotary = None
-    if hasattr(model, "model") and hasattr(model.model, "rotary_emb"):
-        rotary = model.model.rotary_emb
-    elif hasattr(model, "transformer") and hasattr(model.transformer, "rotary_emb"):
-        rotary = model.transformer.rotary_emb
+    rotary = _rotary_embedding_module(model)
     if rotary is None:
         return None
     return rotary(hidden_states, position_ids)

packages/node/meshnet_node/model_catalog.py

@@ -64,6 +64,17 @@ def _load_model_metadata() -> dict[str, dict]:
 _MODEL_METADATA = _load_model_metadata()
 
 
+def _local_model_path(hf_repo: str, cache_dir: Path | None) -> Path | None:
+    if cache_dir is None:
+        return None
+    if (cache_dir / "config.json").exists():
+        return cache_dir
+    candidate = cache_dir / hf_repo.split("/")[-1]
+    if (candidate / "config.json").exists():
+        return candidate
+    return None
+
+
 CURATED_MODELS: list[ModelPreset] = [
     ModelPreset(
         name="Qwen2.5-0.5B-Instruct",
@@ -159,6 +170,30 @@ CURATED_MODELS: list[ModelPreset] = [
 ]
 
 
+def layers_from_config(cfg) -> int | None:
+    """Extract the transformer layer count from a HuggingFace config object.
+
+    Composite configs (vision-language, some MoE) nest the decoder settings in
+    ``text_config`` — e.g. Qwen3.5-MoE has no top-level ``num_hidden_layers``.
+    """
+    candidates = [cfg]
+    get_text_config = getattr(cfg, "get_text_config", None)
+    if callable(get_text_config):
+        try:
+            candidates.append(get_text_config())
+        except Exception:
+            pass
+    nested = getattr(cfg, "text_config", None)
+    if nested is not None:
+        candidates.append(nested)
+    for candidate in candidates:
+        for attr in ("num_hidden_layers", "num_layers", "n_layer", "n_layers"):
+            value = getattr(candidate, attr, None)
+            if value is not None:
+                return int(value)
+    return None
+
+
 def detect_num_layers(hf_repo: str) -> int | None:
     """Return num_hidden_layers from HuggingFace config.json (downloads ~1 KB only)."""
     # Check curated list first (no network call)
@@ -168,7 +203,7 @@ def detect_num_layers(hf_repo: str) -> int | None:
     try:
         from transformers import AutoConfig  # type: ignore[import]
         cfg = AutoConfig.from_pretrained(hf_repo)
-        return int(cfg.num_hidden_layers)
+        return layers_from_config(cfg)
     except Exception:
         return None
 
@@ -191,10 +226,14 @@ def model_metadata_for(
     try:
         from transformers import AutoConfig  # type: ignore[import]
 
+        local_model = _local_model_path(hf_repo, cache_dir)
+        load_source = str(local_model) if local_model is not None else hf_repo
         cfg = AutoConfig.from_pretrained(
-            hf_repo,
-            cache_dir=str(cache_dir) if cache_dir is not None else None,
+            load_source,
+            cache_dir=str(cache_dir) if cache_dir is not None and local_model is None else None,
         )
+        # Composite configs (VLM/MoE) nest decoder fields in text_config.
+        text_cfg = getattr(cfg, "text_config", None) or cfg
         for attr, key in (
             ("model_type", "architecture"),
             ("num_hidden_layers", "num_layers"),
@@ -204,8 +243,14 @@ def model_metadata_for(
             ("max_position_embeddings", "context_length"),
         ):
             value = getattr(cfg, attr, None)
+            if value is None:
+                value = getattr(text_cfg, attr, None)
             if value is not None:
                 metadata[key] = value
+        if "num_layers" not in metadata:
+            layers = layers_from_config(cfg)
+            if layers is not None:
+                metadata["num_layers"] = layers
     except Exception:
         pass
     return metadata

packages/node/meshnet_node/relay_bridge.py

@@ -5,14 +5,18 @@ from __future__ import annotations
 import base64
 import json
 import logging
+import os
 import threading
 import time
 import urllib.error
 import urllib.request
+from concurrent.futures import ThreadPoolExecutor
 from dataclasses import dataclass
 
 log = logging.getLogger(__name__)
 
+DEFAULT_MAX_CONCURRENCY = 8
+
 
 @dataclass(frozen=True)
 class RelayBridgeInfo:
@@ -32,8 +36,23 @@ def _make_envelope(topic: str, payload: dict, peer_id: str) -> dict:
     }
 
 
+def _max_concurrency_from_env() -> int:
+    try:
+        value = int(os.environ.get("MESHNET_RELAY_CONCURRENCY", DEFAULT_MAX_CONCURRENCY))
+    except (TypeError, ValueError):
+        return DEFAULT_MAX_CONCURRENCY
+    return max(1, value)
+
+
 class RelayHttpBridge:
-    """Connect outbound to a relay and proxy relay HTTP requests to localhost."""
+    """Connect outbound to a relay and proxy relay HTTP requests to localhost.
+
+    Requests are dispatched on a bounded worker pool (US-037) so a node that is
+    the head of one route can still serve per-token ``/forward`` hops of another.
+    Streaming responses (``text/event-stream``) are forwarded as multiple chunk
+    frames sharing one ``request_id`` (US-036); everything else stays a single
+    frame for backward compatibility.
+    """
 
     def __init__(
         self,
@@ -42,15 +61,20 @@ class RelayHttpBridge:
         local_base_url: str,
         advertised_addr: str,
         reconnect_interval: float = 3.0,
+        max_concurrency: int | None = None,
     ) -> None:
         self.relay_url = relay_url.rstrip("/")
         self.peer_id = peer_id
         self.local_base_url = local_base_url.rstrip("/")
         self.advertised_addr = advertised_addr
         self.reconnect_interval = reconnect_interval
+        self.max_concurrency = max(1, max_concurrency) if max_concurrency else _max_concurrency_from_env()
         self._running = False
         self._thread: threading.Thread | None = None
         self._connected = threading.Event()
+        self._executor: ThreadPoolExecutor | None = None
+        self._send_lock = threading.Lock()
+        self._ws = None
 
     @property
     def relay_addr(self) -> str:
@@ -61,6 +85,9 @@ class RelayHttpBridge:
 
     def start(self) -> RelayBridgeInfo:
         self._running = True
+        self._executor = ThreadPoolExecutor(
+            max_workers=self.max_concurrency, thread_name_prefix="relay-http-worker"
+        )
         self._thread = threading.Thread(target=self._run, daemon=True, name="relay-http-bridge")
         self._thread.start()
         return RelayBridgeInfo(peer_id=self.peer_id, relay_addr=self.relay_addr)
@@ -72,6 +99,8 @@ class RelayHttpBridge:
         self._running = False
         if self._thread:
             self._thread.join(timeout=3.0)
+        if self._executor is not None:
+            self._executor.shutdown(wait=False)
 
     def _run(self) -> None:
         import websockets.sync.client as wsc  # type: ignore[import]
@@ -79,6 +108,7 @@ class RelayHttpBridge:
         while self._running:
             try:
                 with wsc.connect(self.relay_url, open_timeout=5) as ws:
+                    self._ws = ws
                     self._connected.set()
                     ws.send(json.dumps(_make_envelope(
                         "peer-register",
@@ -99,19 +129,36 @@ class RelayHttpBridge:
                         payload = envelope.get("payload", {})
                         if payload.get("target_peer") not in {None, self.peer_id}:
                             continue
-                        response = self._handle_request(payload)
-                        ws.send(json.dumps(_make_envelope(
-                            "relay-http-response",
-                            response,
-                            self.peer_id,
-                        )))
+                        if self._executor is None:
+                            break
+                        self._executor.submit(self._process_request, payload)
             except Exception as exc:
                 self._connected.clear()
+                self._ws = None
                 if self._running:
                     log.debug("relay bridge disconnected: %s", exc)
                     time.sleep(self.reconnect_interval)
+        self._ws = None
 
-    def _handle_request(self, payload: dict) -> dict:
+    def _send_response_frame(self, payload: dict) -> bool:
+        """Send one relay-http-response frame; False if the socket is gone.
+
+        The lock is held per frame so concurrent workers interleave whole
+        frames on the shared WebSocket, never torn ones.
+        """
+        ws = self._ws
+        if ws is None:
+            return False
+        message = json.dumps(_make_envelope("relay-http-response", payload, self.peer_id))
+        try:
+            with self._send_lock:
+                ws.send(message)
+            return True
+        except Exception as exc:
+            log.debug("relay bridge send failed (request orphaned): %s", exc)
+            return False
+
+    def _process_request(self, payload: dict) -> None:
         request_id = str(payload.get("request_id") or "")
         method = str(payload.get("method") or "POST").upper()
         path = str(payload.get("path") or "/")
@@ -130,10 +177,14 @@ class RelayHttpBridge:
         req = urllib.request.Request(url, data=data, headers=headers, method=method)
         try:
             with urllib.request.urlopen(req, timeout=300.0) as resp:
-                resp_bytes = resp.read()
                 resp_headers = dict(resp.headers)
+                content_type = resp.headers.get("Content-Type", "")
+                if "text/event-stream" in content_type:
+                    self._stream_response(request_id, resp, resp_headers)
+                    return
+                resp_bytes = resp.read()
                 # Forward all X-Meshnet-* headers so the caller can reconstruct the activation.
-                is_binary = "octet-stream" in resp.headers.get("Content-Type", "")
+                is_binary = "octet-stream" in content_type
                 result: dict = {
                     "request_id": request_id,
                     "status": resp.status,
@@ -143,21 +194,66 @@ class RelayHttpBridge:
                     result["body_base64"] = base64.b64encode(resp_bytes).decode()
                 else:
                     result["body"] = resp_bytes.decode(errors="replace")
-                return result
+                self._send_response_frame(result)
         except urllib.error.HTTPError as exc:
-            return {
+            self._send_response_frame({
                 "request_id": request_id,
                 "status": exc.code,
                 "headers": {"Content-Type": exc.headers.get("Content-Type", "application/json")},
                 "body": exc.read().decode(errors="replace"),
-            }
+            })
         except Exception as exc:
-            return {
+            self._send_response_frame({
                 "request_id": request_id,
                 "status": 503,
                 "headers": {"Content-Type": "application/json"},
                 "body": json.dumps({"error": f"relay bridge local request failed: {exc}"}),
-            }
+            })
+
+    def _stream_response(self, request_id: str, resp, resp_headers: dict) -> None:
+        """Forward an SSE response as chunk frames, one per complete SSE event.
+
+        Frame order: header frame (status + headers), chunk frames, done frame.
+        A receiver that sees no ``stream`` key treats the frame as a complete
+        legacy response, so non-streaming peers are unaffected.
+        """
+        sent = self._send_response_frame({
+            "request_id": request_id,
+            "status": resp.status,
+            "headers": resp_headers,
+            "stream": True,
+            "done": False,
+        })
+        if not sent:
+            return
+        event_lines: list[str] = []
+        for raw_line in resp:
+            line = raw_line.decode(errors="replace")
+            event_lines.append(line)
+            if line.strip():
+                continue
+            # Blank line terminates one SSE event — flush it as a frame.
+            if not self._send_response_frame({
+                "request_id": request_id,
+                "stream": True,
+                "chunk": "".join(event_lines),
+                "done": False,
+            }):
+                return
+            event_lines = []
+        if event_lines:
+            if not self._send_response_frame({
+                "request_id": request_id,
+                "stream": True,
+                "chunk": "".join(event_lines),
+                "done": False,
+            }):
+                return
+        self._send_response_frame({
+            "request_id": request_id,
+            "stream": True,
+            "done": True,
+        })
 
 
 def peer_id_from_wallet(wallet_address: str) -> str:

packages/node/meshnet_node/safetensors_selection.py

@@ -0,0 +1,210 @@
+"""Layer-aware SafeTensors snapshot file selection."""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Any
+
+INDEX_FILENAME = "model.safetensors.index.json"
+
+_LAYER_RE = re.compile(
+    r"(?:^|\.)"
+    r"(?:model\.layers|layers|h|blocks|decoder\.layers|encoder\.layers)"
+    r"\.(\d+)(?:\.|$)"
+)
+
+METADATA_FILENAMES = {
+    INDEX_FILENAME,
+    "config.json",
+    "generation_config.json",
+    "preprocessor_config.json",
+    "special_tokens_map.json",
+    "tokenizer.json",
+    "tokenizer.model",
+    "tokenizer_config.json",
+    "vocab.json",
+    "merges.txt",
+    "added_tokens.json",
+}
+
+_METADATA_PREFIXES = ("config.", "tokenizer.", "tokenizer_", "vocab.")
+
+_HEAD_MARKERS = (
+    "embed",
+    "embedding",
+    "embed_tokens",
+    "wte",
+    "wpe",
+)
+
+_TAIL_EXACT = {
+    "lm_head.weight",
+    "lm_head.bias",
+    "model.norm.weight",
+    "model.norm.bias",
+    "transformer.ln_f.weight",
+    "transformer.ln_f.bias",
+    "decoder.final_layer_norm.weight",
+    "decoder.final_layer_norm.bias",
+}
+
+_TAIL_MARKERS = (
+    ".lm_head.",
+    ".norm.",
+    ".ln_f.",
+    ".final_layer_norm.",
+)
+
+
+def select_safetensors_files_for_layers(
+    model_dir: str | Path,
+    start_layer: int,
+    end_layer: int,
+    *,
+    total_layers: int | None = None,
+) -> list[str]:
+    """Return relative snapshot files needed for an inclusive layer range.
+
+    The returned list always includes root-level config/tokenizer metadata and
+    the SafeTensors index. Weight shard files are included only when at least one
+    tensor in the index belongs to the assigned layer range, or when the tensor
+    is needed by the head/tail shard.
+    """
+    if start_layer < 0:
+        raise ValueError("start_layer must be non-negative")
+    if end_layer < start_layer:
+        raise ValueError("end_layer must be greater than or equal to start_layer")
+
+    root = Path(model_dir)
+    index_path = root / INDEX_FILENAME
+    try:
+        index = json.loads(index_path.read_text(encoding="utf-8"))
+    except FileNotFoundError as exc:
+        raise FileNotFoundError(f"missing SafeTensors index: {index_path}") from exc
+
+    weight_map = index.get("weight_map")
+    if not isinstance(weight_map, dict):
+        raise ValueError(f"{INDEX_FILENAME} must contain a weight_map object")
+
+    inferred_total_layers = total_layers if total_layers is not None else _read_total_layers(root)
+    selected = _metadata_files(root)
+    selected |= select_files_for_layers_from_index(
+        weight_map, start_layer, end_layer, total_layers=inferred_total_layers
+    )
+    return sorted(selected)
+
+
+def select_files_for_layers_from_index(
+    weight_map: dict[str, str],
+    start_layer: int,
+    end_layer: int,
+    *,
+    total_layers: int | None = None,
+) -> set[str]:
+    """Pure variant of the weight-file selection: takes an already-parsed
+    ``weight_map`` (no local snapshot directory needed), so callers that only
+    have the index fetched over the network — not a full local snapshot — can
+    still compute which shard files they need. Combine the result with
+    ``METADATA_FILENAMES`` for a complete download pattern set.
+    """
+    selected: set[str] = set()
+    for tensor_name, rel_file in weight_map.items():
+        if not isinstance(tensor_name, str) or not isinstance(rel_file, str):
+            continue
+        if _tensor_belongs_to_range(tensor_name, start_layer, end_layer, total_layers):
+            selected.add(_normalise_relative_file(rel_file))
+    return selected
+
+
+def select_tensor_names_for_layers_from_index(
+    weight_map: dict[str, str],
+    start_layer: int,
+    end_layer: int,
+    *,
+    total_layers: int | None = None,
+) -> set[str]:
+    """Pure variant that returns checkpoint tensor names instead of file paths."""
+    selected: set[str] = set()
+    for tensor_name, rel_file in weight_map.items():
+        if not isinstance(tensor_name, str) or not isinstance(rel_file, str):
+            continue
+        if _tensor_belongs_to_range(tensor_name, start_layer, end_layer, total_layers):
+            selected.add(tensor_name)
+    return selected
+
+
+def _tensor_belongs_to_range(
+    tensor_name: str,
+    start_layer: int,
+    end_layer: int,
+    total_layers: int | None,
+) -> bool:
+    layer = _layer_index(tensor_name)
+    if layer is not None:
+        return start_layer <= layer <= end_layer
+
+    if start_layer == 0 and _is_head_tensor(tensor_name):
+        return True
+
+    if total_layers is not None and end_layer >= total_layers - 1 and _is_tail_tensor(tensor_name):
+        return True
+
+    return False
+
+
+def _layer_index(tensor_name: str) -> int | None:
+    match = _LAYER_RE.search(tensor_name)
+    if match is None:
+        return None
+    return int(match.group(1))
+
+
+def _is_head_tensor(tensor_name: str) -> bool:
+    lowered = tensor_name.lower()
+    return any(marker in lowered for marker in _HEAD_MARKERS)
+
+
+def _is_tail_tensor(tensor_name: str) -> bool:
+    lowered = tensor_name.lower()
+    return lowered in _TAIL_EXACT or any(marker in lowered for marker in _TAIL_MARKERS)
+
+
+def _metadata_files(root: Path) -> set[str]:
+    files = {INDEX_FILENAME}
+    for path in root.iterdir():
+        if not path.is_file():
+            continue
+        name = path.name
+        if name in METADATA_FILENAMES or name.startswith(_METADATA_PREFIXES):
+            files.add(name)
+    return files
+
+
+def _read_total_layers(root: Path) -> int | None:
+    config_path = root / "config.json"
+    if not config_path.exists():
+        return None
+    config = json.loads(config_path.read_text(encoding="utf-8"))
+    return layers_from_config_dict(config)
+
+
+def layers_from_config_dict(config: dict[str, Any]) -> int | None:
+    for key in ("num_hidden_layers", "num_layers", "n_layer", "n_layers"):
+        value = config.get(key)
+        if isinstance(value, int) and value > 0:
+            return value
+
+    text_config = config.get("text_config")
+    if isinstance(text_config, dict):
+        return layers_from_config_dict(text_config)
+
+    return None
+
+
+def _normalise_relative_file(rel_file: str) -> str:
+    path = Path(rel_file)
+    if path.is_absolute() or ".." in path.parts:
+        raise ValueError(f"unsafe relative file in {INDEX_FILENAME}: {rel_file}")
+    return path.as_posix()

packages/node/meshnet_node/startup.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import json
+import os
 import socket
 import sys
 import threading
@@ -25,6 +26,86 @@ from .wallet import load_or_create_wallet
 _DEFAULT_BYTES_PER_LAYER = 30 * 1024 * 1024
 
 
+def _downloaded_model_inventory(
+    model: str,
+    shard_start: int,
+    shard_end: int,
+    shard_path: Path,
+    hf_repo: str | None = None,
+    model_sources: list[dict] | None = None,
+) -> list[dict]:
+    """Return a cheap local inventory record without reading model file contents."""
+    file_count = 0
+    total_bytes = 0
+    existing_rel_files: set[str] = set()
+    if shard_path.exists():
+        for path in shard_path.rglob("*"):
+            if not path.is_file():
+                continue
+            file_count += 1
+            try:
+                existing_rel_files.add(path.relative_to(shard_path).as_posix())
+            except ValueError:
+                pass
+            try:
+                total_bytes += path.stat().st_size
+            except OSError:
+                pass
+    record = {
+        "model": model,
+        "shard_start": shard_start,
+        "shard_end": shard_end,
+        "path": str(shard_path),
+        "file_count": file_count,
+        "total_bytes": total_bytes,
+    }
+    if hf_repo is not None:
+        record["hf_repo"] = hf_repo
+    expected_files: set[str] = set()
+    file_sizes: dict[str, int] = {}
+    for source in model_sources or []:
+        for rel in source.get("full_files") or source.get("files") or []:
+            if isinstance(rel, str) and rel and not rel.startswith("/") and ".." not in Path(rel).parts:
+                expected_files.add(rel)
+        sizes = source.get("file_sizes")
+        if isinstance(sizes, dict):
+            for rel, size in sizes.items():
+                if isinstance(rel, str) and isinstance(size, int):
+                    file_sizes[rel] = size
+    if expected_files:
+        expected_bytes = sum(file_sizes.get(rel, 0) for rel in expected_files)
+        local_expected_files = existing_rel_files & expected_files
+        local_expected_bytes = sum(file_sizes.get(rel, 0) for rel in local_expected_files)
+        record["expected_file_count"] = len(expected_files)
+        record["local_expected_file_count"] = len(local_expected_files)
+        record["expected_bytes"] = expected_bytes
+        record["local_expected_bytes"] = local_expected_bytes
+        record["local_model_percentage"] = (
+            round((local_expected_bytes / expected_bytes) * 100, 4)
+            if expected_bytes > 0
+            else 0.0
+        )
+    return [record]
+
+
+def _registration_shard_checksum(model: str, shard_path: Path) -> str | None:
+    """Only checksum tiny stub shards; real model folders are too large to hash at startup."""
+    if model != "stub-model":
+        return None
+    return compute_shard_checksum(shard_path)
+
+
+def _model_cache_path(model_id: str, cache_dir: Path | None) -> Path | None:
+    if cache_dir is None:
+        return None
+    if (cache_dir / "config.json").exists():
+        return cache_dir
+    candidate = cache_dir / model_id.split("/")[-1]
+    if candidate.exists():
+        return candidate
+    return None
+
+
 def _memory_budget(device: str, vram_mb: int, ram_mb: int, shared_vram_mb: int = 0) -> tuple[int, str]:
     """Return the capacity budget in MB and whether it came from VRAM or RAM."""
     if device == "cuda" and vram_mb > 0:
@@ -34,6 +115,23 @@ def _memory_budget(device: str, vram_mb: int, ram_mb: int, shared_vram_mb: int =
     return max(0, ram_mb), "RAM"
 
 
+def _full_model_sources(model_sources: list[dict]) -> list[dict]:
+    """Use tracker full-snapshot URLs for real HF model loading."""
+    full_sources: list[dict] = []
+    for source in model_sources:
+        full_url = source.get("full_url")
+        if isinstance(full_url, str) and full_url:
+            full_sources.append({
+                **source,
+                "url": full_url,
+                # full_files (when advertised) enables robust per-file download
+                # of the whole snapshot; empty list falls back to the tar stream.
+                "files": source.get("full_files") or [],
+                "type": f"{source.get('type') or 'model-source'}-full",
+            })
+    return full_sources
+
+
 def _hardware_label(device: str, gpu_name: str | None = None) -> str:
     if device == "cuda":
         return "CUDA"
@@ -42,30 +140,112 @@ def _hardware_label(device: str, gpu_name: str | None = None) -> str:
     return "CPU"
 
 
-def _max_assignable_layers(memory_mb: int, total_layers: int | None) -> int:
+def _positive_int(value: int | str | None, name: str) -> int | None:
+    if value is None or value == "":
+        return None
+    try:
+        parsed = int(value)
+    except (TypeError, ValueError) as exc:
+        raise ValueError(f"{name} must be a positive integer") from exc
+    if parsed < 1:
+        raise ValueError(f"{name} must be a positive integer")
+    return parsed
+
+
+def _configure_torch_threads(
+    torch_threads: int | None = None,
+    torch_interop_threads: int | None = None,
+) -> dict[str, int]:
+    """Apply PyTorch CPU thread settings before model load/benchmark."""
+    intra_threads = _positive_int(
+        torch_threads if torch_threads is not None else os.environ.get("MESHNET_TORCH_THREADS"),
+        "--torch-threads",
+    )
+    interop_threads = _positive_int(
+        torch_interop_threads
+        if torch_interop_threads is not None
+        else os.environ.get("MESHNET_TORCH_INTEROP_THREADS"),
+        "--torch-interop-threads",
+    )
+
+    if intra_threads is not None:
+        os.environ.setdefault("OMP_NUM_THREADS", str(intra_threads))
+        os.environ.setdefault("MKL_NUM_THREADS", str(intra_threads))
+    try:
+        import torch
+    except ModuleNotFoundError:
+        return {}
+
+    if intra_threads is not None:
+        torch.set_num_threads(intra_threads)
+    if interop_threads is not None:
+        torch.set_num_interop_threads(interop_threads)
+
+    active: dict[str, int] = {}
+    try:
+        active["torch_threads"] = int(torch.get_num_threads())
+    except Exception:
+        pass
+    try:
+        active["torch_interop_threads"] = int(torch.get_num_interop_threads())
+    except Exception:
+        pass
+    return active
+
+
+def _max_assignable_layers(
+    memory_mb: int,
+    total_layers: int | None,
+    bytes_per_layer: int | None = None,
+) -> int:
     if total_layers is None or total_layers <= 0 or memory_mb <= 0:
         return 0
     budget_bytes = memory_mb * 1024 * 1024
-    return min(total_layers, int((budget_bytes * 0.8) // _DEFAULT_BYTES_PER_LAYER))
+    layer_bytes = bytes_per_layer or _DEFAULT_BYTES_PER_LAYER
+    return min(total_layers, int((budget_bytes * 0.8) // layer_bytes))
 
 
-def _shard_budget_line(memory_mb: int, memory_source: str, total_layers: int | None, quantization: str) -> str:
+def _shard_budget_line(
+    memory_mb: int,
+    memory_source: str,
+    total_layers: int | None,
+    quantization: str,
+    bytes_per_layer: int | None = None,
+) -> str:
     memory_gb = memory_mb / 1024
     gb_str = f"{memory_gb:.1f} GB"
+    budget_quantization = "bfloat16" if quantization == "auto" else quantization
     if total_layers is None or total_layers <= 0:
         return f"Memory budget: {gb_str} {memory_source}; shard budget: unknown model layer count"
-    max_layers = _max_assignable_layers(memory_mb, total_layers)
+    max_layers = _max_assignable_layers(memory_mb, total_layers, bytes_per_layer=bytes_per_layer)
     # Remaining capacity after one full model load (rough estimate)
-    shard_bytes = max_layers * _DEFAULT_BYTES_PER_LAYER
+    shard_bytes = max_layers * (bytes_per_layer or _DEFAULT_BYTES_PER_LAYER)
     remaining_gb = (memory_mb * 1024 * 1024 - shard_bytes) / (1024 ** 3)
     remaining_str = f"; {remaining_gb:.1f} GB remaining after full load" if remaining_gb > 1 else ""
     return (
         f"Memory budget: {gb_str} {memory_source}; "
-        f"Shard budget: up to {max_layers}/{total_layers} layers at {quantization}"
+        f"Shard budget: up to {max_layers}/{total_layers} layers at {budget_quantization}"
         f"{remaining_str}"
     )
 
 
+def _assignment_bytes_per_layer(assignment: dict, quantization: str) -> int | None:
+    bytes_per_layer = assignment.get("bytes_per_layer")
+    if isinstance(bytes_per_layer, int) and bytes_per_layer > 0:
+        return bytes_per_layer
+    if not isinstance(bytes_per_layer, dict):
+        return None
+    keys = [quantization, "bfloat16", "bf16", "int8", "nf4"]
+    for key in keys:
+        value = bytes_per_layer.get(key)
+        if isinstance(value, int) and value > 0:
+            return value
+    for value in bytes_per_layer.values():
+        if isinstance(value, int) and value > 0:
+            return value
+    return None
+
+
 def _post_json(url: str, payload: dict, timeout: float = 10.0) -> dict:
     data = json.dumps(payload).encode()
     req = urllib.request.Request(
@@ -151,6 +331,9 @@ def _attach_relay_bridge(node: StubNodeServer | TorchNodeServer, bridge: RelayHt
     node.stop = _stop_with_bridge  # type: ignore[method-assign]
 
 
+_PENDING_NODE_ID = "pending"
+
+
 def _start_heartbeat(
     tracker_url: str,
     node_id: str,
@@ -188,10 +371,33 @@ def _start_heartbeat(
         try:
             resp = _post_json(f"{tracker_url}/v1/nodes/register", register_payload)
             node_id = resp.get("node_id", node_id)
+            if node_ref is not None:
+                setattr(node_ref, "tracker_node_id", node_id)
             return True
         except Exception:
             return False
 
+    def _register_additional_assignment(applied: dict) -> None:
+        model_id = str(applied.get("model") or register_payload.get("hf_repo") or register_payload.get("model"))
+        extra_payload = {
+            **register_payload,
+            "model": model_id.split("/")[-1],
+            "hf_repo": model_id if "/" in model_id else register_payload.get("hf_repo"),
+            "shard_start": applied["shard_start"],
+            "shard_end": applied["shard_end"],
+            "quantization": applied.get("quantization", register_payload.get("quantization")),
+            "tracker_mode": bool(applied.get("tracker_mode", False)),
+            "managed_assignment": True,
+        }
+        try:
+            reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", extra_payload)
+            print(
+                f"  [node] registered additional model — node ID: {reg_resp.get('node_id')}",
+                flush=True,
+            )
+        except Exception as exc:
+            print(f"  [node] WARNING: additional model registration failed: {exc}", flush=True)
+
     def _apply_directives(directives: list[dict]) -> None:
         if not directives:
             return
@@ -204,6 +410,9 @@ def _start_heartbeat(
             print(f"  [node] WARNING: failed to apply tracker directives: {exc}", flush=True)
             return
         if applied:
+            if applied.get("action") == "ADD_SHARD":
+                _register_additional_assignment(applied)
+                return
             model_id = applied.get("model", register_payload.get("hf_repo") or register_payload.get("model"))
             register_payload["model"] = str(model_id).split("/")[-1]
             register_payload["hf_repo"] = model_id
@@ -215,7 +424,7 @@ def _start_heartbeat(
     def _loop() -> None:
         nonlocal node_id
         hb_url = f"{tracker_url}/v1/nodes/{node_id}/heartbeat"
-        outage_streak = 0  # consecutive intervals where tracker was unreachable
+        outage_streak = 1 if node_id == _PENDING_NODE_ID else 0
 
         while True:
             time.sleep(interval)
@@ -243,11 +452,12 @@ def _start_heartbeat(
                 new_asgn = resp.get("new_assignment")
                 if new_asgn:
                     print(
-                        f"  [node] tracker reassignment received: "
-                        f"model={new_asgn.get('model')!r} "
+                        f"  [node] tracker assignment received: "
+                        f"action={new_asgn.get('action')!r} model={new_asgn.get('model')!r} "
                         f"shards={new_asgn.get('shard_start')}-{new_asgn.get('shard_end')}",
                         flush=True,
                     )
+                    _apply_directives([new_asgn])
             except urllib.error.HTTPError as exc:
                 if exc.code == 404:
                     # Node was purged (e.g. long gap before restart noticed) — re-register now.
@@ -269,6 +479,34 @@ def _start_heartbeat(
     return t
 
 
+def _register_with_tracker(
+    tracker_url: str,
+    reg_payload: dict,
+    node: Any,
+    start_time: float,
+) -> str | None:
+    """Register with the tracker, or start background retries when it is unreachable."""
+    try:
+        reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", reg_payload)
+        tracker_node_id = str(reg_resp.get("node_id") or "?")
+        setattr(node, "tracker_node_id", tracker_node_id)
+        print(f"  Registered with tracker — node ID: {tracker_node_id}", flush=True)
+        _start_heartbeat(tracker_url, tracker_node_id, reg_payload, node_ref=node, start_time=start_time)
+        return tracker_node_id
+    except Exception as exc:
+        setattr(node, "tracker_node_id", None)
+        print(f"  Warning: tracker registration failed: {exc}", flush=True)
+        print("  [node] will retry registration in the background", flush=True)
+        _start_heartbeat(
+            tracker_url,
+            _PENDING_NODE_ID,
+            reg_payload,
+            node_ref=node,
+            start_time=start_time,
+        )
+        return None
+
+
 def _warn_virtual_network_ip(ip: str | None) -> None:
     """Print a warning when the auto-detected advertise IP is in a known virtual-network range.
 
@@ -302,11 +540,11 @@ def _warn_virtual_network_ip(ip: str | None) -> None:
 def run_startup(
     tracker_url: str,
     port: int = 0,
-    model: str = "stub-model",
+    model: str | None = None,
     model_id: str | None = None,
     shard_start: int | None = None,
     shard_end: int | None = None,
-    quantization: str = "bfloat16",
+    quantization: str = "auto",
     wallet_path: Path | None = None,
     cache_dir: Path | None = None,
     host: str = "127.0.0.1",
@@ -316,6 +554,9 @@ def run_startup(
     vram_mb_override: int | None = None,
     max_loaded_shards: int = 1,
     debug: bool = False,
+    tracker_source_disabled: bool = False,
+    torch_threads: int | None = None,
+    torch_interop_threads: int | None = None,
 ) -> StubNodeServer | TorchNodeServer:
     """Execute the full startup sequence and return a running node server.
 
@@ -356,6 +597,12 @@ def run_startup(
 
     print("Detecting hardware...", flush=True)
     hw = detect_hardware()
+    torch_thread_config = _configure_torch_threads(torch_threads, torch_interop_threads)
+    if torch_thread_config:
+        hw.update(torch_thread_config)
+        intra = torch_thread_config.get("torch_threads", "?")
+        interop = torch_thread_config.get("torch_interop_threads", "?")
+        print(f"  PyTorch threads: intra-op={intra}, inter-op={interop}", flush=True)
     device: str = hw["device"]
     gpu_name: str | None = hw.get("gpu_name")
     vram_mb: int = hw.get("vram_mb", 0)
@@ -419,8 +666,12 @@ def run_startup(
     if probationary_line is not None:
         print(f"  {probationary_line}", flush=True)
 
+    pinned_shard_start = shard_start
+    pinned_shard_end = shard_end
+    user_pinned_shard = pinned_shard_start is not None or pinned_shard_end is not None
+
     if model_id:  # treat "" the same as None — no explicit model given
-        user_pinned_shard = shard_start is not None or shard_end is not None
+        full_sources: list[dict] = []
         # Auto-detect shard range from model config if not explicitly provided
         if shard_start is None or shard_end is None:
             try:
@@ -442,6 +693,19 @@ def run_startup(
                     if net_asgn.get("hf_repo") == model_id and net_asgn.get("gap_found"):
                         shard_start = net_asgn["shard_start"]
                         shard_end = net_asgn["shard_end"]
+                        full_sources = (
+                            [] if tracker_source_disabled
+                            else _full_model_sources(net_asgn.get("model_sources", []))
+                        )
+                        if full_sources:
+                            cache_dir = download_shard(
+                                model_id.split("/")[-1],
+                                shard_start,
+                                shard_end,
+                                cache_dir=cache_dir or Path.home() / ".cache" / "meshnet" / "shards",
+                                hf_repo=model_id,
+                                model_sources=full_sources,
+                            )
                         print(
                             f"  Tracker found uncovered shard: "
                             f"layers {shard_start}–{shard_end} (of {detected})",
@@ -465,6 +729,7 @@ def run_startup(
             route_timeout=route_timeout,
             cache_dir=cache_dir,
             debug=debug,
+            max_loaded_shards=max_loaded_shards,
         )
         _node_start_time = time.monotonic()
         actual_port = node.start()
@@ -487,6 +752,7 @@ def run_startup(
         _attach_relay_bridge(node, relay_bridge)
         # Register with tracker so other nodes can auto-join this model.
         total_layers = getattr(getattr(node, "backend", None), "total_layers", None)
+        model_cache_path = _model_cache_path(model_id, cache_dir)
         reg_payload = {
             "endpoint": endpoint,
             "model": model_id.split("/")[-1],
@@ -501,19 +767,24 @@ def run_startup(
             "tracker_mode": (shard_start == 0),
             "managed_assignment": not user_pinned_shard,
             "model_metadata": model_metadata_for(model_id, total_layers, cache_dir=cache_dir),
+            "downloaded_models": (
+                _downloaded_model_inventory(
+                    model_id.split("/")[-1],
+                    shard_start,
+                    shard_end,
+                    model_cache_path,
+                    hf_repo=model_id,
+                    model_sources=full_sources,
+                )
+                if model_cache_path is not None
+                else []
+            ),
             **registration_capabilities,
             **relay_fields,
         }
-        tracker_node_id: str | None = None
-        try:
-            reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", reg_payload)
-            tracker_node_id = str(reg_resp.get("node_id") or "?")
-            setattr(node, "tracker_node_id", tracker_node_id)
-            print(f"  Registered with tracker — node ID: {tracker_node_id}", flush=True)
-            _start_heartbeat(tracker_url, tracker_node_id, reg_payload, node_ref=node, start_time=_node_start_time)
-        except Exception as exc:
-            setattr(node, "tracker_node_id", None)
-            print(f"  Warning: tracker registration failed: {exc}", flush=True)
+        tracker_node_id = _register_with_tracker(
+            tracker_url, reg_payload, node, _node_start_time,
+        )
 
         print(
             f"\n{'=' * 32}\n"
@@ -531,30 +802,57 @@ def run_startup(
             flush=True,
         )
         return node
-    if shard_start is not None or shard_end is not None:
-        raise ValueError("--shard-start / --shard-end require --model-id")
+    if user_pinned_shard and not model:
+        raise ValueError("--shard-start / --shard-end require --model")
 
     # 3a. Auto-join: query tracker for network-wide HF model assignment.
-    print("Querying tracker for network assignment...", flush=True)
-    assign_qs = urllib.parse.urlencode({"device": device, "vram_mb": assignment_vram_mb, "ram_mb": ram_mb})
+    # Skipped when the user explicitly requested a model — the shard-assignment
+    # query below (/v1/nodes/assign?model=…) is authoritative there, and a fresh
+    # tracker would otherwise print a scary 503 for the model-less auto-join.
     net_assignment: dict = {}
-    try:
-        net_assignment = _get_json(f"{tracker_url}/v1/network/assign?{assign_qs}")
-    except Exception as exc:
-        print(f"  (auto-join unavailable: {exc})", flush=True)
+    if model_id or (model and model != "stub-model"):
+        if model:
+            print(f"Model {model!r} requested explicitly — skipping network auto-join.", flush=True)
+    else:
+        print("Querying tracker for network assignment...", flush=True)
+        assign_qs = urllib.parse.urlencode({"device": device, "vram_mb": assignment_vram_mb, "ram_mb": ram_mb})
+        try:
+            net_assignment = _get_json(f"{tracker_url}/v1/network/assign?{assign_qs}")
+        except Exception as exc:
+            print(f"  (auto-join unavailable: {exc})", flush=True)
     assigned_hf_repo: str | None = net_assignment.get("hf_repo")
     _gap_found: bool = bool(net_assignment.get("gap_found", False))
 
-    if assigned_hf_repo and _gap_found:
+    if assigned_hf_repo:
         assigned_shard_start: int = net_assignment["shard_start"]
         assigned_shard_end: int = net_assignment["shard_end"]
         assigned_num_layers: int = net_assignment["num_layers"]
-        print(
-            f"  Assigned: {assigned_hf_repo} "
-            f"layers {assigned_shard_start}–{assigned_shard_end} "
-            f"(of {assigned_num_layers})",
-            flush=True,
-        )
+        assigned_model_sources: list[dict] = net_assignment.get("model_sources", [])
+        if _gap_found:
+            print(
+                f"  Assigned gap: {assigned_hf_repo} "
+                f"layers {assigned_shard_start}–{assigned_shard_end} "
+                f"(of {assigned_num_layers})",
+                flush=True,
+            )
+        else:
+            print(
+                f"  Assigned redundant copy: {assigned_hf_repo} "
+                f"layers {assigned_shard_start}–{assigned_shard_end} "
+                f"(of {assigned_num_layers})",
+                flush=True,
+            )
+        full_sources = [] if tracker_source_disabled else _full_model_sources(assigned_model_sources)
+        if full_sources:
+            print("Downloading assigned model snapshot...", flush=True)
+            cache_dir = download_shard(
+                assigned_hf_repo.split("/")[-1],
+                assigned_shard_start,
+                assigned_shard_end,
+                cache_dir=cache_dir or Path.home() / ".cache" / "meshnet" / "shards",
+                hf_repo=assigned_hf_repo,
+                model_sources=full_sources,
+            )
         print("Loading real PyTorch model shard...", flush=True)
         node = TorchNodeServer(
             host=host,
@@ -567,6 +865,7 @@ def run_startup(
             route_timeout=route_timeout,
             cache_dir=cache_dir,
             debug=debug,
+            max_loaded_shards=max_loaded_shards,
         )
         _node_start_time = time.monotonic()
         actual_port = node.start()
@@ -581,6 +880,7 @@ def run_startup(
             relay_url=relay_url,
         )
         _attach_relay_bridge(node, relay_bridge)
+        model_cache_path = _model_cache_path(assigned_hf_repo, cache_dir)
         auto_reg_payload = {
             "endpoint": endpoint,
             "model": assigned_hf_repo.split("/")[-1],
@@ -595,19 +895,24 @@ def run_startup(
             "tracker_mode": (assigned_shard_start == 0),
             "managed_assignment": True,
             "model_metadata": model_metadata_for(assigned_hf_repo, assigned_num_layers, cache_dir=cache_dir),
+            "downloaded_models": (
+                _downloaded_model_inventory(
+                    assigned_hf_repo.split("/")[-1],
+                    assigned_shard_start,
+                    assigned_shard_end,
+                    model_cache_path,
+                    hf_repo=assigned_hf_repo,
+                    model_sources=full_sources,
+                )
+                if model_cache_path is not None
+                else []
+            ),
             **registration_capabilities,
             **relay_fields,
         }
-        tracker_node_id = None
-        try:
-            reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", auto_reg_payload)
-            tracker_node_id = str(reg_resp.get("node_id") or "?")
-            setattr(node, "tracker_node_id", tracker_node_id)
-            print(f"  Registered with tracker — node ID: {tracker_node_id}", flush=True)
-            _start_heartbeat(tracker_url, tracker_node_id, auto_reg_payload, node_ref=node, start_time=_node_start_time)
-        except Exception as exc:
-            setattr(node, "tracker_node_id", None)
-            print(f"  Warning: tracker registration failed: {exc}", flush=True)
+        tracker_node_id = _register_with_tracker(
+            tracker_url, auto_reg_payload, node, _node_start_time,
+        )
         shard_count = assigned_shard_end - assigned_shard_start + 1
         print(
             f"\n{'=' * 32}\n"
@@ -627,12 +932,21 @@ def run_startup(
         )
         return node
 
-    # 3b. Shard assignment from tracker (stub-model / preset-based path)
+    if not assigned_hf_repo and model is None:
+        raise RuntimeError(
+            "Tracker did not assign a model. Join a network that already serves one, "
+            "or start with --model <HF_REPO>."
+        )
+
+    # 3b. Stub preset path (tests / explicit stub-model) or named preset models.
     print("Querying tracker for shard assignment...", flush=True)
     assign_qs = urllib.parse.urlencode({
-        "model": model,
+        "model": model or "stub-model",
         "device": device,
-        "vram_mb": vram_mb,
+        # CPU-mode nodes must be sized by RAM: a detected-but-unusable GPU's
+        # VRAM would otherwise cap the shard (e.g. 8 GB VRAM → 3 layers on a
+        # 79 GB box whose Torch has no CUDA).
+        "vram_mb": assignment_vram_mb,
         "ram_mb": ram_mb,
     })
     try:
@@ -641,12 +955,25 @@ def run_startup(
         print(f"  ERROR: Cannot reach tracker at {tracker_url}: {exc}", file=sys.stderr, flush=True)
         raise
 
-    shard_start: int = assignment["shard_start"]
-    shard_end: int = assignment["shard_end"]
+    shard_start = assignment["shard_start"]
+    shard_end = assignment["shard_end"]
+    if user_pinned_shard:
+        if pinned_shard_start is not None:
+            shard_start = pinned_shard_start
+        if pinned_shard_end is not None:
+            shard_end = pinned_shard_end
     assigned_model: str = assignment.get("model", model)
     hf_repo: str | None = assignment.get("hf_repo")
     peers: list[dict] = assignment.get("peers", [])
-    print(f"  Shard: layers {shard_start}-{shard_end} of {assigned_model}", flush=True)
+    model_sources: list[dict] = [] if tracker_source_disabled else assignment.get("model_sources", [])
+    assignment_bytes_per_layer = _assignment_bytes_per_layer(assignment, quantization)
+    if user_pinned_shard:
+        print(
+            f"  Shard: layers {shard_start}-{shard_end} of {assigned_model} (pinned)",
+            flush=True,
+        )
+    else:
+        print(f"  Shard: layers {shard_start}-{shard_end} of {assigned_model}", flush=True)
 
     # 4. Download shard
     print("Downloading shard...", flush=True)
@@ -657,8 +984,18 @@ def run_startup(
         dl_kwargs["hf_repo"] = hf_repo
     if peers:
         dl_kwargs["peers"] = peers
+    if model_sources:
+        dl_kwargs["model_sources"] = model_sources
     shard_path = download_shard(assigned_model, shard_start, shard_end, **dl_kwargs)
-    shard_checksum = compute_shard_checksum(shard_path)
+    shard_checksum = _registration_shard_checksum(assigned_model, shard_path)
+    downloaded_models = _downloaded_model_inventory(
+        assigned_model,
+        shard_start,
+        shard_end,
+        shard_path,
+        hf_repo=hf_repo,
+        model_sources=model_sources,
+    )
     print(f"  Cached at: {shard_path}", flush=True)
 
     # 5. Start HTTP server
@@ -672,6 +1009,7 @@ def run_startup(
         model=assigned_model,
         shard_path=shard_path,
     )
+    _node_start_time = time.monotonic()
     actual_port = node.start()
     public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host)
     endpoint = f"http://{public_host}:{actual_port}"
@@ -687,24 +1025,28 @@ def run_startup(
 
     # 6. Register with tracker
     print("Registering with tracker...", flush=True)
+    reg_payload = {
+        "endpoint": endpoint,
+        "model": assigned_model,
+        "shard_start": shard_start,
+        "shard_end": shard_end,
+        "shard_checksum": shard_checksum,
+        "downloaded_models": downloaded_models,
+        "hardware_profile": hw,
+        "wallet_address": address,
+        "score": 1.0,
+        "managed_assignment": not user_pinned_shard,
+        **registration_capabilities,
+        **relay_fields,
+    }
     try:
         reg_resp = _post_json(
             f"{tracker_url}/v1/nodes/register",
-            {
-                "endpoint": endpoint,
-                "model": assigned_model,
-                "shard_start": shard_start,
-                "shard_end": shard_end,
-                "shard_checksum": shard_checksum,
-                "hardware_profile": hw,
-                "wallet_address": address,
-                "score": 1.0,
-                **registration_capabilities,
-                **relay_fields,
-            },
+            reg_payload,
         )
         node_id = str(reg_resp["node_id"])
         setattr(node, "tracker_node_id", node_id)
+        _start_heartbeat(tracker_url, node_id, reg_payload, node_ref=node, start_time=_node_start_time)
     except Exception:
         node.stop()
         raise
@@ -718,7 +1060,7 @@ def run_startup(
         f"meshnet-node ready\n"
         f"  Wallet:   {address}\n"
         f"  Shard:    layers {shard_start}-{shard_end} ({assigned_model})\n"
-        f"  {_shard_budget_line(memory_budget_mb, memory_budget_source, assignment.get('model_layers_end', shard_end) + 1, quantization)}\n"
+        f"  {_shard_budget_line(memory_budget_mb, memory_budget_source, assignment.get('model_layers_end', shard_end) + 1, quantization, bytes_per_layer=assignment_bytes_per_layer)}\n"
         f"  Endpoint: {endpoint}\n"
         f"  Node ID:  {node_id}\n"
         f"  Hardware: {hw_str}\n"
@@ -734,11 +1076,22 @@ def _detect_num_layers(model_id: str, cache_dir: Path | None = None) -> int | No
     """Fetch num_hidden_layers from HuggingFace model config (downloads ~1 KB config.json only)."""
     try:
         from transformers import AutoConfig  # type: ignore[import]
+
+        from .model_catalog import layers_from_config
+
+        local_model = _model_cache_path(model_id, cache_dir)
+        load_source = str(local_model) if local_model is not None else model_id
         cfg = AutoConfig.from_pretrained(
-            model_id,
-            cache_dir=str(cache_dir) if cache_dir is not None else None,
+            load_source,
+            cache_dir=str(cache_dir) if cache_dir is not None and local_model is None else None,
         )
-        return int(cfg.num_hidden_layers)
+        layers = layers_from_config(cfg)
+        if layers is None:
+            print(
+                f"  Warning: no layer count in {model_id} config "
+                "(checked top level and text_config)", flush=True,
+            )
+        return layers
     except Exception as exc:
         print(f"  Warning: could not read model config from HF: {exc}", flush=True)
         return None

packages/node/meshnet_node/torch_server.py

@@ -75,20 +75,39 @@ class _TorchHTTPServer(http.server.HTTPServer):
         tracker_url: str | None = None,
         route_timeout: float = 30.0,
         debug: bool = False,
+        max_loaded_shards: int = 1,
     ):
         super().__init__(addr, handler)
         self.backend = backend
+        self.backends: dict[str, TorchModelShard] = {backend.model_id: backend}
         self.received_activations = False
         self.forward_chunk_count = 0
         self.tracker_mode = tracker_mode
         self.tracker_url = tracker_url
         self.route_timeout = route_timeout
         self.debug = debug
+        self.max_loaded_shards = max(1, max_loaded_shards)
         self.total_requests: int = 0
         self.failed_requests: int = 0
         self.queue_depth: int = 0
         self._stats_lock = threading.Lock()
 
+    def resolve_backend(self, model_name: str | None) -> TorchModelShard | None:
+        if not model_name:
+            return self.backend
+        wanted = model_name.strip().lower()
+        for key, shard_backend in self.backends.items():
+            key_l = key.lower()
+            if key_l == wanted or key_l.rsplit("/", 1)[-1] == wanted:
+                return shard_backend
+        return self.backend
+
+    def chat_enabled(self) -> bool:
+        return any(
+            shard_backend.is_head
+            for shard_backend in self.backends.values()
+        )
+
 
 class _TorchHandler(http.server.BaseHTTPRequestHandler):
     def log_message(self, fmt, *args):  # noqa: suppress request logs in tests
@@ -100,7 +119,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             self._handle_forward()
         elif self.path == "/v1/infer":
             self._handle_infer()
-        elif self.path == "/v1/chat/completions" and server.tracker_mode:
+        elif self.path == "/v1/chat/completions" and server.chat_enabled():
             self._handle_chat_completions()
         else:
             self.send_response(404)
@@ -284,22 +303,26 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             messages = []
         stream = bool(body.get("stream", False))
         model_name = str(body.get("model", ""))
+        backend = server.resolve_backend(model_name)
+        if backend is None or not backend.is_head:
+            self._send_json(400, {"error": "model not loaded on this node"})
+            return
         max_tokens = int(body.get("max_tokens") or body.get("max_new_tokens") or 256)
         temperature = float(body.get("temperature") or 1.0)
         top_p = float(body.get("top_p") or 1.0)
 
         # Fast path: this node owns the complete model — use HF generate() with KV cache.
         # Avoids the single-token-per-forward-pass limitation of the distributed path.
-        if server.backend.is_head and server.backend.is_tail:
+        if backend.is_head and backend.is_tail:
             try:
                 if stream:
                     self._stream_openai_response(
-                        server.backend.generate_text_streaming(messages, max_tokens, temperature, top_p),
+                        backend.generate_text_streaming(messages, max_tokens, temperature, top_p),
                         model_name,
                     )
                 else:
-                    text = server.backend.generate_text(messages, max_tokens, temperature, top_p)
-                    self._send_openai_response(text, model_name, False, messages)
+                    text = backend.generate_text(messages, max_tokens, temperature, top_p)
+                    self._send_openai_response(text, model_name, False, messages, backend=backend)
             except Exception as exc:
                 self._record_failed_request()
                 self._send_json(500, {"error": f"generation failed: {exc}"})
@@ -309,7 +332,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         # We do N single-step forward passes (no cross-node KV cache), which is slow
         # but correct. Each step: head encodes current sequence → forwards through route
         # → tail returns the next token string → append → repeat.
-        remaining_route = self._get_remaining_route(model_name)
+        remaining_route = self._get_remaining_route(model_name, backend=backend)
         print(
             f"  [node] chat route model={model_name!r} max_tokens={max_tokens} "
             f"downstream={remaining_route}",
@@ -318,11 +341,10 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         if not remaining_route:
             self._send_openai_response(
                 "error: no downstream route — check tracker connectivity",
-                model_name, False, messages,
+                model_name, False, messages, backend=backend,
             )
             return
 
-        backend = server.backend
         # Format with chat template so the model knows it's in assistant mode.
         try:
             if hasattr(backend.tokenizer, "apply_chat_template"):
@@ -342,13 +364,17 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         generated: list[str] = []
         current_text = prompt_text
 
+        stream_emit = None
+        if stream:
+            stream_emit = self._start_openai_stream(model_name)
+
         for _ in range(max_tokens):
             try:
                 payload = backend.encode_prompt(current_text)
             except Exception as exc:
                 print(f"  [node] distributed encode error: {exc}", flush=True)
                 break
-            token_str = self._run_downstream_pipeline(payload, remaining_route)
+            token_str = self._run_downstream_pipeline(payload, remaining_route, backend=backend)
             if not token_str:
                 break
             # Stop on error responses or EOS.
@@ -357,12 +383,17 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             if eos_token and token_str == eos_token:
                 break
             generated.append(token_str)
+            if stream_emit is not None:
+                stream_emit(token_str)
             current_text = current_text + token_str
 
         result_text = "".join(generated)
-        self._send_openai_response(result_text, model_name, stream, messages)
+        if stream_emit is not None:
+            stream_emit(None)
+            return
+        self._send_openai_response(result_text, model_name, stream, messages, backend=backend)
 
-    def _get_remaining_route(self, model: str) -> list[dict]:
+    def _get_remaining_route(self, model: str, *, backend: TorchModelShard | None = None) -> list[dict]:
         """Return downstream hops as dicts with endpoint, start_layer, and optional relay_addr.
 
         Fast path reads X-Meshnet-Route header injected by the tracker.
@@ -395,9 +426,10 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
 
         # Slow path: query the tracker (direct node-to-node calls, or tracker didn't inject).
         server: _TorchHTTPServer = self.server  # type: ignore[assignment]
+        active_backend = backend or server.backend
         if server.tracker_url is None:
             return []
-        route_model = getattr(server.backend, "model_id", None) or model
+        route_model = getattr(active_backend, "model_id", None) or model
         try:
             url = f"{server.tracker_url}/v1/route?model={urllib.parse.quote(route_model)}"
             with urllib.request.urlopen(url, timeout=server.route_timeout) as r:
@@ -424,18 +456,19 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             print(f"  [node] WARNING: route lookup failed for {route_model!r}: {exc}", flush=True)
             return []
 
-    def _run_downstream_pipeline(self, payload: object, route: list[dict]) -> str:
+    def _run_downstream_pipeline(self, payload: object, route: list[dict], *, backend: TorchModelShard | None = None) -> str:
         server: _TorchHTTPServer = self.server  # type: ignore[assignment]
+        active_backend = backend or server.backend
         if not route:
             # Partial shard at tail: decode the activation from the previous node.
             # Full single-node (head+tail) is handled before entering this method.
-            if server.backend.is_tail:
+            if active_backend.is_tail:
                 try:
-                    tensor = server.backend.torch.frombuffer(
+                    tensor = active_backend.torch.frombuffer(
                         bytearray(payload.body),  # type: ignore[union-attr]
-                        dtype=server.backend.torch.bfloat16,
-                    ).reshape(payload.shape).to(server.backend.device)  # type: ignore[union-attr]
-                    return server.backend.decode_tail(tensor)
+                        dtype=active_backend.torch.bfloat16,
+                    ).reshape(payload.shape).to(active_backend.device)  # type: ignore[union-attr]
+                    return active_backend.decode_tail(tensor)
                 except Exception as exc:
                     return f"decode error: {exc}"
             return "no downstream route available for non-tail shard"
@@ -526,6 +559,15 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
 
     def _stream_openai_response(self, token_iter, model: str) -> None:
         """Stream tokens from an iterator as SSE chunks."""
+        emit = self._start_openai_stream(model)
+        for token_text in token_iter:
+            if not token_text:
+                continue
+            emit(token_text)
+        emit(None)
+
+    def _start_openai_stream(self, model: str):
+        """Open an OpenAI-compatible SSE response and return a token emitter."""
         chunk_id = "chatcmpl-node"
         created = int(time.time())
         self.send_response(200)
@@ -537,7 +579,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             try:
                 self.wfile.write(f"data: {data}\n\n".encode())
                 self.wfile.flush()
-            except BrokenPipeError:
+            except (BrokenPipeError, ConnectionResetError):
                 pass
 
         _emit(json.dumps({
@@ -545,24 +587,27 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             "model": model,
             "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
         }))
-        for token_text in token_iter:
-            if not token_text:
-                continue
+
+        def emit_token(token_text: str | None) -> None:
+            if token_text is None:
+                _emit(json.dumps({
+                    "id": chunk_id, "object": "chat.completion.chunk", "created": created,
+                    "model": model,
+                    "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
+                }))
+                try:
+                    self.wfile.write(b"data: [DONE]\n\n")
+                    self.wfile.flush()
+                except (BrokenPipeError, ConnectionResetError):
+                    pass
+                return
             _emit(json.dumps({
                 "id": chunk_id, "object": "chat.completion.chunk", "created": created,
                 "model": model,
                 "choices": [{"index": 0, "delta": {"content": token_text}, "finish_reason": None}],
             }))
-        _emit(json.dumps({
-            "id": chunk_id, "object": "chat.completion.chunk", "created": created,
-            "model": model,
-            "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
-        }))
-        try:
-            self.wfile.write(b"data: [DONE]\n\n")
-            self.wfile.flush()
-        except BrokenPipeError:
-            pass
+
+        return emit_token
 
     def _send_openai_response(
         self,
@@ -570,11 +615,13 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         model: str,
         stream: bool,
         messages: list[dict] | None = None,
+        backend: TorchModelShard | None = None,
     ) -> None:
         chunk_id = "chatcmpl-node"
         created = int(time.time())
+        active_backend = backend or self.server.backend  # type: ignore[attr-defined]
         if not stream:
-            usage = _usage_for_response(self.server.backend, messages or [], text)  # type: ignore[attr-defined]
+            usage = _usage_for_response(active_backend, messages or [], text)
             self._send_json(200, {
                 "id": chunk_id,
                 "object": "chat.completion",
@@ -685,9 +732,11 @@ class TorchNodeServer:
         route_timeout: float = 30.0,
         cache_dir: Path | None = None,
         debug: bool = False,
+        max_loaded_shards: int = 1,
     ) -> None:
         self._host = host
         self._requested_port = port
+        self._max_loaded_shards = max(1, max_loaded_shards)
         self._backend = backend or _load_backend(
             model_id,
             shard_start,
@@ -695,6 +744,7 @@ class TorchNodeServer:
             quantization,
             cache_dir,
         )
+        self._backends: dict[str, TorchModelShard] = {self._backend.model_id: self._backend}
         # Auto-detect tracker mode: enabled when shard_start == 0 or explicitly set
         self._tracker_mode = tracker_mode if tracker_mode is not None else (shard_start == 0)
         self._tracker_url = tracker_url
@@ -733,41 +783,64 @@ class TorchNodeServer:
     def queue_depth(self) -> int:
         return self._server.queue_depth if self._server is not None else 0
 
+    @property
+    def loaded_model_ids(self) -> list[str]:
+        return list(self._backends.keys())
+
     def apply_tracker_directives(self, directives: list[dict]) -> dict | None:
-        """Apply tracker LOAD_SHARD directives by hot-swapping the loaded backend."""
+        """Apply tracker shard directives (LOAD_SHARD replace, ADD_SHARD load-more)."""
+        add_directive = next(
+            (directive for directive in reversed(directives) if directive.get("action") == "ADD_SHARD"),
+            None,
+        )
         load_directive = next(
             (directive for directive in reversed(directives) if directive.get("action") == "LOAD_SHARD"),
             None,
         )
-        if load_directive is None:
+        directive = add_directive or load_directive
+        if directive is None:
             return None
-        shard_start = int(load_directive["shard_start"])
-        shard_end = int(load_directive["shard_end"])
-        quantization = str(load_directive.get("quantization") or self._backend.quantization)
-        model_id = str(load_directive.get("model") or self._backend.model_id)
+        shard_start = int(directive["shard_start"])
+        shard_end = int(directive["shard_end"])
+        quantization = str(directive.get("quantization") or self._backend.quantization)
+        model_id = str(directive.get("model") or self._backend.model_id)
+        replacing = directive.get("action") == "LOAD_SHARD"
+        if not replacing and len(self._backends) >= self._max_loaded_shards:
+            print(
+                f"  [node] WARNING: ignoring ADD_SHARD for {model_id!r} — "
+                f"loaded {len(self._backends)}/{self._max_loaded_shards} slots full",
+                flush=True,
+            )
+            return None
+        action_label = "reassigned" if replacing else "additional"
         print(
-            f"  [node] loading reassigned shard: {model_id} layers {shard_start}-{shard_end}",
+            f"  [node] loading {action_label} shard: {model_id} layers {shard_start}-{shard_end}",
             flush=True,
         )
         try:
             new_backend = _load_backend(model_id, shard_start, shard_end, quantization, self._cache_dir)
         except TypeError:
             new_backend = _load_backend(model_id, shard_start, shard_end, quantization)
-        self._backend = new_backend
-        self._tracker_mode = shard_start == 0
-        if self._server is not None:
-            self._server.backend = new_backend
-            self._server.tracker_mode = self._tracker_mode
+        self._backends[model_id] = new_backend
+        if replacing or shard_start == 0:
+            self._backend = new_backend
+            self._tracker_mode = shard_start == 0
         print(
-            f"  [node] loaded reassigned shard: {model_id} layers {shard_start}-{shard_end}",
+            f"  [node] loaded {action_label} shard: {model_id} layers {shard_start}-{shard_end}",
             flush=True,
         )
+        if self._server is not None:
+            self._server.backends = dict(self._backends)
+            if replacing or shard_start == 0:
+                self._server.backend = new_backend
+                self._server.tracker_mode = self._tracker_mode
         return {
+            "action": directive.get("action"),
             "model": model_id,
             "shard_start": shard_start,
             "shard_end": shard_end,
             "quantization": quantization,
-            "tracker_mode": self._tracker_mode,
+            "tracker_mode": shard_start == 0,
         }
 
     def start(self) -> int:
@@ -781,7 +854,9 @@ class TorchNodeServer:
             self._tracker_url,
             self._route_timeout,
             self._debug,
+            self._max_loaded_shards,
         )
+        self._server.backends = dict(self._backends)
         self.port = self._server.server_address[1]
         self._thread = threading.Thread(target=self._server.serve_forever, daemon=True)
         self._thread.start()

packages/node/pyproject.toml

@@ -19,6 +19,7 @@ dependencies = [
     "transformers>=4.39",
     "websockets>=13",
     "zstandard>=0.22",
+    "kernels>=0.11.1,<0.16",
 ]
 
 [project.scripts]

packages/relay/meshnet_relay/server.py

@@ -57,7 +57,9 @@ class RelayServer:
         self._ready = threading.Event()
         self._running = False
         self._stop_event: asyncio.Event | None = None
-        self._pending_rpc: dict[str, asyncio.Future] = {}
+        # request_id → queue of relay-http-response frames (US-036: a streamed
+        # response is a sequence of frames; a frame without "stream" is terminal).
+        self._pending_rpc: dict[str, asyncio.Queue] = {}
 
     @property
     def registry(self) -> PeerRegistry:
@@ -172,9 +174,9 @@ class RelayServer:
                 if topic == "relay-http-response":
                     payload = envelope.get("payload", {})
                     request_id = payload.get("request_id")
-                    fut = self._pending_rpc.pop(request_id, None)
-                    if fut is not None and not fut.done():
-                        fut.set_result(payload)
+                    queue = self._pending_rpc.get(request_id)
+                    if queue is not None:
+                        queue.put_nowait(payload)
                     continue
 
                 # Fan out to all other registered peers
@@ -240,8 +242,12 @@ class RelayServer:
         request_id = str(payload.get("request_id") or uuid.uuid4())
         payload["request_id"] = request_id
         payload["target_peer"] = target_peer_id
-        fut = self._loop.create_future() if self._loop is not None else asyncio.get_running_loop().create_future()
-        self._pending_rpc[request_id] = fut
+        queue: asyncio.Queue = asyncio.Queue()
+        self._pending_rpc[request_id] = queue
+        overall_timeout = 310.0
+        idle_timeout = 120.0
+        loop = asyncio.get_running_loop()
+        deadline = loop.time() + overall_timeout
         try:
             await target.ws.send(json.dumps({
                 "topic": "relay-http-request",
@@ -249,8 +255,19 @@ class RelayServer:
                 "from_peer": "relay",
                 "payload": payload,
             }))
-            response = await asyncio.wait_for(fut, timeout=310.0)
-            await ws_requester.send(json.dumps(response))
+            # Forward frames until a terminal one: streamed responses (US-036)
+            # end with {"stream": true, "done": true}; a frame without "stream"
+            # is a complete legacy single response.
+            while True:
+                remaining = deadline - loop.time()
+                if remaining <= 0:
+                    raise asyncio.TimeoutError
+                frame = await asyncio.wait_for(
+                    queue.get(), timeout=min(idle_timeout, remaining)
+                )
+                await ws_requester.send(json.dumps(frame))
+                if not frame.get("stream") or frame.get("done"):
+                    break
         except asyncio.TimeoutError:
             await ws_requester.send(json.dumps({
                 "request_id": request_id,

packages/tracker/meshnet_tracker/accounts.py

@@ -176,6 +176,14 @@ class AccountStore:
         with self._lock:
             return api_key in self._revoked_keys
 
+    def is_active_key(self, api_key: str) -> bool:
+        with self._lock:
+            return api_key in self._active_keys
+
+    def owner_of_key(self, api_key: str) -> str | None:
+        with self._lock:
+            return self._active_keys.get(api_key)
+
     # ---- views ----
 
     def _public_view(self, record: dict) -> dict:

packages/tracker/meshnet_tracker/billing.py

@@ -24,6 +24,18 @@ DEFAULT_BILLING_DB_PATH = "billing.sqlite"
 NODE_REVENUE_SHARE = 0.90  # nodes 90% / protocol cut 10% (ADR-0015)
 
 
+def _normalize_rates(value: "float | tuple[float, float] | dict") -> tuple[float, float]:
+    """Coerce a price spec into an (input_per_1k, output_per_1k) pair."""
+    if isinstance(value, dict):
+        base = value.get("price")
+        inp = value.get("input", base)
+        out = value.get("output", base)
+        return (float(inp), float(out))
+    if isinstance(value, (tuple, list)):
+        return (float(value[0]), float(value[1]))
+    return (float(value), float(value))
+
+
 class BillingLedger:
     """Thread-safe USDT ledger with SQLite persistence and event replication."""
 
@@ -33,13 +45,17 @@ class BillingLedger:
         self,
         db_path: str | None = None,
         *,
-        prices: dict[str, float] | None = None,
+        prices: dict[str, float | tuple[float, float]] | None = None,
         default_price_per_1k: float = DEFAULT_PRICE_PER_1K_TOKENS,
         starting_credit: float = DEFAULT_STARTING_CREDIT,
         node_share: float = NODE_REVENUE_SHARE,
     ) -> None:
         self._db_path = db_path
-        self._prices = dict(prices) if prices else {}
+        # US-045: per-model (input_per_1k, output_per_1k). A bare float in
+        # ``prices`` sets both rates (legacy single-rate models).
+        self._prices: dict[str, tuple[float, float]] = {
+            model: _normalize_rates(value) for model, value in (prices or {}).items()
+        }
         self._default_price_per_1k = default_price_per_1k
         self._starting_credit = starting_credit
         self._node_share = node_share
@@ -62,11 +78,23 @@ class BillingLedger:
     # ---- pricing ----
 
     def price_for(self, model: str) -> float:
-        return self._prices.get(model, self._default_price_per_1k)
+        """Blended (average) per-1k rate — kept for estimators and history logs."""
+        rates = self._prices.get(model)
+        if rates is None:
+            return self._default_price_per_1k
+        return (rates[0] + rates[1]) / 2.0
+
+    def prices_for(self, model: str) -> tuple[float, float]:
+        """(input_per_1k, output_per_1k) for a model (US-045)."""
+        return self._prices.get(model, (self._default_price_per_1k, self._default_price_per_1k))
 
     def set_price(self, model: str, price_per_1k: float) -> None:
+        """Legacy single-rate setter — applies the same rate to input and output."""
+        self.set_prices(model, price_per_1k, price_per_1k)
+
+    def set_prices(self, model: str, input_per_1k: float, output_per_1k: float) -> None:
         with self._lock:
-            self._prices[model] = price_per_1k
+            self._prices[model] = (float(input_per_1k), float(output_per_1k))
 
     # ---- local operations (create + apply + log an event) ----
 
@@ -87,6 +115,29 @@ class BillingLedger:
     def has_funds(self, api_key: str) -> bool:
         return self.ensure_client(api_key) > 0.0
 
+    def grant_caller_credit(self, api_key: str, account_id: str, amount: float) -> bool:
+        """Grant the one-time Caller Credit for an account (US-039).
+
+        The event id is derived from the account, not the key, so the grant is
+        idempotent per account — across retries, additional keys, and hive
+        gossip replication alike. Returns True only when credit was applied.
+        """
+        if amount <= 0:
+            return False
+        event_id = f"caller-credit-{account_id}"
+        with self._lock:
+            if event_id in self._seen_event_ids:
+                return False
+            self._apply_locked({
+                "id": event_id,
+                "type": "credit",
+                "api_key": api_key,
+                "amount": amount,
+                "ts": time.time(),
+                "note": "caller-credit",
+            })
+            return True
+
     def credit_client(self, api_key: str, amount: float, *, note: str = "deposit") -> float:
         if amount <= 0:
             raise ValueError("credit amount must be positive")
@@ -107,9 +158,17 @@ class BillingLedger:
         model: str,
         total_tokens: int,
         node_work: list[tuple[str | None, int]],
+        *,
+        input_tokens: int | None = None,
+        output_tokens: int | None = None,
     ) -> dict:
         """Debit the client and split the fee 90/10.
 
+        With ``input_tokens``/``output_tokens`` (US-045) the cost is
+        ``input·in_rate + output·out_rate``; without them, legacy behavior —
+        ``total_tokens`` at the blended rate. Replayed/gossiped events apply
+        their recorded ``cost``, so old events are unaffected either way.
+
         ``node_work`` is ``[(wallet_address | None, work_units), ...]`` for the
         nodes that served the request. Work units of nodes without a wallet
         accrue to the protocol cut — there is nowhere to pay them out.
@@ -117,7 +176,14 @@ class BillingLedger:
         request is then rejected by ``has_funds`` (post-pay drift, standard
         metered-billing behavior).
         """
-        cost = self.price_for(model) * max(0, total_tokens) / 1000.0
+        if input_tokens is not None or output_tokens is not None:
+            in_rate, out_rate = self.prices_for(model)
+            in_tokens = max(0, input_tokens or 0)
+            out_tokens = max(0, output_tokens or 0)
+            cost = (in_tokens * in_rate + out_tokens * out_rate) / 1000.0
+            total_tokens = in_tokens + out_tokens
+        else:
+            cost = self.price_for(model) * max(0, total_tokens) / 1000.0
         total_work = sum(max(0, w) for _, w in node_work)
         node_pool = cost * self._node_share
         shares: dict[str, float] = {}
@@ -142,6 +208,9 @@ class BillingLedger:
                 "api_key": api_key,
                 "model": model,
                 "total_tokens": total_tokens,
+                **({"input_tokens": max(0, input_tokens or 0),
+                    "output_tokens": max(0, output_tokens or 0)}
+                   if (input_tokens is not None or output_tokens is not None) else {}),
                 "cost": cost,
                 "shares": shares,
                 "protocol_amount": protocol_amount,
@@ -384,13 +453,13 @@ class BillingLedger:
         with self._lock:
             return self._node_pending.get(wallet, 0.0)
 
-    def usage_for(self, api_keys: list[str], *, recent_limit: int = 20) -> dict:
+    def usage_for(self, api_keys: list[str], *, recent_limit: int | None = None) -> dict:
         """Aggregate charge history for a set of API keys (dashboard view)."""
         keys = set(api_keys)
         requests = 0
         total_tokens = 0
         total_cost = 0.0
-        recent: list[dict] = []
+        records: list[dict] = []
         with self._lock:
             for event in self._event_log:
                 if event.get("type") != "charge" or event.get("api_key") not in keys:
@@ -398,18 +467,20 @@ class BillingLedger:
                 requests += 1
                 total_tokens += int(event.get("total_tokens", 0))
                 total_cost += float(event.get("cost", 0.0))
-                recent.append({
+                records.append({
                     "api_key": event["api_key"],
                     "model": event.get("model"),
                     "total_tokens": event.get("total_tokens", 0),
                     "cost": event.get("cost", 0.0),
                     "ts": event.get("ts", 0.0),
                 })
+        recent = records[-recent_limit:] if recent_limit is not None else records
         return {
             "requests": requests,
             "total_tokens": total_tokens,
             "total_cost": total_cost,
-            "recent": recent[-recent_limit:],
+            "records": records,
+            "recent": recent,
         }
 
     def snapshot(self) -> dict:

packages/tracker/meshnet_tracker/cli.py

@@ -1,18 +1,58 @@
 """meshnet-tracker CLI entry point."""
 
 import argparse
+import os
 import sys
 import time
+from pathlib import Path
 
 from .accounts import DEFAULT_ACCOUNTS_DB_PATH
 from .billing import DEFAULT_BILLING_DB_PATH
 from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH
-from .server import TrackerServer, derive_relay_url_from_public_tracker_url
+from .server import (
+    DEFAULT_CALLER_CREDIT_USDT,
+    DEFAULT_DEVNET_TOPUP_USDT,
+    TrackerServer,
+    derive_relay_url_from_public_tracker_url,
+)
 
 DEFAULT_REGISTRY_DB_PATH = "meshnet_registry.sqlite3"
 
 
+def _load_env_file(path: Path) -> None:
+    """Load simple KEY=VALUE pairs from an env file without overriding env vars."""
+    if not path.exists():
+        return
+    try:
+        lines = path.read_text().splitlines()
+    except OSError:
+        return
+    for line in lines:
+        text = line.strip()
+        if not text or text.startswith("#"):
+            continue
+        if text.startswith("export "):
+            text = text[len("export "):].strip()
+        if "=" not in text:
+            continue
+        key, value = text.split("=", 1)
+        key = key.strip()
+        if not key or key in os.environ:
+            continue
+        value = value.strip()
+        if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
+            value = value[1:-1]
+        os.environ[key] = value
+
+
+def _load_env_defaults() -> None:
+    """Load local and user-level tracker env defaults before parsing arguments."""
+    _load_env_file(Path.cwd() / ".env")
+    _load_env_file(Path.home() / ".config" / "meshnet" / "secrets.env")
+
+
 def main() -> None:
+    _load_env_defaults()
     common = argparse.ArgumentParser(add_help=False)
     common.add_argument("--host", default="0.0.0.0", help="Host interface to listen on")
     common.add_argument("--port", type=int, default=8080, help="Port to listen on")
@@ -66,6 +106,28 @@ def main() -> None:
             "token bound would cost more than this many USDT"
         ),
     )
+    common.add_argument(
+        "--starting-credit",
+        type=float,
+        default=DEFAULT_CALLER_CREDIT_USDT,
+        metavar="USDT",
+        help=(
+            "One-time Caller Credit granted when an account creates its first "
+            f"API key (default: {DEFAULT_CALLER_CREDIT_USDT}; set 0 to require "
+            "deposits before inference)"
+        ),
+    )
+    common.add_argument(
+        "--devnet-topup",
+        type=float,
+        default=DEFAULT_DEVNET_TOPUP_USDT,
+        metavar="USDT",
+        help=(
+            "Dashboard devnet top-up faucet: each click credits this many USDT "
+            f"to one of the account's keys (default: {DEFAULT_DEVNET_TOPUP_USDT}; "
+            "MUST be 0 on mainnet deployments)"
+        ),
+    )
     common.add_argument(
         "--registry-db",
         default=DEFAULT_REGISTRY_DB_PATH,
@@ -193,6 +255,12 @@ def main() -> None:
         default=86400.0,
         help="Seconds between dynamic pricing refresh passes (default: daily)",
     )
+    common.add_argument(
+        "--models-dir",
+        default=None,
+        metavar="PATH",
+        help="Local HuggingFace snapshot root advertised as tracker model-file source (default: MESHNET_MODELS_DIR)",
+    )
 
     parser = argparse.ArgumentParser(
         prog="meshnet-tracker",
@@ -231,6 +299,8 @@ def main() -> None:
             enable_billing=not args.no_billing,
             billing_db=None if args.no_billing else args.billing_db,
             max_charge_per_request=args.max_charge_per_request,
+            starting_credit=args.starting_credit,
+            devnet_topup_amount=args.devnet_topup,
             contracts=contracts,
             accounts_db=None if args.no_accounts else args.accounts_db,
             treasury=treasury,
@@ -248,6 +318,7 @@ def main() -> None:
                 or (DEFAULT_HF_PRICING_LOG_DB_PATH if args.enable_hf_pricing else None)
             ),
             hf_pricing_refresh_interval=args.hf_pricing_refresh_interval,
+            models_dir=args.models_dir,
         )
         port = server.start()
         print(f"meshnet-tracker listening on http://{args.host}:{port}", flush=True)

packages/tracker/meshnet_tracker/dashboard.html

@@ -39,11 +39,55 @@
   .form-row { display:flex; gap:8px; }
   .form-row button { white-space:nowrap; }
   .error-msg { color:var(--bad); font-size:12px; min-height:16px; }
-  .keybox { word-break:break-all; background:var(--bg); border:1px solid var(--border);
+  .keybox { display:flex; flex-wrap:wrap; align-items:center; gap:6px;
+          position:relative;
+          word-break:break-all; background:var(--bg); border:1px solid var(--border);
           border-radius:6px; padding:4px 8px; margin:4px 0; font-size:11px; }
+  .key-text { cursor:text; flex:1 1 auto; min-width:12rem; }
+  .copy-tooltip {
+    position:absolute; right:8px; top:-26px;
+    background:var(--panel); border:1px solid var(--border); color:var(--ok);
+    padding:2px 8px; border-radius:4px; font-size:11px;
+    pointer-events:none; z-index:1; white-space:nowrap;
+  }
   .tabs { display:flex; gap:10px; margin-bottom:8px; }
   .tabs a { color:var(--dim); cursor:pointer; }
   .tabs a.active { color:var(--accent); border-bottom:1px solid var(--accent); }
+  .dashboard-tabs { display:flex; gap:10px; padding:10px 20px 0; border-bottom:1px solid var(--border); }
+  .dashboard-tabs button { border:0; border-bottom:1px solid transparent; border-radius:0;
+          background:transparent; color:var(--dim); padding:5px 0 8px; }
+  .dashboard-tabs button.active { color:var(--accent); border-bottom-color:var(--accent); }
+  .wide { grid-column:1 / -1; }
+  section[hidden] { display:none !important; }
+  .chat-shell { display:grid; grid-template-columns:minmax(0, 1.35fr) minmax(320px, 0.65fr); gap:12px; }
+  .chat-pane { display:flex; flex-direction:column; gap:10px; min-width:0; }
+  .chat-panel { background:var(--bg); border:1px solid var(--border); border-radius:6px; padding:10px; }
+  .chat-controls { display:flex; gap:10px; align-items:end; flex-wrap:wrap; }
+  .chat-controls label { display:flex; flex-direction:column; gap:4px; color:var(--dim); }
+  .chat-controls select { min-width:220px; }
+  .chat-history { display:flex; flex-direction:column; gap:8px; min-height:220px; max-height:420px; overflow:auto; }
+  .chat-message { border:1px solid #21262d; border-radius:6px; padding:8px 10px; background:#10151d; }
+  .chat-role { color:var(--dim); font-size:11px; text-transform:uppercase; letter-spacing:.06em; margin-bottom:4px; }
+  .chat-role-user { color:var(--accent); }
+  .chat-role-assistant { color:var(--ok); }
+  .chat-role-error { color:var(--bad); }
+  .chat-compose { display:flex; flex-direction:column; gap:8px; }
+  .chat-compose textarea { min-height:112px; resize:vertical; width:100%; }
+  .chat-status { color:var(--dim); font-size:12px; }
+  .console {
+    background:var(--bg); border:1px solid var(--border); border-radius:6px;
+    min-height:160px; max-height:280px; overflow:auto; padding:7px 9px;
+    white-space:pre-wrap; word-break:break-word; font-size:11px;
+  }
+  .console-line { padding:1px 0; border-bottom:1px solid #161b22; }
+  .console-time { color:var(--dim); }
+  .console-level-info { color:var(--accent); }
+  .console-level-warn { color:var(--warn); }
+  .console-level-error { color:var(--bad); }
+  .status-pending { color:var(--warn); }
+  .status-processing { color:var(--accent); }
+  .status-failed { color:var(--bad); }
+  .status-complete { color:var(--ok); }
 </style>
 </head>
 <body>
@@ -52,17 +96,52 @@
   <span class="meta" id="self-url"></span>
   <span class="meta" id="refreshed"></span>
 </header>
+<nav class="dashboard-tabs" aria-label="Dashboard sections">
+  <button id="tab-overview" class="active" onclick="switchDashboardTab('overview')">Overview</button>
+  <button id="tab-chat" onclick="switchDashboardTab('chat')">Chat</button>
+  <button id="tab-billing" style="display:none" onclick="switchDashboardTab('billing')">Billing</button>
+  <button id="tab-admin" style="display:none" onclick="switchDashboardTab('admin')">Admin</button>
+</nav>
 <main>
-  <section id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
-  <section id="admin-section" style="display:none"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
-  <section><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
-  <section><h2>Nodes &amp; coverage</h2><div id="nodes" class="empty">loading…</div></section>
-  <section><h2>Client balances</h2><div id="clients" class="empty">loading…</div></section>
-  <section><h2>Node pending payouts</h2><div id="pending" class="empty">loading…</div></section>
-  <section><h2>Settlement history</h2><div id="settlements" class="empty">loading…</div></section>
-  <section><h2>Strikes / bans / forfeitures</h2><div id="fraud" class="empty">loading…</div></section>
-  <section><h2>Model usage (RPM)</h2><div id="stats" class="empty">loading…</div></section>
-  <section><h2>Node throughput</h2><div id="throughput" class="empty">loading…</div></section>
+  <section data-tab="overview" id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
+  <section data-tab="overview"><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
+  <section data-tab="overview"><h2>Nodes &amp; coverage</h2><div id="nodes" class="empty">loading…</div></section>
+  <section data-tab="overview"><h2>Model usage (RPM)</h2><div id="stats" class="empty">loading…</div></section>
+  <section data-tab="overview" class="wide"><h2>Call wall</h2><div id="call-wall" class="empty">loading...</div></section>
+  <section data-tab="chat" class="wide">
+    <h2>Chat / inference</h2>
+    <div class="chat-shell">
+      <div class="chat-pane">
+        <div class="chat-panel chat-controls">
+          <label>Model
+            <select id="chat-model" onchange="selectChatModel(this.value)"></select>
+          </label>
+          <button class="small" onclick="clearChatHistory()">clear history</button>
+        </div>
+        <div class="chat-panel chat-compose">
+          <textarea id="chat-prompt" placeholder="Ask a question or describe the task"></textarea>
+          <div class="form-row">
+            <button onclick="sendChat()" id="chat-send">Send</button>
+          </div>
+        </div>
+      </div>
+      <div class="chat-pane">
+        <div class="chat-panel">
+          <div id="chat-status" class="chat-status">select a model to start</div>
+          <div id="chat-history" class="chat-history empty">no messages yet</div>
+        </div>
+      </div>
+    </div>
+  </section>
+  <section data-tab="billing" data-logged-in-only><h2>Usage summary</h2><div id="usage-summary" class="empty">login required</div></section>
+  <section data-tab="billing" data-logged-in-only><h2>Node throughput</h2><div id="node-throughput" class="empty">login required</div></section>
+  <section data-tab="billing"><h2>Request history</h2><div id="billing-usage" class="empty">login required</div></section>
+  <section data-tab="billing" data-admin-only><h2>Node pending payouts</h2><div id="pending" class="empty">admin login required</div></section>
+  <section data-tab="billing" data-admin-only><h2>Settlement history</h2><div id="settlements" class="empty">admin login required</div></section>
+  <section data-tab="admin" id="admin-section"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
+  <section data-tab="admin" data-admin-only><h2>Strikes / bans / forfeitures</h2><div id="fraud" class="empty">admin login required</div></section>
+  <section data-tab="admin"><h2>Client balances</h2><div id="clients" class="empty">admin login required</div></section>
+  <section data-tab="admin" class="wide"><h2>Console output</h2><div id="console" class="console empty">admin login required</div></section>
 </main>
 <script>
 "use strict";
@@ -71,6 +150,7 @@ const esc = s => String(s).replace(/[&<>"]/g,
   c => ({"&":"&amp;","<":"&lt;",">":"&gt;",'"':"&quot;"}[c]));
 const usdt = v => (Math.round(v * 1e6) / 1e6).toFixed(6);
 const tps = v => (v === null || v === undefined) ? "?" : (Math.round(v * 10) / 10).toFixed(1);
+const copies = v => (v === null || v === undefined) ? "?" : Number(v).toFixed(2);
 const short = (s, n=14) => { s = String(s); return s.length > n ? s.slice(0, 6) + "…" + s.slice(-5) : s; };
 
 async function fetchJson(path) {
@@ -116,16 +196,17 @@ function renderNodes(map) {
   }
   let html = "";
   for (const [model, group] of Object.entries(byModel)) {
-    html += `<div><b>${esc(model)}</b> <span class="dim">(${group.length} node${group.length===1?"":"s"})</span></div>`;
-    html += table(["node", "shard", "tps (1h)", "queue", "health"], group.map(n => {
+    const supply = group.find(n => n.model_supply && n.model_supply.served_model_copies !== undefined);
+    const served = supply && supply.model_supply && supply.model_supply.served_model_copies;
+    html += `<div><b>${esc(model)}</b> <span class="dim">(${group.length} node${group.length===1?"":"s"} · ${esc(copies(served))} served)</span></div>`;
+    html += table(["node", "shard", "tps (1h)", "queue", "served"], group.map(n => {
       const modelStats = (n.throughput && (n.throughput[n.hf_repo] || n.throughput[n.model])) || {};
       return [
       esc(short(n.node_id || "?")),
       esc(`${n.shard_start ?? "?"}-${n.shard_end ?? "?"}`),
       `<span class="num">${esc(tps(modelStats.tokens_per_sec_last_hour))}</span>`,
       esc(String((n.stats && n.stats.queue_depth) ?? 0)),
-      (n.stats && (n.stats.alive === false || n.stats.healthy === false))
-        ? '<span class="bad">down</span>' : '<span class="ok">up</span>',
+      `<span class="num">${esc(copies(n.model_supply && n.model_supply.served_model_copies))}</span>`,
     ]; }));
   }
   $("nodes").innerHTML = html;
@@ -196,7 +277,7 @@ function renderStats(stats) {
   $("stats").innerHTML = table(["model", "rpm (1h)", "rpm (24h)", "rpm (30d)"], rows);
 }
 
-function renderThroughput(stats) {
+function renderThroughputHtml(stats) {
   const nodes = (stats && stats.nodes) || {};
   const rows = [];
   for (const [nodeId, nodeStats] of Object.entries(nodes)) {
@@ -209,17 +290,406 @@ function renderThroughput(stats) {
       ]);
     }
   }
-  $("throughput").innerHTML = table(["node", "model", "tps (1h)", "samples"], rows);
+  if (!rows.length) return '<div class="empty">no throughput samples yet</div>';
+  return table(["node", "model", "tps (1h)", "samples"], rows);
+}
+
+function hiveThroughputSummary(stats) {
+  const nodes = (stats && stats.nodes) || {};
+  let totalTps = 0;
+  let samples = 0;
+  for (const nodeStats of Object.values(nodes)) {
+    for (const s of Object.values((nodeStats && nodeStats.models) || {})) {
+      const t = Number(s.tokens_per_sec_last_hour);
+      if (Number.isFinite(t)) totalTps += t;
+      samples += Number(s.sample_count_last_hour || 0);
+    }
+  }
+  return { totalTps, samples };
+}
+
+function buildCallWallStates(events) {
+  const byId = new Map();
+  for (const e of events) {
+    const f = e.fields || {};
+    const id = f.request_id;
+    if (!id) continue;
+    let rec = byId.get(id);
+    if (!rec) {
+      rec = { id, events: [] };
+      byId.set(id, rec);
+    }
+    rec.events.push(e);
+    const msg = e.message;
+    if (msg === "proxy route selected") {
+      rec.status = "pending";
+      rec.started = e.ts;
+      rec.model = f.model || f.route_model || "?";
+      rec.route = f.route || f.nodes;
+      rec.nodes = f.nodes;
+      rec.stream = f.stream;
+    } else if (msg === "proxy via relay" || msg === "proxy connected") {
+      rec.status = "processing";
+      if (!rec.started) rec.started = e.ts;
+      rec.model = rec.model || f.model || f.route_model || "?";
+    } else if (msg === "proxy progress") {
+      rec.status = "processing";
+      rec.model = rec.model || f.model || f.route_model || "?";
+      rec.tokens = f.tokens;
+      rec.tps = f.tokens_per_sec;
+      rec.elapsed = f.elapsed_seconds;
+      rec.stream = f.stream;
+    } else if (msg === "relay proxy failed, trying direct") {
+      rec.status = "processing";
+      rec.warn = "relay failed, trying direct";
+    } else if (msg === "proxy complete") {
+      rec.status = "complete";
+      rec.model = rec.model || f.model || f.route_model || "?";
+      rec.tokens = f.tokens;
+      rec.tps = f.tokens_per_sec;
+      rec.elapsed = f.elapsed_seconds;
+      rec.stream = f.stream;
+      rec.terminal = e;
+    } else if (msg === "proxy failed" || msg === "direct proxy failed after relay") {
+      rec.status = "failed";
+      rec.model = rec.model || f.model || f.route_model || "?";
+      rec.error = f.error || msg;
+      rec.terminal = e;
+    }
+  }
+  return byId;
+}
+
+function callWallAgeSeconds(rec, nowSec) {
+  const start = rec.started || (rec.events[0] && rec.events[0].ts) || nowSec;
+  return Math.max(0, nowSec - start);
+}
+
+function callWallMaxQueue(rec) {
+  const nodes = rec.nodes || [];
+  const nodeQueues = Array.isArray(nodes) ? nodes.map(n => Number(n.queue_depth || 0)) : [];
+  return nodeQueues.length ? Math.max(...nodeQueues) : 0;
+}
+
+function renderCallWall(consoleData, stats) {
+  const events = (consoleData && consoleData.events) || [];
+  const nowSec = Date.now() / 1000;
+  const states = buildCallWallStates(events);
+  const active = [];
+  const terminal = [];
+  for (const rec of states.values()) {
+    if (rec.status === "pending" || rec.status === "processing") active.push(rec);
+    else if (rec.status === "complete" || rec.status === "failed") terminal.push(rec);
+  }
+  active.sort((a, b) => (a.started || 0) - (b.started || 0));
+  terminal.sort((a, b) => (b.terminal && b.terminal.ts) - (a.terminal && a.terminal.ts));
+
+  const hive = hiveThroughputSummary(stats);
+  const pending = active.filter(r => r.status === "pending").length;
+  const processing = active.filter(r => r.status === "processing").length;
+  const failedRecent = terminal.filter(r => r.status === "failed").length;
+  let queuedEstimate = 0;
+  for (const rec of active) queuedEstimate += Math.max(0, callWallMaxQueue(rec) - 1);
+
+  let html =
+    `<div class="dim" style="margin-bottom:6px">` +
+    `hive tps (1h): <b>${esc(tps(hive.totalTps))}</b> · samples: <b>${hive.samples}</b> · ` +
+    `active: <span class="status-processing">${processing}</span> processing · ` +
+    `<span class="status-pending">${pending}</span> pending` +
+    (queuedEstimate ? ` · queued estimate: <b>${queuedEstimate}</b>` : "") +
+    (failedRecent ? ` · <span class="status-failed">${failedRecent} recent failures</span>` : "") +
+    `</div>`;
+
+  if (active.length) {
+    html += table(["status", "age", "model", "request", "live tps", "tokens", "queue", "route / note"], active.map(rec => {
+      const statusCls = rec.status === "pending" ? "status-pending" : "status-processing";
+      const note = rec.warn || (rec.route ? short(String(rec.route), 28) : "");
+      return [
+        `<span class="${statusCls}">${esc(rec.status)}</span>`,
+        `<span class="num">${esc(callWallAgeSeconds(rec, nowSec).toFixed(1))}s</span>`,
+        esc(short(rec.model || "?", 28)),
+        esc(short(rec.id, 18)),
+        `<span class="num">${esc(tps(rec.tps))}</span>`,
+        `<span class="num">${esc(String(rec.tokens ?? "—"))}</span>`,
+        `<span class="num">${esc(String(callWallMaxQueue(rec)))}</span>`,
+        esc(note),
+      ];
+    }));
+  } else {
+    html += '<div class="empty">no in-flight requests</div>';
+  }
+
+  const historyRows = terminal.slice(0, 40).map(rec => {
+    const e = rec.terminal || {};
+    const f = e.fields || {};
+    const statusCls = rec.status === "failed" ? "status-failed" : "status-complete";
+    const detail = rec.status === "failed"
+      ? esc(short(rec.error || "?", 40))
+      : (f.stream ? "stream" : "json");
+    return [
+      new Date((e.ts || 0) * 1000).toLocaleTimeString(),
+      `<span class="${statusCls}">${esc(rec.status)}</span>`,
+      esc(short(rec.model || "?", 28)),
+      esc(short(rec.id, 18)),
+      `<span class="num">${esc(tps(rec.tps ?? f.tokens_per_sec))}</span>`,
+      `<span class="num">${esc(String(rec.tokens ?? f.tokens ?? "?"))}</span>`,
+      `<span class="num">${esc(String(rec.elapsed ?? f.elapsed_seconds ?? "?"))}</span>`,
+      detail,
+    ];
+  });
+  html += '<div style="margin-top:8px"><b class="dim">recent completed / failed</b></div>';
+  html += historyRows.length
+    ? table(["time", "status", "model", "request", "tps", "tokens", "sec", "detail"], historyRows)
+    : '<div class="empty">no completed requests yet</div>';
+  $("call-wall").innerHTML = html;
+}
+
+function startOfLocalDay(tsSec) {
+  const d = new Date(tsSec * 1000);
+  d.setHours(0, 0, 0, 0);
+  return d.getTime() / 1000;
+}
+
+function formatUsageDayLabel(tsSec) {
+  return new Date(tsSec * 1000).toLocaleDateString();
+}
+
+function summarizeUsageBuckets(records) {
+  const now = Date.now() / 1000;
+  const todayStart = startOfLocalDay(now);
+  const daySec = 86400;
+  const empty = () => ({ requests: 0, tokens: 0, cost: 0 });
+  const daily = [0, 1, 2].map(offset => ({
+    label: offset === 0 ? "Today" : offset === 1 ? "Yesterday" : formatUsageDayLabel(todayStart - offset * daySec),
+    ...empty(),
+  }));
+  const last7 = { label: "Last 7 days", ...empty() };
+  const last30 = { label: "Last 30 days", ...empty() };
+  const total = { label: "All time", ...empty() };
+
+  for (const u of records) {
+    const ts = Number(u.ts || 0);
+    const tokens = Number(u.total_tokens || 0);
+    const cost = Number(u.cost || 0);
+    total.requests += 1;
+    total.tokens += tokens;
+    total.cost += cost;
+    if (ts >= now - 30 * daySec) {
+      last30.requests += 1;
+      last30.tokens += tokens;
+      last30.cost += cost;
+    }
+    if (ts >= now - 7 * daySec) {
+      last7.requests += 1;
+      last7.tokens += tokens;
+      last7.cost += cost;
+    }
+    for (let offset = 0; offset < 3; offset++) {
+      const start = todayStart - offset * daySec;
+      const end = start + daySec;
+      if (ts >= start && ts < end) {
+        daily[offset].requests += 1;
+        daily[offset].tokens += tokens;
+        daily[offset].cost += cost;
+      }
+    }
+  }
+  return [...daily, last7, last30, total];
+}
+
+function renderUsageSummary(records) {
+  const el = $("usage-summary");
+  if (!el) return;
+  if (!sessionToken) {
+    el.innerHTML = '<div class="empty">login required</div>';
+    return;
+  }
+  if (!records.length) {
+    el.innerHTML = '<div class="empty">no billed requests yet</div>';
+    return;
+  }
+  const rows = summarizeUsageBuckets(records).map(b => [
+    esc(b.label),
+    `<span class="num">${b.requests}</span>`,
+    `<span class="num">${esc(String(b.tokens))}</span>`,
+    `<span class="num">${usdt(b.cost)}</span>`,
+  ]);
+  el.innerHTML =
+    '<div class="dim" style="margin-bottom:6px">per-request detail on Request history below</div>' +
+    table(["period", "requests", "tokens", "cost (USDT)"], rows);
+}
+
+function renderNodeThroughput(stats) {
+  const el = $("node-throughput");
+  if (!el) return;
+  if (!sessionToken) {
+    el.innerHTML = '<div class="empty">login required</div>';
+    return;
+  }
+  el.innerHTML = renderThroughputHtml(stats);
+}
+
+function renderBillingUsage(records) {
+  const el = $("billing-usage");
+  if (!el) return;
+  if (!sessionToken) {
+    el.innerHTML = '<div class="empty">login required</div>';
+    return;
+  }
+  if (!records.length) {
+    el.innerHTML = '<div class="empty">no billed requests yet</div>';
+    return;
+  }
+  const rows = records.slice().reverse().map(u => [
+    new Date((u.ts || 0) * 1000).toLocaleString(),
+    esc(short(u.model || "?", 28)),
+    esc(short(u.api_key || "?", 14)),
+    `<span class="num">${esc(String(u.total_tokens))}</span>`,
+    `<span class="num">${usdt(u.cost)}</span>`,
+  ]);
+  el.innerHTML = `<div class="dim" style="margin-bottom:6px">${records.length} request${records.length === 1 ? "" : "s"}</div>` +
+    table(["time", "model", "api key", "tokens", "cost (USDT)"], rows);
+}
+
+function renderConsole(data) {
+  const events = (data && data.events) || [];
+  if (!events.length) {
+    $("console").innerHTML = '<div class="empty">no console events</div>';
+    return;
+  }
+  $("console").innerHTML = events.slice(-120).map(e => {
+    const level = String(e.level || "info");
+    const cls = level === "error" ? "console-level-error" : level === "warn" ? "console-level-warn" : "console-level-info";
+    const fields = e.fields && Object.keys(e.fields).length ? " " + JSON.stringify(e.fields) : "";
+    return `<div class="console-line"><span class="console-time">${new Date((e.ts || 0) * 1000).toLocaleTimeString()}</span> ` +
+      `<span class="${cls}">${esc(level.toUpperCase())}</span> ${esc(e.message || "")}${esc(fields)}</div>`;
+  }).join("");
 }
 
 // ---- account panel (registration / login / balance / usage / API keys) ----
 
 let sessionToken = localStorage.getItem("meshnet_session") || null;
 let authTab = "login";
+let dashboardTab = "overview";
+let isAdmin = false;
+let isLoggedIn = false;
+let accountApiKeys = [];
+let accountUsageRecords = [];
+let lastStats = null;
+let availableModels = [];
+let chatHistory = [];
+let chatBusy = false;
+let selectedChatModel = localStorage.getItem("meshnet_chat_model") || "";
 
-async function apiCall(path, method, body) {
+function switchDashboardTab(name) {
+  if (name === "admin" && !isAdmin) name = "overview";
+  if (name === "billing" && !isLoggedIn) name = "overview";
+  dashboardTab = name;
+  updateSectionVisibility();
+  for (const tabName of ["overview", "chat", "billing", "admin"]) {
+    const button = $("tab-" + tabName);
+    if (button) button.classList.toggle("active", tabName === dashboardTab);
+  }
+}
+
+function updateSectionVisibility() {
+  for (const section of document.querySelectorAll("main section[data-tab]")) {
+    const onTab = section.dataset.tab === dashboardTab;
+    const adminOnly = section.hasAttribute("data-admin-only");
+    const loggedInOnly = section.hasAttribute("data-logged-in-only");
+    section.hidden = !onTab || (adminOnly && !isAdmin) || (loggedInOnly && !isLoggedIn);
+  }
+}
+
+function renderChatStatus(text) {
+  $("chat-status").textContent = text;
+}
+
+function renderChatHistory() {
+  const history = $("chat-history");
+  if (!chatHistory.length) {
+    history.classList.add("empty");
+    history.innerHTML = "no messages yet";
+    return;
+  }
+  history.classList.remove("empty");
+  history.innerHTML = chatHistory.map(msg => {
+    const roleClass = msg.role === "user" ? "chat-role-user" : msg.role === "assistant" ? "chat-role-assistant" : "chat-role-error";
+    const label = msg.role === "user" ? "user" : msg.role === "assistant" ? "assistant" : "error";
+    const meta = msg.model ? ` <span class="dim">· ${esc(short(msg.model, 24))}</span>` : "";
+    return `<div class="chat-message"><div class="chat-role ${roleClass}">${label}${meta}</div><div>${esc(msg.content)}</div></div>`;
+  }).join("");
+  history.scrollTop = history.scrollHeight;
+}
+
+function renderChatModels() {
+  const select = $("chat-model");
+  if (!select) return;
+  const models = availableModels.slice();
+  if (!models.length) {
+    select.innerHTML = '<option value="">no models available</option>';
+    select.disabled = true;
+    return;
+  }
+  select.disabled = false;
+  const preferred = models.find(m => m.id === selectedChatModel)
+    || models[0];
+  selectedChatModel = preferred.id;
+  localStorage.setItem("meshnet_chat_model", selectedChatModel);
+  select.innerHTML = models.map(model => {
+    const label = model.name && model.name !== model.id
+      ? `${model.name} (${model.id})`
+      : model.id;
+    const suffix = model.recommended ? " [recommended]" : "";
+    return `<option value="${esc(model.id)}"${model.id === selectedChatModel ? " selected" : ""}>${esc(label + suffix)}</option>`;
+  }).join("");
+  select.value = selectedChatModel;
+}
+
+function selectChatModel(value) {
+  selectedChatModel = value || "";
+  localStorage.setItem("meshnet_chat_model", selectedChatModel);
+}
+
+function clearChatHistory() {
+  chatHistory = [];
+  renderChatHistory();
+  renderChatStatus("history cleared");
+}
+
+function chatAuthToken() {
+  if (accountApiKeys.length) return accountApiKeys[0];
+  return null;
+}
+
+function setAdminMode(enabled) {
+  isAdmin = enabled;
+  $("tab-admin").style.display = enabled ? "" : "none";
+  if (!enabled && dashboardTab === "admin") {
+    switchDashboardTab("overview");
+  } else {
+    updateSectionVisibility();
+  }
+}
+
+function setLoggedInMode(enabled) {
+  isLoggedIn = enabled;
+  $("tab-billing").style.display = enabled ? "" : "none";
+  if (!enabled) {
+    accountUsageRecords = [];
+    renderBillingUsage([]);
+    renderUsageSummary([]);
+    renderNodeThroughput(null);
+    if (dashboardTab === "billing") switchDashboardTab("overview");
+  } else {
+    updateSectionVisibility();
+  }
+}
+
+async function apiCall(path, method, body, bearerToken) {
   const headers = { "Content-Type": "application/json" };
-  if (sessionToken) headers["Authorization"] = "Bearer " + sessionToken;
+  const token = bearerToken === undefined ? sessionToken : bearerToken;
+  if (token) headers["Authorization"] = "Bearer " + token;
   try {
     const r = await fetch(path, {
       method: method || "GET",
@@ -255,7 +725,10 @@ function renderAuthForms(errorMsg) {
   $("account").innerHTML =
     `<div class="tabs">${tab("login", "Log in")}${tab("register", "Register")}</div>` +
     form + `<div class="error-msg">${errorMsg ? esc(errorMsg) : ""}</div>`;
-  $("admin-section").style.display = "none";
+  accountApiKeys = [];
+  renderChatAuthHint();
+  setLoggedInMode(false);
+  setAdminMode(false);
 }
 
 function switchAuthTab(name) { authTab = name; renderAuthForms(); }
@@ -297,15 +770,88 @@ async function revokeKey(key) {
   await renderAccountPanel();
 }
 
+async function topupKey(key) {
+  const r = await apiCall("/v1/account/topup", "POST", { api_key: key });
+  if (!r.ok) alert(r.data.error || "top-up failed");
+  await renderAccountPanel();
+}
+
+const COPY_TOOLTIP_MS = 2000;
+
+function showCopiedTooltip(anchor) {
+  const box = (anchor && anchor.closest && anchor.closest(".keybox")) || anchor;
+  if (!box) return;
+  const existing = box.querySelector(".copy-tooltip");
+  if (existing) existing.remove();
+  const tip = document.createElement("span");
+  tip.className = "copy-tooltip";
+  tip.textContent = "Copied!";
+  tip.setAttribute("role", "status");
+  box.appendChild(tip);
+  setTimeout(() => tip.remove(), COPY_TOOLTIP_MS);
+}
+
+async function copyApiKeyText(text, anchor) {
+  try {
+    await navigator.clipboard.writeText(text);
+  } catch {
+    const ta = document.createElement("textarea");
+    ta.value = text;
+    ta.style.position = "fixed";
+    ta.style.left = "-9999px";
+    document.body.appendChild(ta);
+    ta.select();
+    try { document.execCommand("copy"); } catch { /* ignore */ }
+    document.body.removeChild(ta);
+  }
+  if (anchor) showCopiedTooltip(anchor);
+}
+
+function selectApiKeyText(el) {
+  const range = document.createRange();
+  range.selectNodeContents(el);
+  const sel = window.getSelection();
+  if (!sel) return;
+  sel.removeAllRanges();
+  sel.addRange(range);
+}
+
+function copyApiKeyFromTextEl(el) {
+  const key = el.dataset.key || el.textContent || "";
+  return copyApiKeyText(key, el);
+}
+
+function copyApiKeyFromButton(button) {
+  const el = button.closest(".keybox") && button.closest(".keybox").querySelector(".key-text");
+  const key = (el && el.dataset.key) || "";
+  return copyApiKeyText(key, button);
+}
+
+function renderChatAuthHint() {
+  if (chatAuthToken()) {
+    renderChatStatus("ready to send with your active API key");
+  } else if (sessionToken) {
+    renderChatStatus("create an API key in Account to use chat on a billing-enabled tracker");
+  } else {
+    renderChatStatus("log in if this tracker requires an API key");
+  }
+}
+
 async function renderAccountPanel() {
   const r = await apiCall("/v1/account");
   if (r.status === 404) {  // accounts disabled on this tracker
     $("account-section").style.display = "none";
-    $("admin-section").style.display = "none";
+    accountApiKeys = [];
+    accountUsageRecords = [];
+    renderChatAuthHint();
+    setLoggedInMode(false);
+    setAdminMode(false);
     return;
   }
   if (!r.ok) { setSession(null); renderAuthForms(); return; }
-  const { account, api_keys, balances, total_balance, usage } = r.data;
+  const { account, api_keys, balances, total_balance, usage, topup_amount } = r.data;
+  accountApiKeys = Array.isArray(api_keys) ? api_keys.slice() : [];
+  accountUsageRecords = (usage && (usage.records || usage.recent)) || [];
   const who = account.email || account.wallet || account.account_id;
   let html =
     `<div><b>${esc(who)}</b> <span class="pill">${esc(account.role)}</span> ` +
@@ -317,31 +863,86 @@ async function renderAccountPanel() {
     '<button class="small" onclick="createKey()">+ new key</button></div>';
   if (api_keys.length) {
     for (const key of api_keys) {
-      html += `<div class="keybox">${esc(key)}` +
-        ` <span class="dim">(${usdt(balances[key] ?? 0)} USDT)</span>` +
+      html += `<div class="keybox">` +
+        `<span class="key-text" data-key="${esc(key)}" onclick="selectApiKeyText(this)" ondblclick="copyApiKeyFromTextEl(this)">${esc(key)}</span>` +
+        `<span class="dim">(${usdt(balances[key] ?? 0)} USDT)</span>` +
+        `<button class="small" type="button" onclick="copyApiKeyFromButton(this)">copy</button>` +
+        (topup_amount > 0
+          ? ` <button class="small" onclick="topupKey('${esc(key)}')">+${usdt(topup_amount)} (devnet)</button>`
+          : "") +
         ` <button class="small" onclick="revokeKey('${esc(key)}')">revoke</button></div>`;
     }
   } else {
     html += '<div class="empty">no active keys</div>';
   }
-  if (usage.recent && usage.recent.length) {
-    html += '<div style="margin-top:6px"><b class="dim">recent usage</b></div>' +
-      table(["time", "model", "tokens", "cost"], usage.recent.slice().reverse().map(u => [
-        new Date(u.ts * 1000).toLocaleTimeString(),
-        esc(short(u.model || "?", 24)),
-        `<span class="num">${esc(String(u.total_tokens))}</span>`,
-        `<span class="num">${usdt(u.cost)}</span>`,
-      ]));
-  }
   $("account").innerHTML = html;
+  renderUsageSummary(accountUsageRecords);
+  renderNodeThroughput(lastStats);
+  renderBillingUsage(accountUsageRecords);
+  renderChatAuthHint();
+  renderChatModels();
+  renderChatHistory();
+  setLoggedInMode(true);
+  setAdminMode(account.role === "admin");
   if (account.role === "admin") await renderAdminPanel();
-  else $("admin-section").style.display = "none";
+}
+
+async function sendChat() {
+  const promptEl = $("chat-prompt");
+  const prompt = promptEl.value.trim();
+  if (!prompt || chatBusy) return;
+  if (!selectedChatModel) {
+    renderChatStatus("select a model first");
+    return;
+  }
+  const bearerToken = chatAuthToken();
+  const body = {
+    model: selectedChatModel,
+    messages: [
+      ...chatHistory
+        .filter(msg => msg.role === "user" || msg.role === "assistant")
+        .map(msg => ({ role: msg.role, content: msg.content })),
+      { role: "user", content: prompt },
+    ],
+    stream: false,
+    max_tokens: 256,
+  };
+  chatBusy = true;
+  $("chat-send").disabled = true;
+  promptEl.value = "";
+  chatHistory.push({ role: "user", content: prompt, model: selectedChatModel });
+  renderChatHistory();
+  renderChatStatus("sending request…");
+  const r = await apiCall("/v1/chat/completions", "POST", body, bearerToken);
+  chatBusy = false;
+  $("chat-send").disabled = false;
+  if (!r.ok) {
+    const error = r.data && r.data.error
+      ? (typeof r.data.error === "string" ? r.data.error : r.data.error.message || "request failed")
+      : "request failed";
+    chatHistory.push({ role: "error", content: error, model: selectedChatModel });
+    renderChatHistory();
+    renderChatStatus(error);
+    promptEl.focus();
+    return;
+  }
+  const reply = (r.data && r.data.choices && r.data.choices[0] && r.data.choices[0].message && r.data.choices[0].message.content) || "";
+  const usage = r.data && r.data.usage;
+  chatHistory.push({
+    role: "assistant",
+    content: reply || "(empty response)",
+    model: selectedChatModel,
+  });
+  renderChatHistory();
+  renderChatStatus(usage
+    ? `done: ${usage.total_tokens ?? "?"} tokens`
+    : "done");
+  promptEl.focus();
 }
 
 async function renderAdminPanel() {
   const r = await apiCall("/v1/admin/accounts");
-  if (!r.ok) { $("admin-section").style.display = "none"; return; }
-  $("admin-section").style.display = "";
+  if (!r.ok) { setAdminMode(false); return; }
   const rows = (r.data.accounts || []).map(a => {
     const balance = Object.values(a.balances || {}).reduce((x, y) => x + y, 0);
     return [
@@ -357,25 +958,44 @@ async function renderAdminPanel() {
 
 async function refresh() {
   $("self-url").textContent = location.host;
-  const [raft, map, summary, settlements, wallets, stats] = await Promise.all([
+  const [raft, map, stats, models, consoleData, adminData] = await Promise.all([
     fetchJson("/v1/raft/status"),
     fetchJson("/v1/network/map"),
-    fetchJson("/v1/billing/summary"),
-    fetchJson("/v1/billing/settlements"),
-    fetchJson("/v1/registry/wallets"),
     fetchJson("/v1/stats"),
+    fetchJson("/v1/models"),
+    fetchJson("/v1/console"),
+    isAdmin ? Promise.all([
+      fetchJson("/v1/billing/summary"),
+      fetchJson("/v1/billing/settlements"),
+      fetchJson("/v1/registry/wallets"),
+    ]) : Promise.resolve([null, null, null]),
   ]);
+  const [summary, settlements, wallets] = adminData;
+  lastStats = stats;
+  availableModels = ((models && models.data) || []).map(model => ({
+    id: model.id,
+    name: model.name || model.id,
+    recommended: Boolean(model.recommended),
+    aliases: model.aliases || [],
+  })).filter(model => model.id);
   renderHive(raft);
   renderNodes(map);
   renderBilling(summary);
   renderSettlements(settlements);
   renderFraud(wallets, summary);
   renderStats(stats);
-  renderThroughput(stats);
+  renderCallWall(consoleData, stats);
+  renderConsole(consoleData);
+  renderNodeThroughput(stats);
+  renderChatModels();
+  renderChatHistory();
   $("refreshed").textContent = "refreshed " + new Date().toLocaleTimeString();
 }
 refresh();
 renderAccountPanel();
+renderChatModels();
+renderChatHistory();
+renderChatAuthHint();
 setInterval(refresh, 4000);
 setInterval(() => { if (sessionToken) renderAccountPanel(); }, 8000);
 </script>

packages/tracker/meshnet_tracker/hf_pricing.py

@@ -295,6 +295,10 @@ def refresh_preset_price(
         "model": model_name,
         "old_price_per_1k": current_price,
         "new_price_per_1k": new_price,
+        # US-045: per-side rates (per 1k tokens) so the ledger bills input
+        # and output at the provider's actual asymmetry, not the average.
+        "new_input_price_per_1k": round(quote.input_per_1m * price_fraction / 1000.0, 6),
+        "new_output_price_per_1k": round(quote.output_per_1m * price_fraction / 1000.0, 6),
         "source_repo_id": quote.repo_id,
         "source_provider": quote.provider,
     }

packages/tracker/meshnet_tracker/model_files.py

@@ -0,0 +1,172 @@
+"""Helpers for serving layer-scoped model files from tracker-local snapshots."""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Any
+
+INDEX_FILENAME = "model.safetensors.index.json"
+
+_LAYER_RE = re.compile(
+    r"(?:^|\.)"
+    r"(?:model\.layers|layers|h|blocks|decoder\.layers|encoder\.layers)"
+    r"\.(\d+)(?:\.|$)"
+)
+
+_METADATA_FILENAMES = {
+    INDEX_FILENAME,
+    "config.json",
+    "generation_config.json",
+    "preprocessor_config.json",
+    "special_tokens_map.json",
+    "tokenizer.json",
+    "tokenizer.model",
+    "tokenizer_config.json",
+    "vocab.json",
+    "merges.txt",
+    "added_tokens.json",
+}
+
+_METADATA_PREFIXES = ("config.", "tokenizer.", "tokenizer_", "vocab.")
+
+_HEAD_MARKERS = ("embed", "embedding", "embed_tokens", "wte", "wpe")
+
+_TAIL_EXACT = {
+    "lm_head.weight",
+    "lm_head.bias",
+    "model.norm.weight",
+    "model.norm.bias",
+    "transformer.ln_f.weight",
+    "transformer.ln_f.bias",
+    "decoder.final_layer_norm.weight",
+    "decoder.final_layer_norm.bias",
+}
+
+_TAIL_MARKERS = (".lm_head.", ".norm.", ".ln_f.", ".final_layer_norm.")
+
+
+def snapshot_dir_for_repo(models_dir: Path, repo_id: str) -> Path | None:
+    """Return the most likely local HF snapshot directory for *repo_id*."""
+    candidates = [
+        models_dir / repo_id,
+        models_dir / repo_id.replace("/", "--"),
+        models_dir / f"models--{repo_id.replace('/', '--')}",
+    ]
+    for candidate in candidates:
+        if (candidate / "snapshots").is_dir():
+            snapshots = sorted(p for p in (candidate / "snapshots").iterdir() if p.is_dir())
+            if snapshots:
+                return snapshots[-1]
+        if candidate.is_dir():
+            return candidate
+    return None
+
+
+def files_for_layer_range(snapshot_dir: Path, shard_start: int, shard_end: int) -> list[str]:
+    """Select files needed to load a conservative safetensors shard subset."""
+    return select_safetensors_files_for_layers(snapshot_dir, shard_start, shard_end)
+
+
+def select_safetensors_files_for_layers(
+    model_dir: str | Path,
+    start_layer: int,
+    end_layer: int,
+    *,
+    total_layers: int | None = None,
+) -> list[str]:
+    if start_layer < 0:
+        raise ValueError("start_layer must be non-negative")
+    if end_layer < start_layer:
+        raise ValueError("end_layer must be greater than or equal to start_layer")
+
+    root = Path(model_dir)
+    index_path = root / INDEX_FILENAME
+    try:
+        index = json.loads(index_path.read_text(encoding="utf-8"))
+    except FileNotFoundError:
+        return sorted(p.name for p in root.glob("*.safetensors") if p.is_file())
+
+    weight_map = index.get("weight_map")
+    if not isinstance(weight_map, dict):
+        raise ValueError(f"{INDEX_FILENAME} must contain a weight_map object")
+
+    inferred_total_layers = total_layers if total_layers is not None else _read_total_layers(root)
+    selected = _metadata_files(root)
+
+    for tensor_name, rel_file in weight_map.items():
+        if not isinstance(tensor_name, str) or not isinstance(rel_file, str):
+            continue
+        if _tensor_belongs_to_range(tensor_name, start_layer, end_layer, inferred_total_layers):
+            selected.add(_normalise_relative_file(rel_file))
+
+    return sorted(rel for rel in selected if (root / rel).is_file())
+
+
+def _tensor_belongs_to_range(
+    tensor_name: str,
+    start_layer: int,
+    end_layer: int,
+    total_layers: int | None,
+) -> bool:
+    layer = _layer_index(tensor_name)
+    if layer is not None:
+        return start_layer <= layer <= end_layer
+    if start_layer == 0 and _is_head_tensor(tensor_name):
+        return True
+    if total_layers is not None and end_layer >= total_layers - 1 and _is_tail_tensor(tensor_name):
+        return True
+    return False
+
+
+def _layer_index(tensor_name: str) -> int | None:
+    match = _LAYER_RE.search(tensor_name)
+    return int(match.group(1)) if match else None
+
+
+def _is_head_tensor(tensor_name: str) -> bool:
+    lowered = tensor_name.lower()
+    return any(marker in lowered for marker in _HEAD_MARKERS)
+
+
+def _is_tail_tensor(tensor_name: str) -> bool:
+    lowered = tensor_name.lower()
+    return lowered in _TAIL_EXACT or any(marker in lowered for marker in _TAIL_MARKERS)
+
+
+def _metadata_files(root: Path) -> set[str]:
+    files = {INDEX_FILENAME}
+    for path in root.iterdir():
+        if not path.is_file():
+            continue
+        name = path.name
+        if name in _METADATA_FILENAMES or name.startswith(_METADATA_PREFIXES):
+            files.add(name)
+    return files
+
+
+def _read_total_layers(root: Path) -> int | None:
+    config_path = root / "config.json"
+    if not config_path.exists():
+        return None
+    config = json.loads(config_path.read_text(encoding="utf-8"))
+    return _layers_from_config(config)
+
+
+def _layers_from_config(config: dict[str, Any]) -> int | None:
+    for key in ("num_hidden_layers", "num_layers", "n_layer", "n_layers"):
+        value = config.get(key)
+        if isinstance(value, int) and value > 0:
+            return value
+    text_config = config.get("text_config")
+    if isinstance(text_config, dict):
+        return _layers_from_config(text_config)
+    return None
+
+
+def _normalise_relative_file(rel_file: str) -> str:
+    path = Path(rel_file)
+    if path.is_absolute() or ".." in path.parts:
+        raise ValueError(f"unsafe relative file in {INDEX_FILENAME}: {rel_file}")
+    return path.as_posix()

packages/tracker/meshnet_tracker/model_presets.json

@@ -12,7 +12,7 @@
       "recommended": true,
       "deployment_status": "recommended",
       "hf_aliases": [],
-      "hf_verified_match_note": "Pending human curation (issue 23) — no HF inference-marketplace listing has been confirmed as a comparable params/quantization match for this preset yet. Leave empty until a human signs off; an empty hf_aliases list keeps this model on the static default price.",
+      "hf_verified_match_note": "Pending human curation (issue 23) \u2014 no HF inference-marketplace listing has been confirmed as a comparable params/quantization match for this preset yet. Leave empty until a human signs off; an empty hf_aliases list keeps this model on the static default price.",
       "required_model_bytes": 638876385280,
       "download_size_bytes": 638876385280,
       "native_quantization": "int4",
@@ -38,6 +38,43 @@
           "KTransformers"
         ]
       }
+    },
+    "qwen3.6-35b-a3b": {
+      "layers_start": 0,
+      "layers_end": 39,
+      "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+      "aliases": [
+        "qwen3.6-35b-a3b",
+        "Qwen3.6-35B-A3B",
+        "unsloth/Qwen3.6-35B-A3B",
+        "Qwen/Qwen3.6-35B-A3B"
+      ],
+      "recommended": true,
+      "deployment_status": "recommended",
+      "price_per_1k_tokens": 0.00044,
+      "hf_aliases": [
+        "qwen/qwen3.6-35b-a3b"
+      ],
+      "hf_verified_match_note": "Verified 2026-07-06: unsloth/Qwen3.6-35B-A3B is a bf16 mirror of Qwen/Qwen3.6-35B-A3B; deepinfra and featherless-ai serve the official weights on the HF inference marketplace, so their rates are a fair comparable. Rates are 80% of deepinfra: input 0.00012/1k ($0.15/1M), output 0.00076/1k ($0.95/1M); price_per_1k_tokens keeps the blended 0.00044 for display/back-compat. The nightly refresher tracks both sides.",
+      "required_model_bytes": 71903776776,
+      "download_size_bytes": 71903776776,
+      "native_quantization": "bfloat16",
+      "canonical_audit_dtype": "bfloat16",
+      "canonical_audit_quantization": "bfloat16",
+      "bytes_per_layer": {
+        "bfloat16": 1797594419
+      },
+      "metadata": {
+        "architecture": "Mixture-of-Experts (MoE, hybrid linear attention)",
+        "total_parameters": "35B",
+        "activated_parameters": "3B",
+        "num_layers": 40,
+        "context_length": 262144,
+        "native_quantization": "bfloat16",
+        "download_size_gb": 72
+      },
+      "input_price_per_1k_tokens": 0.00012,
+      "output_price_per_1k_tokens": 0.00076
     }
   }
 }

tests/test_accounts.py

@@ -125,8 +125,15 @@ def _call(url, method="GET", body=None, token=None):
 
 @pytest.fixture
 def account_tracker():
+    """Tracker with credit features pinned OFF (defaults are devnet-friendly 1.0)."""
     ledger = BillingLedger(starting_credit=0.0, default_price_per_1k=0.02)
-    tracker = TrackerServer(billing=ledger, accounts=AccountStore(), hive_secret=HIVE_SECRET)
+    tracker = TrackerServer(
+        billing=ledger,
+        accounts=AccountStore(),
+        hive_secret=HIVE_SECRET,
+        starting_credit=0.0,
+        devnet_topup_amount=0.0,
+    )
     port = tracker.start()
     yield f"http://127.0.0.1:{port}", ledger
     tracker.stop()
@@ -228,3 +235,85 @@ def test_accounts_endpoints_404_when_disabled():
         assert exc_info.value.code == 404
     finally:
         tracker.stop()
+
+
+# ------------------------------------------- US-039/US-040: credit and top-up
+
+
+@pytest.fixture
+def funded_tracker():
+    """Tracker with Caller Credit and the devnet top-up faucet enabled."""
+    ledger = BillingLedger(starting_credit=0.0, default_price_per_1k=0.02)
+    tracker = TrackerServer(
+        billing=ledger,
+        accounts=AccountStore(),
+        hive_secret=HIVE_SECRET,
+        starting_credit=1.0,
+        devnet_topup_amount=10.0,
+    )
+    port = tracker.start()
+    yield f"http://127.0.0.1:{port}", ledger
+    tracker.stop()
+
+
+def test_caller_credit_granted_once_per_account(funded_tracker):
+    url, ledger = funded_tracker
+    reg = _call(f"{url}/v1/auth/register", "POST",
+                {"email": "c@example.com", "password": "secret-123"})
+    token = reg["session_token"]
+    first_key = reg["api_key"]
+    assert ledger.get_client_balance(first_key) == pytest.approx(1.0)
+
+    # A second key never re-grants — not even after revoking the first.
+    second = _call(f"{url}/v1/account/keys", "POST", {}, token=token)
+    assert second["caller_credit_granted"] is False
+    assert ledger.get_client_balance(second["api_key"]) == pytest.approx(0.0)
+    _call(f"{url}/v1/account/keys/revoke", "POST", {"api_key": first_key}, token=token)
+    third = _call(f"{url}/v1/account/keys", "POST", {}, token=token)
+    assert third["caller_credit_granted"] is False
+    assert ledger.get_client_balance(third["api_key"]) == pytest.approx(0.0)
+
+
+def test_unknown_bearer_key_rejected_by_proxy(funded_tracker):
+    url, ledger = funded_tracker
+    with pytest.raises(urllib.error.HTTPError) as exc_info:
+        _call(f"{url}/v1/chat/completions", "POST",
+              {"model": "any", "messages": []}, token="sk-mesh-made-up-key")
+    assert exc_info.value.code == 401
+    assert "unknown API key" in exc_info.value.read().decode()
+    # The invented key must not have become a billable client.
+    assert ledger.get_client_balance("sk-mesh-made-up-key") == pytest.approx(0.0)
+
+
+def test_devnet_topup_credits_own_key_only(funded_tracker):
+    url, ledger = funded_tracker
+    owner = _call(f"{url}/v1/auth/register", "POST",
+                  {"email": "own@example.com", "password": "secret-123"})
+    other = _call(f"{url}/v1/auth/register", "POST",
+                  {"email": "oth@example.com", "password": "secret-123"})
+
+    me = _call(f"{url}/v1/account", token=owner["session_token"])
+    assert me["topup_amount"] == pytest.approx(10.0)
+
+    result = _call(f"{url}/v1/account/topup", "POST",
+                   {"api_key": owner["api_key"]}, token=owner["session_token"])
+    assert result["credited"] == pytest.approx(10.0)
+    assert result["balance"] == pytest.approx(11.0)  # 1.0 caller credit + 10.0
+
+    with pytest.raises(urllib.error.HTTPError) as exc_info:
+        _call(f"{url}/v1/account/topup", "POST",
+              {"api_key": owner["api_key"]}, token=other["session_token"])
+    assert exc_info.value.code == 403
+    assert ledger.get_client_balance(owner["api_key"]) == pytest.approx(11.0)
+
+
+def test_topup_404_when_disabled(account_tracker):
+    url, _ = account_tracker
+    reg = _call(f"{url}/v1/auth/register", "POST",
+                {"email": "t@example.com", "password": "secret-123"})
+    me = _call(f"{url}/v1/account", token=reg["session_token"])
+    assert me["topup_amount"] == pytest.approx(0.0)
+    with pytest.raises(urllib.error.HTTPError) as exc_info:
+        _call(f"{url}/v1/account/topup", "POST",
+              {"api_key": reg["api_key"]}, token=reg["session_token"])
+    assert exc_info.value.code == 404

tests/test_dashboard.py

@@ -1,6 +1,7 @@
 """US-035: tracker web dashboard — served from any tracker, embedded asset."""
 
 import json
+import time
 import urllib.request
 
 from meshnet_contracts import LocalSolanaContracts
@@ -11,7 +12,10 @@ from meshnet_tracker.server import TrackerServer
 PANELS = [
     "Tracker hive", "Nodes & coverage", "Client balances",
     "Node pending payouts", "Settlement history",
-    "Strikes / bans / forfeitures", "Model usage", "Node throughput",
+    "Strikes / bans / forfeitures", "Model usage", "Call wall",
+    "Usage summary", "Node throughput", "Request history",
+    "Chat / inference",
+    "Console output",
 ]
 
 
@@ -64,3 +68,71 @@ def test_registry_wallets_endpoint():
         assert data["wallets"]["wallet-a"]["banned"] is False
     finally:
         tracker.stop()
+
+
+def test_console_endpoint_exposes_tracker_events():
+    tracker = TrackerServer()
+    port = tracker.start()
+    try:
+        body = json.dumps({
+            "endpoint": "http://127.0.0.1:9001",
+            "model": "stub-model",
+            "shard_start": 0,
+            "shard_end": 3,
+            "hardware_profile": {},
+        }).encode()
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{port}/v1/nodes/register",
+            data=body,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        urllib.request.urlopen(req).read()
+
+        data = json.loads(urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/console").read())
+    finally:
+        tracker.stop()
+
+    assert any(event["message"] == "node registered" for event in data["events"])
+
+
+def test_console_node_lifecycle_events_include_model_health():
+    tracker = TrackerServer(heartbeat_timeout=0.05)
+    port = tracker.start()
+    try:
+        body = json.dumps({
+            "endpoint": "http://127.0.0.1:9002",
+            "model": "console-health-test",
+            "hf_repo": "example/console-health-test",
+            "num_layers": 4,
+            "shard_start": 0,
+            "shard_end": 1,
+            "hardware_profile": {},
+        }).encode()
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{port}/v1/nodes/register",
+            data=body,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        urllib.request.urlopen(req).read()
+
+        registered = json.loads(urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/console").read())
+        registered_event = next(
+            event for event in registered["events"]
+            if event["message"] == "node registered"
+        )
+        assert registered_event["fields"]["model_health"]["served_model_copies"] == 0.5
+        assert registered_event["fields"]["model_health"]["coverage_percentage"] == 50.0
+
+        time.sleep(0.06)
+        urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/network/map").read()
+        expired = json.loads(urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/console").read())
+        expired_event = next(
+            event for event in expired["events"]
+            if event["message"] == "node expired"
+        )
+        assert expired_event["fields"]["model_health"]["served_model_copies"] == 0.0
+        assert expired_event["fields"]["model_health"]["coverage_percentage"] == 0.0
+    finally:
+        tracker.stop()

tests/test_gossip_and_relay.py

@@ -525,3 +525,293 @@ def test_mdns_start_and_stop_without_zeroconf(monkeypatch):
     disc = MdnsDiscovery(peer_id="x", port=8001)
     disc.start()  # should not raise
     disc.stop()   # should not raise
+
+
+# ---------------------------------------------------------------------------
+# US-036: streamed responses over relay RPC / US-037: bridge concurrency
+# ---------------------------------------------------------------------------
+
+def _start_local_http_server(handler_cls):
+    from http.server import ThreadingHTTPServer
+
+    server = ThreadingHTTPServer(("127.0.0.1", 0), handler_cls)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    return server, server.server_address[1]
+
+
+def _run_streaming_peer(port: int, peer_id: str, ready: threading.Event):
+    """Peer that answers one rpc request with a 4-frame streamed response."""
+    import websockets.sync.client as wsc  # type: ignore[import]
+
+    with wsc.connect(f"ws://127.0.0.1:{port}/ws") as ws:
+        ws.send(json.dumps({
+            "topic": "peer-register",
+            "version": 1,
+            "from_peer": peer_id,
+            "msg_id": f"{peer_id}-reg",
+            "payload": {"peer_id": peer_id, "addr": ""},
+        }))
+        ws.recv()
+        ready.set()
+        envelope = json.loads(ws.recv(timeout=5))
+        rid = envelope["payload"]["request_id"]
+
+        def frame(payload):
+            ws.send(json.dumps({
+                "topic": "relay-http-response",
+                "version": 1,
+                "from_peer": peer_id,
+                "payload": payload,
+            }))
+
+        frame({"request_id": rid, "status": 200,
+               "headers": {"Content-Type": "text/event-stream"},
+               "stream": True, "done": False})
+        frame({"request_id": rid, "stream": True,
+               "chunk": 'data: {"choices": [{"delta": {"content": "hello"}}]}\n\n',
+               "done": False})
+        frame({"request_id": rid, "stream": True,
+               "chunk": "data: [DONE]\n\n", "done": False})
+        frame({"request_id": rid, "stream": True, "done": True})
+
+
+def test_relay_rpc_forwards_streamed_frames_in_order():
+    """A streamed response traverses the relay as multiple frames ending with done."""
+    import websockets.sync.client as wsc  # type: ignore[import]
+    from meshnet_relay.server import RelayServer
+
+    relay = RelayServer(host="127.0.0.1", port=0)
+    port = relay.start()
+    ready = threading.Event()
+    peer_thread = threading.Thread(
+        target=_run_streaming_peer, args=(port, "stream_peer", ready), daemon=True
+    )
+    peer_thread.start()
+    assert ready.wait(timeout=5)
+
+    frames = []
+    with wsc.connect(f"ws://127.0.0.1:{port}/rpc/stream_peer") as ws:
+        ws.send(json.dumps({
+            "request_id": "req-stream-1",
+            "method": "POST",
+            "path": "/v1/chat/completions",
+            "headers": {},
+            "body": "{}",
+        }))
+        while True:
+            frames.append(json.loads(ws.recv(timeout=5)))
+            if not frames[-1].get("stream") or frames[-1].get("done"):
+                break
+
+    relay.stop()
+    peer_thread.join(timeout=3)
+
+    assert len(frames) == 4
+    assert frames[0]["status"] == 200 and frames[0]["stream"] is True
+    assert frames[1]["chunk"].startswith("data:")
+    assert frames[2]["chunk"] == "data: [DONE]\n\n"
+    assert frames[-1]["done"] is True
+
+
+def test_tracker_relay_http_request_collapses_streamed_frames():
+    """Buffered wrapper joins chunk frames into one response body."""
+    from meshnet_relay.server import RelayServer
+    from meshnet_tracker.server import _relay_http_request
+
+    relay = RelayServer(host="127.0.0.1", port=0)
+    port = relay.start()
+    ready = threading.Event()
+    peer_thread = threading.Thread(
+        target=_run_streaming_peer, args=(port, "buffer_peer", ready), daemon=True
+    )
+    peer_thread.start()
+    assert ready.wait(timeout=5)
+
+    response = _relay_http_request(
+        f"ws://127.0.0.1:{port}/rpc/buffer_peer",
+        path="/v1/chat/completions",
+        body=b"{}",
+        headers={},
+    )
+    relay.stop()
+    peer_thread.join(timeout=3)
+
+    assert response is not None
+    assert response["status"] == 200
+    assert '"content": "hello"' in response["body"]
+    assert "data: [DONE]" in response["body"]
+
+
+def test_stream_line_tokens_accounting():
+    from meshnet_tracker.server import _stream_line_tokens
+
+    assert _stream_line_tokens(b": comment") == (0, None)
+    assert _stream_line_tokens(b"data: [DONE]") == (0, None)
+    observed, reported = _stream_line_tokens(
+        b'data: {"choices": [{"delta": {"content": "two words"}}]}'
+    )
+    assert observed == 2 and reported is None
+    observed, reported = _stream_line_tokens(b'data: {"usage": {"total_tokens": 42}}')
+    assert reported == 42
+    assert _stream_line_tokens(b"data: not-json") == (1, None)
+
+
+def test_relay_bridge_streams_sse_as_chunk_frames():
+    """Bridge forwards a local SSE response as header + chunk frames + done."""
+    from http.server import BaseHTTPRequestHandler
+    from meshnet_node.relay_bridge import RelayHttpBridge
+
+    class SSEHandler(BaseHTTPRequestHandler):
+        def do_POST(self):
+            self.rfile.read(int(self.headers.get("Content-Length", 0) or 0))
+            self.send_response(200)
+            self.send_header("Content-Type", "text/event-stream")
+            self.end_headers()
+            for i in range(3):
+                self.wfile.write(f'data: {{"i": {i}}}\n\n'.encode())
+                self.wfile.flush()
+            self.wfile.write(b"data: [DONE]\n\n")
+
+        def log_message(self, *args):
+            pass
+
+    server, port = _start_local_http_server(SSEHandler)
+    try:
+        bridge = RelayHttpBridge(
+            relay_url="ws://unused.example/ws",
+            peer_id="peer-sse",
+            local_base_url=f"http://127.0.0.1:{port}",
+            advertised_addr="",
+        )
+        frames = []
+        bridge._send_response_frame = lambda payload: (frames.append(payload), True)[1]
+        bridge._process_request({
+            "request_id": "req-sse-1",
+            "method": "POST",
+            "path": "/v1/chat/completions",
+            "headers": {"Content-Type": "application/json"},
+            "body": "{}",
+        })
+    finally:
+        server.shutdown()
+
+    assert frames[0]["stream"] is True and frames[0]["status"] == 200
+    chunk_frames = [f for f in frames if f.get("chunk")]
+    assert len(chunk_frames) >= 2
+    assert frames[-1] == {"request_id": "req-sse-1", "stream": True, "done": True}
+    assert "".join(f["chunk"] for f in chunk_frames).count("data:") == 4
+
+
+def test_relay_bridge_non_stream_response_stays_single_frame():
+    from http.server import BaseHTTPRequestHandler
+    from meshnet_node.relay_bridge import RelayHttpBridge
+
+    class JSONHandler(BaseHTTPRequestHandler):
+        def do_POST(self):
+            self.rfile.read(int(self.headers.get("Content-Length", 0) or 0))
+            body = json.dumps({"ok": True}).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+        def log_message(self, *args):
+            pass
+
+    server, port = _start_local_http_server(JSONHandler)
+    try:
+        bridge = RelayHttpBridge(
+            relay_url="ws://unused.example/ws",
+            peer_id="peer-json",
+            local_base_url=f"http://127.0.0.1:{port}",
+            advertised_addr="",
+        )
+        frames = []
+        bridge._send_response_frame = lambda payload: (frames.append(payload), True)[1]
+        bridge._process_request({
+            "request_id": "req-json-1",
+            "method": "POST",
+            "path": "/v1/chat/completions",
+            "headers": {"Content-Type": "application/json"},
+            "body": "{}",
+        })
+    finally:
+        server.shutdown()
+
+    assert len(frames) == 1
+    assert "stream" not in frames[0]
+    assert json.loads(frames[0]["body"]) == {"ok": True}
+
+
+def test_relay_bridge_serves_concurrent_requests():
+    """US-037: a slow relayed request must not block a fast one."""
+    import websockets.sync.client as wsc  # type: ignore[import]
+    from http.server import BaseHTTPRequestHandler
+    from meshnet_node.relay_bridge import RelayHttpBridge
+    from meshnet_relay.server import RelayServer
+
+    class Handler(BaseHTTPRequestHandler):
+        def do_POST(self):
+            self.rfile.read(int(self.headers.get("Content-Length", 0) or 0))
+            if self.path == "/slow":
+                time.sleep(2.0)
+            body = json.dumps({"path": self.path}).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+        def log_message(self, *args):
+            pass
+
+    http_server, http_port = _start_local_http_server(Handler)
+    relay = RelayServer(host="127.0.0.1", port=0)
+    relay_port = relay.start()
+    bridge = RelayHttpBridge(
+        relay_url=f"ws://127.0.0.1:{relay_port}/ws",
+        peer_id="conc_peer",
+        local_base_url=f"http://127.0.0.1:{http_port}",
+        advertised_addr="",
+    )
+    bridge.start()
+    results = {}
+    try:
+        assert bridge.wait_connected(timeout=5)
+        deadline = time.time() + 5
+        while time.time() < deadline and relay.registry.get("conc_peer") is None:
+            time.sleep(0.05)
+        assert relay.registry.get("conc_peer") is not None
+
+        def call(path, key):
+            with wsc.connect(f"ws://127.0.0.1:{relay_port}/rpc/conc_peer") as ws:
+                ws.send(json.dumps({
+                    "request_id": f"req-{key}",
+                    "method": "POST",
+                    "path": path,
+                    "headers": {"Content-Type": "application/json"},
+                    "body": "{}",
+                }))
+                results[key] = (json.loads(ws.recv(timeout=10)), time.monotonic())
+
+        start = time.monotonic()
+        slow_thread = threading.Thread(target=call, args=("/slow", "slow"), daemon=True)
+        slow_thread.start()
+        time.sleep(0.3)  # let the slow request reach the bridge first
+        call("/fast", "fast")
+        fast_elapsed = results["fast"][1] - start
+        slow_thread.join(timeout=10)
+    finally:
+        bridge.stop()
+        relay.stop()
+        http_server.shutdown()
+
+    assert results["fast"][0]["status"] == 200
+    assert results["slow"][0]["status"] == 200
+    # The fast request finished while the slow one was still in flight.
+    assert fast_elapsed < 1.5
+    # Responses matched by request_id despite out-of-order completion.
+    assert json.loads(results["fast"][0]["body"]) == {"path": "/fast"}
+    assert json.loads(results["slow"][0]["body"]) == {"path": "/slow"}

tests/test_hf_pricing.py

@@ -148,3 +148,41 @@ def test_hf_pricing_log_persists_and_is_queryable(tmp_path):
     # Reopening against the same db path recovers the log (billing.py pattern).
     reopened = HfPricingLog(db_path=db_path)
     assert len(reopened.history()) == 1
+
+
+def test_preset_price_keys_cover_name_repo_and_aliases():
+    from meshnet_tracker.server import _preset_price_keys
+
+    preset = {
+        "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+        "aliases": ["qwen3.6-35b-a3b", "Qwen/Qwen3.6-35B-A3B"],
+    }
+    keys = _preset_price_keys("qwen3.6-35b-a3b", preset)
+    assert keys == {
+        "qwen3.6-35b-a3b",
+        "unsloth/Qwen3.6-35B-A3B",
+        "Qwen/Qwen3.6-35B-A3B",
+    }
+    assert _preset_price_keys("bare", {}) == {"bare"}
+
+
+def test_qwen_preset_prices_apply_to_all_aliases(tmp_path):
+    """Requests naming the repo id (what nodes register) bill at the preset price."""
+    from meshnet_tracker.server import TrackerServer
+
+    import pytest
+
+    tracker = TrackerServer(billing_db=str(tmp_path / "billing.sqlite"))
+    try:
+        billing = tracker._billing
+        assert billing is not None
+        for key in (
+            "qwen3.6-35b-a3b",
+            "unsloth/Qwen3.6-35B-A3B",
+            "Qwen/Qwen3.6-35B-A3B",
+        ):
+            assert billing.price_for(key) == pytest.approx(0.00044), key
+        # Unknown models keep the default rate
+        assert billing.price_for("some/other-model") == pytest.approx(0.02)
+    finally:
+        pass

tests/test_mining_cli.py

@@ -388,6 +388,149 @@ def test_legacy_start_treats_repo_model_as_model_id(monkeypatch):
     assert captured["model_id"] == "Qwen/Qwen2.5-0.5B-Instruct"
 
 
+def test_legacy_start_catalog_model_with_pinned_shards(monkeypatch):
+    """Catalog model names accept --shard-start/--shard-end without --model-id."""
+    from meshnet_node.cli import main
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--tracker", "http://192.168.0.179:8080",
+        "--model", "Qwen3.6-35B-A3B",
+        "--shard-start", "0",
+        "--shard-end", "44",
+        "--port", "0",
+    ])
+
+    with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+        with patch("time.sleep", side_effect=KeyboardInterrupt):
+            try:
+                main()
+            except SystemExit as exc:
+                assert exc.code == 0
+
+    assert captured["model"] == "Qwen3.6-35B-A3B"
+    assert captured["model_id"] is None
+    assert captured["shard_start"] == 0
+    assert captured["shard_end"] == 44
+
+
+def test_legacy_start_model_id_alias_for_catalog_name(monkeypatch):
+    """--model-id with a catalog name routes through the tracker preset path."""
+    from meshnet_node.cli import main
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--tracker", "http://192.168.0.179:8080",
+        "--model-id", "Qwen3.6-35B-A3B",
+        "--port", "0",
+    ])
+
+    with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+        with patch("time.sleep", side_effect=KeyboardInterrupt):
+            try:
+                main()
+            except SystemExit as exc:
+                assert exc.code == 0
+
+    assert captured["model"] == "Qwen3.6-35B-A3B"
+    assert captured["model_id"] is None
+
+
+def test_legacy_start_hf_repo_with_pinned_shards(monkeypatch):
+    """HF repo --model with pinned shards still enters the torch startup path."""
+    from meshnet_node.cli import main
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--tracker", "http://192.168.0.179:8081",
+        "--model", "Qwen/Qwen2.5-0.5B-Instruct",
+        "--shard-start", "12",
+        "--shard-end", "23",
+        "--port", "0",
+    ])
+
+    with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+        with patch("time.sleep", side_effect=KeyboardInterrupt):
+            try:
+                main()
+            except SystemExit as exc:
+                assert exc.code == 0
+
+    assert captured["model"] == "Qwen2.5-0.5B-Instruct"
+    assert captured["model_id"] == "Qwen/Qwen2.5-0.5B-Instruct"
+    assert captured["shard_start"] == 12
+    assert captured["shard_end"] == 23
+
+
+def test_legacy_start_falls_back_to_env_tracker_and_model(monkeypatch):
+    """`meshnet-node start` uses env defaults when tracker/model flags are omitted."""
+    import importlib
+
+    from meshnet_node import config as config_mod
+    from meshnet_node.cli import main
+
+    monkeypatch.setenv("MESHNET_TRACKER_URL", "http://env-tracker:8081")
+    monkeypatch.setenv("MESHNET_MODEL", "Qwen/Qwen2.5-0.5B-Instruct")
+    importlib.reload(config_mod)
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--port", "0",
+    ])
+
+    try:
+        with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+            with patch("time.sleep", side_effect=KeyboardInterrupt):
+                try:
+                    main()
+                except SystemExit as exc:
+                    assert exc.code == 0
+    finally:
+        monkeypatch.delenv("MESHNET_TRACKER_URL", raising=False)
+        monkeypatch.delenv("MESHNET_MODEL", raising=False)
+        importlib.reload(config_mod)
+
+    assert captured["tracker_url"] == "http://env-tracker:8081"
+    assert captured["model"] == "Qwen2.5-0.5B-Instruct"
+    assert captured["model_id"] == "Qwen/Qwen2.5-0.5B-Instruct"
+
+
 def test_legacy_start_without_port_uses_next_available_port(monkeypatch):
     """Omitting --port skips an occupied default port before startup loads the model."""
     from meshnet_node.cli import main

tests/test_node_startup.py

@@ -2,8 +2,13 @@
 
 import json
 import io
+import os
 import sys
+import tarfile
+import threading
+import time
 import types
+import urllib.error
 import urllib.request
 from pathlib import Path
 
@@ -12,6 +17,7 @@ import pytest
 from meshnet_node.downloader import download_shard, write_shard_archive
 from meshnet_node.hardware import detect_hardware, benchmark_throughput
 from meshnet_node.startup import (
+    _configure_torch_threads,
     _hardware_label,
     _infer_relay_url_from_tracker,
     _memory_budget,
@@ -152,11 +158,35 @@ def test_benchmark_throughput_fallback_on_bad_device():
     assert result == 1.0
 
 
+def test_configure_torch_threads_applies_explicit_settings(monkeypatch):
+    """Node startup can tune PyTorch CPU thread pools before loading a model."""
+    calls: dict[str, int] = {}
+
+    fake_torch = types.SimpleNamespace(
+        set_num_threads=lambda value: calls.update({"threads": value}),
+        set_num_interop_threads=lambda value: calls.update({"interop_threads": value}),
+        get_num_threads=lambda: calls["threads"],
+        get_num_interop_threads=lambda: calls["interop_threads"],
+    )
+
+    monkeypatch.setitem(sys.modules, "torch", fake_torch)
+    monkeypatch.delenv("OMP_NUM_THREADS", raising=False)
+    monkeypatch.delenv("MKL_NUM_THREADS", raising=False)
+
+    active = _configure_torch_threads(torch_threads=12, torch_interop_threads=2)
+
+    assert calls == {"threads": 12, "interop_threads": 2}
+    assert os.environ["OMP_NUM_THREADS"] == "12"
+    assert os.environ["MKL_NUM_THREADS"] == "12"
+    assert active == {"torch_threads": 12, "torch_interop_threads": 2}
+
+
 def test_benchmark_throughput_is_registered_in_payload(monkeypatch, tmp_path):
     """benchmark_tokens_per_sec from the benchmark is included in the tracker registration."""
     import meshnet_node.startup as startup_mod
 
     captured: dict = {}
+    thread_calls: dict[str, int] = {}
 
     class FakeNode:
         backend = None
@@ -174,6 +204,16 @@ def test_benchmark_throughput_is_registered_in_payload(monkeypatch, tmp_path):
     monkeypatch.setattr(startup_mod, "detect_hardware",
                         lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0, "ram_mb": 16384})
     monkeypatch.setattr(startup_mod, "benchmark_throughput_checked", lambda _device: (42.5, True, None))
+    monkeypatch.setitem(
+        sys.modules,
+        "torch",
+        types.SimpleNamespace(
+            set_num_threads=lambda value: thread_calls.update({"threads": value}),
+            set_num_interop_threads=lambda value: thread_calls.update({"interop_threads": value}),
+            get_num_threads=lambda: thread_calls["threads"],
+            get_num_interop_threads=lambda: thread_calls["interop_threads"],
+        ),
+    )
     monkeypatch.setattr(startup_mod, "TorchNodeServer", lambda **_kw: FakeNode())
     monkeypatch.setattr(startup_mod, "_detect_num_layers", lambda _model_id: 24)
     monkeypatch.setattr(startup_mod, "RelayHttpBridge", None)
@@ -189,10 +229,14 @@ def test_benchmark_throughput_is_registered_in_payload(monkeypatch, tmp_path):
         shard_start=0,
         shard_end=23,
         wallet_path=tmp_path / "wallet.json",
+        torch_threads=8,
+        torch_interop_threads=1,
     )
     node.stop()
 
     assert captured.get("benchmark_tokens_per_sec") == 42.5
+    assert captured["hardware_profile"]["torch_threads"] == 8
+    assert captured["hardware_profile"]["torch_interop_threads"] == 1
     assert captured["hardware_profile"]["benchmark_device"] == "cpu"
     assert captured["hardware_profile"]["benchmark_ok"] is True
 
@@ -396,7 +440,7 @@ def test_download_shard_uses_huggingface_when_repo_is_assigned(tmp_path, monkeyp
         progress=False,
     )
 
-    assert shard_dir == tmp_path / "tiny-llama" / "layers_0-3"
+    assert shard_dir == tmp_path / "tiny-llama"
     assert calls == [{
         "repo_id": "org/tiny-llama-shards",
         "cache_dir": str(tmp_path),
@@ -404,6 +448,241 @@ def test_download_shard_uses_huggingface_when_repo_is_assigned(tmp_path, monkeyp
     }]
 
 
+def test_download_shard_reuses_model_cache_for_narrower_layer_range(
+    tmp_path,
+    monkeypatch,
+):
+    """A wider cached shard satisfies a later narrower assignment for the same model."""
+    cache_dir = tmp_path / "cache"
+    model_dir = cache_dir / "tiny-llama"
+    model_dir.mkdir(parents=True)
+    (model_dir / "config.json").write_bytes(b"{}")
+    (model_dir / "model-00001-of-00002.safetensors").write_bytes(b"a" * 3)
+    (model_dir / "model-00002-of-00002.safetensors").write_bytes(b"b" * 5)
+
+    def unexpected_urlopen(*args, **kwargs):
+        raise AssertionError("cached files should avoid tracker download")
+
+    def unexpected_snapshot_download(*args, **kwargs):
+        raise AssertionError("cached files should avoid HuggingFace download")
+
+    monkeypatch.setattr(urllib.request, "urlopen", unexpected_urlopen)
+    monkeypatch.setitem(
+        sys.modules,
+        "huggingface_hub",
+        types.SimpleNamespace(snapshot_download=unexpected_snapshot_download),
+    )
+
+    shard_dir = download_shard(
+        "tiny-llama",
+        0,
+        1,
+        cache_dir=cache_dir,
+        hf_repo="org/tiny-llama-shards",
+        model_sources=[{
+            "type": "tracker",
+            "url": "http://tracker/v1/model-files/download?model=tiny-llama",
+            "files": ["config.json", "model-00001-of-00002.safetensors"],
+            "file_sizes": {
+                "config.json": 2,
+                "model-00001-of-00002.safetensors": 3,
+            },
+        }],
+        peers=[{"endpoint": "http://peer", "checksum": "unused"}],
+        progress=False,
+    )
+
+    assert shard_dir == model_dir
+    assert (model_dir / "model-00002-of-00002.safetensors").read_bytes() == b"b" * 5
+
+
+def test_download_shard_prefers_tracker_model_source_over_huggingface(
+    tmp_path,
+    monkeypatch,
+):
+    """A working tracker model source is used exclusively — HF is never contacted."""
+    contents = {
+        "config.json": b"{}",
+        "model-00002-of-00004.safetensors": b"tracker",
+    }
+
+    class FakeFileResponse:
+        def __init__(self, payload: bytes):
+            self._payload = io.BytesIO(payload)
+            self._length = len(payload)
+
+        def __enter__(self):
+            return self
+
+        def __exit__(self, exc_type, exc, tb):
+            return False
+
+        def getheader(self, name: str):
+            if name == "Content-Length":
+                return str(self._length)
+            if name == "Content-Type":
+                return "application/octet-stream"
+            return None
+
+        def read(self, size: int = -1) -> bytes:
+            return self._payload.read(size)
+
+    def fake_urlopen(url, *args, **kwargs):
+        query = urllib.parse.parse_qs(urllib.parse.urlparse(url).query)
+        rel = query.get("file", [None])[0]
+        assert rel in contents, f"unexpected per-file request: {url}"
+        return FakeFileResponse(contents[rel])
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake_urlopen)
+    hf_calls = []
+
+    def fake_snapshot_download(**kwargs):
+        hf_calls.append(kwargs)
+        time.sleep(0.05)
+        local_dir = Path(kwargs["local_dir"])
+        local_dir.mkdir(parents=True, exist_ok=True)
+        (local_dir / "model-00002-of-00004.safetensors").write_text("hf")
+        return str(local_dir)
+
+    monkeypatch.setitem(
+        sys.modules,
+        "huggingface_hub",
+        types.SimpleNamespace(snapshot_download=fake_snapshot_download),
+    )
+
+    shard_dir = download_shard(
+        "tiny-llama",
+        2,
+        3,
+        cache_dir=tmp_path / "cache",
+        hf_repo="org/tiny-llama-shards",
+        model_sources=[{
+            "type": "tracker",
+            "url": "http://tracker/v1/model-files/download?model=tiny-llama",
+            "files": ["config.json", "model-00002-of-00004.safetensors"],
+        }],
+        progress=False,
+    )
+
+    assert (shard_dir / "model-00002-of-00004.safetensors").read_text() == "tracker"
+    assert hf_calls == []
+
+
+def test_download_shard_prefers_tracker_full_model_source_over_huggingface(
+    tmp_path,
+    monkeypatch,
+):
+    """A tracker-advertised full snapshot is sufficient on its own — HF is never contacted."""
+    contents = {
+        "config.json": b"{}",
+        "weights-a.safetensors": b"tracker-a",
+        "weights-b.safetensors": b"tracker-b",
+    }
+
+    class FakeFileResponse:
+        def __init__(self, payload: bytes):
+            self._payload = io.BytesIO(payload)
+            self._length = len(payload)
+
+        def __enter__(self):
+            return self
+
+        def __exit__(self, exc_type, exc, tb):
+            return False
+
+        def getheader(self, name: str):
+            if name == "Content-Length":
+                return str(self._length)
+            if name == "Content-Type":
+                return "application/octet-stream"
+            return None
+
+        def read(self, size: int = -1) -> bytes:
+            return self._payload.read(size)
+
+    def fake_urlopen(url, *args, **kwargs):
+        query = urllib.parse.parse_qs(urllib.parse.urlparse(url).query)
+        rel = query.get("file", [None])[0]
+        assert rel in contents, f"unexpected per-file request: {url}"
+        return FakeFileResponse(contents[rel])
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake_urlopen)
+    hf_calls = []
+
+    def fake_snapshot_download(**kwargs):
+        hf_calls.append(kwargs)
+        raise AssertionError("HuggingFace should not be contacted when tracker full_files are available")
+
+    monkeypatch.setitem(
+        sys.modules,
+        "huggingface_hub",
+        types.SimpleNamespace(snapshot_download=fake_snapshot_download),
+    )
+
+    shard_dir = download_shard(
+        "tiny-llama",
+        0,
+        3,
+        cache_dir=tmp_path / "cache",
+        hf_repo="org/tiny-llama-shards",
+        model_sources=[{
+            "type": "tracker-full",
+            "url": "http://tracker/v1/model-files/download?model=tiny-llama&full=1",
+            "files": ["config.json", "weights-a.safetensors", "weights-b.safetensors"],
+            "full_files": ["config.json", "weights-a.safetensors", "weights-b.safetensors"],
+        }],
+        progress=False,
+    )
+
+    assert (shard_dir / "config.json").read_text() == "{}"
+    assert (shard_dir / "weights-a.safetensors").read_text() == "tracker-a"
+    assert (shard_dir / "weights-b.safetensors").read_text() == "tracker-b"
+    assert hf_calls == []
+
+
+def test_download_shard_falls_back_to_huggingface_when_tracker_source_fails(
+    tmp_path,
+    monkeypatch,
+):
+    """A dead tracker source falls through to HF with allow_patterns from the source files."""
+
+    def failing_urlopen(*args, **kwargs):
+        raise ConnectionResetError("tracker went away")
+
+    monkeypatch.setattr(urllib.request, "urlopen", failing_urlopen)
+    hf_calls = []
+
+    def fake_snapshot_download(**kwargs):
+        hf_calls.append(kwargs)
+        local_dir = Path(kwargs["local_dir"])
+        local_dir.mkdir(parents=True, exist_ok=True)
+        (local_dir / "model-00002-of-00004.safetensors").write_text("hf")
+        return str(local_dir)
+
+    monkeypatch.setitem(
+        sys.modules,
+        "huggingface_hub",
+        types.SimpleNamespace(snapshot_download=fake_snapshot_download),
+    )
+
+    shard_dir = download_shard(
+        "tiny-llama",
+        2,
+        3,
+        cache_dir=tmp_path / "cache",
+        hf_repo="org/tiny-llama-shards",
+        model_sources=[{
+            "type": "tracker",
+            "url": "http://tracker/v1/model-files/download?model=tiny-llama",
+            "files": ["config.json", "model-00002-of-00004.safetensors"],
+        }],
+        progress=False,
+    )
+
+    assert (shard_dir / "model-00002-of-00004.safetensors").read_text() == "hf"
+    assert hf_calls[0]["allow_patterns"] == ["config.json", "model-00002-of-00004.safetensors"]
+
+
 def test_download_shard_logs_huggingface_source(tmp_path, monkeypatch, capsys):
     """Shard download status tells the node operator when HuggingFace was used."""
 
@@ -584,6 +863,83 @@ def test_tracker_assign_returns_huggingface_repo_when_configured():
         tracker.stop()
 
 
+def test_tracker_assign_advertises_local_model_source_and_serves_subset(tmp_path):
+    """Tracker with models_dir advertises and serves only files needed for the shard."""
+    snapshot = tmp_path / "models" / "models--org--tiny-llama-shards" / "snapshots" / "abc"
+    nested = snapshot / "nested"
+    nested.mkdir(parents=True)
+    (snapshot / "config.json").write_text(json.dumps({"num_hidden_layers": 4}))
+    (snapshot / "tokenizer.json").write_text("{}")
+    (snapshot / "model.safetensors.index.json").write_text(json.dumps({
+        "weight_map": {
+            "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+            "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+            "model.layers.1.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+            "model.layers.2.self_attn.q_proj.weight": "nested/model-00002-of-00003.safetensors",
+            "model.layers.3.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+            "lm_head.weight": "model-00003-of-00003.safetensors",
+        },
+    }))
+    for rel in [
+        "model-00001-of-00003.safetensors",
+        "model-00002-of-00003.safetensors",
+        "nested/model-00002-of-00003.safetensors",
+        "model-00003-of-00003.safetensors",
+    ]:
+        (snapshot / rel).write_text(rel)
+
+    tracker = TrackerServer(
+        model_presets={
+            "tiny-llama": {
+                "layers_start": 0,
+                "layers_end": 3,
+                "hf_repo": "org/tiny-llama-shards",
+                "bytes_per_layer": {"bfloat16": 1024 * 1024},
+            },
+        },
+        models_dir=tmp_path / "models",
+    )
+    port = tracker.start()
+    try:
+        data = json.dumps({
+            "endpoint": "http://127.0.0.1:9100",
+            "model": "tiny-llama",
+            "shard_start": 0,
+            "shard_end": 0,
+            "hardware_profile": {},
+            "score": 1.0,
+        }).encode()
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{port}/v1/nodes/register",
+            data=data,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with urllib.request.urlopen(req) as r:
+            r.read()
+        resp = _get_json(
+            f"http://127.0.0.1:{port}/v1/nodes/assign?model=tiny-llama&device=cpu&ram_mb=3"
+        )
+        assert resp["shard_start"] == 1
+        assert resp["shard_end"] == 2
+        assert resp["model_sources"]
+        source = resp["model_sources"][0]
+        assert source["files"] == [
+            "config.json",
+            "model-00002-of-00003.safetensors",
+            "model.safetensors.index.json",
+            "nested/model-00002-of-00003.safetensors",
+            "tokenizer.json",
+        ]
+        with urllib.request.urlopen(source["url"], timeout=5) as response:
+            payload = io.BytesIO(response.read())
+        with tarfile.open(fileobj=payload, mode="r") as tf:
+            names = sorted(tf.getnames())
+        assert names == source["files"]
+    finally:
+        tracker.stop()
+
+
 def test_tracker_assign_lists_peers_for_same_model_shard():
     """A registered node with a completed shard is returned as a same-shard peer."""
     import json as _json
@@ -1077,6 +1433,84 @@ def test_later_node_auto_joins_existing_public_hf_model_with_only_tracker_url(
     assert route_resp["route"] == ["http://203.0.113.20:8001", "http://203.0.113.21:8002"]
 
 
+def test_later_node_auto_joins_redundant_copy_when_model_is_fully_covered(
+    tmp_path,
+    monkeypatch,
+):
+    """Model-less joins should load the served HF model even when gap_found=false."""
+    import meshnet_node.startup as startup_mod
+
+    captured = {}
+
+    class FakeBackend:
+        total_layers = 24
+
+    class FakeTorchNodeServer:
+        def __init__(self, **kwargs):
+            captured.update(kwargs)
+            self.backend = FakeBackend()
+            self.port = None
+            self.chat_completion_count = 0
+            self.total_requests = 0
+            self.failed_requests = 0
+            self.queue_depth = 0
+
+        def start(self):
+            self.port = 8003
+            return self.port
+
+        def stop(self):
+            pass
+
+    monkeypatch.setattr(
+        startup_mod,
+        "detect_hardware",
+        lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
+    )
+    monkeypatch.setattr(startup_mod, "TorchNodeServer", FakeTorchNodeServer)
+
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    tracker_url = f"http://127.0.0.1:{tracker_port}"
+    try:
+        for endpoint, shard_start, shard_end in (
+            ("http://203.0.113.30:8001", 0, 11),
+            ("http://203.0.113.31:8001", 12, 23),
+        ):
+            data = json.dumps({
+                "endpoint": endpoint,
+                "model": "Qwen2.5-0.5B-Instruct",
+                "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+                "num_layers": 24,
+                "shard_start": shard_start,
+                "shard_end": shard_end,
+                "tracker_mode": shard_start == 0,
+                "hardware_profile": {},
+                "score": 1.0,
+            }).encode()
+            req = urllib.request.Request(
+                f"{tracker_url}/v1/nodes/register",
+                data=data,
+                headers={"Content-Type": "application/json"},
+                method="POST",
+            )
+            with urllib.request.urlopen(req) as resp:
+                resp.read()
+
+        node = run_startup(
+            tracker_url=tracker_url,
+            advertise_host="203.0.113.32",
+            wallet_path=tmp_path / "wallet.json",
+        )
+        try:
+            assert captured["model_id"] == "Qwen/Qwen2.5-0.5B-Instruct"
+            assert captured["shard_start"] == 0
+        finally:
+            node.stop()
+    finally:
+        tracker.stop()
+
+
 # ---------------------------------------------------------------------------
 # Full startup integration test
 # ---------------------------------------------------------------------------
@@ -1134,12 +1568,7 @@ def test_full_startup_sequence(tmp_path):
         tracker.stop()
 
 
-def test_second_node_downloads_same_shard_from_peer_without_huggingface(
-    tmp_path,
-    monkeypatch,
-    capsys,
-):
-    """Node A downloads from HF stub; node B downloads same assignment from node A."""
+def test_preset_model_startup_starts_heartbeat(tmp_path, monkeypatch):
     import meshnet_node.startup as startup_mod
 
     monkeypatch.setattr(
@@ -1147,6 +1576,167 @@ def test_second_node_downloads_same_shard_from_peer_without_huggingface(
         "detect_hardware",
         lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
     )
+    heartbeat_calls = []
+    monkeypatch.setattr(
+        startup_mod,
+        "_start_heartbeat",
+        lambda *args, **kwargs: heartbeat_calls.append((args, kwargs)),
+    )
+
+    tracker = TrackerServer(model_presets={"stub-model": {"layers_start": 0, "layers_end": 15}})
+    tracker_port = tracker.start()
+    tracker_url = f"http://127.0.0.1:{tracker_port}"
+    try:
+        node = run_startup(
+            tracker_url=tracker_url,
+            model="stub-model",
+            wallet_path=tmp_path / "wallet.json",
+            cache_dir=tmp_path / "shards",
+        )
+        try:
+            assert len(heartbeat_calls) == 1
+            args, kwargs = heartbeat_calls[0]
+            assert args[0] == tracker_url
+            assert args[2]["model"] == "stub-model"
+            assert kwargs["node_ref"] is node
+        finally:
+            node.stop()
+    finally:
+        tracker.stop()
+
+
+def test_preset_model_startup_honors_pinned_shard_range(tmp_path, monkeypatch):
+    """Explicit --shard-start/--shard-end override tracker auto-assignment."""
+    import meshnet_node.startup as startup_mod
+
+    monkeypatch.setattr(
+        startup_mod,
+        "detect_hardware",
+        lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0, "ram_mb": 16 * 1024},
+    )
+    heartbeat_calls = []
+    monkeypatch.setattr(
+        startup_mod,
+        "_start_heartbeat",
+        lambda *args, **kwargs: heartbeat_calls.append((args, kwargs)),
+    )
+
+    tracker = TrackerServer(model_presets={"stub-model": {"layers_start": 0, "layers_end": 15}})
+    tracker_port = tracker.start()
+    tracker_url = f"http://127.0.0.1:{tracker_port}"
+    try:
+        node = run_startup(
+            tracker_url=tracker_url,
+            model="stub-model",
+            shard_start=0,
+            shard_end=5,
+            wallet_path=tmp_path / "wallet.json",
+            cache_dir=tmp_path / "shards",
+        )
+        try:
+            assert len(heartbeat_calls) == 1
+            args, kwargs = heartbeat_calls[0]
+            reg_payload = args[2]
+            assert reg_payload["shard_start"] == 0
+            assert reg_payload["shard_end"] == 5
+            assert reg_payload["managed_assignment"] is False
+        finally:
+            node.stop()
+    finally:
+        tracker.stop()
+
+
+def test_torch_startup_retries_registration_when_tracker_unreachable(
+    tmp_path,
+    monkeypatch,
+):
+    """Failed initial registration should start background retry, not stay unregistered."""
+    import meshnet_node.startup as startup_mod
+
+    class FakeBackend:
+        total_layers = 24
+
+    class FakeTorchNodeServer:
+        def __init__(self, **kwargs):
+            self.backend = FakeBackend()
+            self.port = None
+            self.chat_completion_count = 0
+            self.tracker_node_id = None
+
+        def start(self):
+            self.port = 7000
+            return self.port
+
+        def stop(self):
+            pass
+
+    monkeypatch.setattr(
+        startup_mod,
+        "detect_hardware",
+        lambda: {"device": "cuda", "gpu_name": "Test GPU", "vram_mb": 8192, "ram_mb": 16 * 1024},
+    )
+    monkeypatch.setattr(startup_mod, "TorchNodeServer", FakeTorchNodeServer)
+    monkeypatch.setattr(
+        startup_mod,
+        "_detect_num_layers",
+        lambda *_args, **_kwargs: 24,
+    )
+
+    heartbeat_calls = []
+    monkeypatch.setattr(
+        startup_mod,
+        "_start_heartbeat",
+        lambda *args, **kwargs: heartbeat_calls.append((args, kwargs)) or threading.Thread(),
+    )
+
+    register_calls = {"count": 0}
+
+    def flaky_register(url, payload):
+        register_calls["count"] += 1
+        raise urllib.error.URLError("connection refused")
+
+    monkeypatch.setattr(startup_mod, "_post_json", flaky_register)
+
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    tracker_url = f"http://127.0.0.1:{tracker_port}"
+    try:
+        node = run_startup(
+            tracker_url=tracker_url,
+            model_id="Qwen/Qwen2.5-0.5B-Instruct",
+            wallet_path=tmp_path / "wallet.json",
+        )
+        try:
+            assert register_calls["count"] == 1
+            assert node.tracker_node_id is None
+            assert len(heartbeat_calls) == 1
+            args, kwargs = heartbeat_calls[0]
+            assert args[1] == startup_mod._PENDING_NODE_ID
+            assert kwargs["node_ref"] is node
+        finally:
+            node.stop()
+    finally:
+        tracker.stop()
+
+
+def test_real_model_startup_registers_downloaded_inventory_without_checksum(
+    tmp_path,
+    monkeypatch,
+    capsys,
+):
+    """Real model folders are reported as inventory without hashing their contents."""
+    import meshnet_node.startup as startup_mod
+
+    monkeypatch.setattr(
+        startup_mod,
+        "detect_hardware",
+        lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
+    )
+    monkeypatch.setattr(
+        startup_mod,
+        "compute_shard_checksum",
+        lambda _path: (_ for _ in ()).throw(AssertionError("real model startup must not hash model files")),
+    )
     hf_calls = []
 
     def fake_snapshot_download(repo_id, cache_dir, local_dir):
@@ -1170,36 +1760,75 @@ def test_second_node_downloads_same_shard_from_peer_without_huggingface(
     })
     tracker_port = tracker.start()
     tracker_url = f"http://127.0.0.1:{tracker_port}"
-    nodes = []
     try:
-        node_a = run_startup(
+        node = run_startup(
             tracker_url=tracker_url,
             model="tiny-llama",
-            wallet_path=tmp_path / "wallet-a.json",
-            cache_dir=tmp_path / "node-a-shards",
+            wallet_path=tmp_path / "wallet.json",
+            cache_dir=tmp_path / "node-shards",
         )
-        nodes.append(node_a)
-        assert len(hf_calls) == 1
-
-        node_b = run_startup(
-            tracker_url=tracker_url,
-            model="tiny-llama",
-            wallet_path=tmp_path / "wallet-b.json",
-            cache_dir=tmp_path / "node-b-shards",
-        )
-        nodes.append(node_b)
-
-        assert len(hf_calls) == 1
-        assert (tmp_path / "node-b-shards" / "tiny-llama" / "layers_0-15" / "weights.json").exists()
-        output = capsys.readouterr().out
-        assert "download source: HuggingFace" in output
-        assert "download source: peer" in output
-    finally:
-        for node in reversed(nodes):
+        try:
+            assert len(hf_calls) == 1
+            assert (tmp_path / "node-shards" / "tiny-llama" / "weights.json").exists()
+            output = capsys.readouterr().out
+            assert "Cached at:" in output
+            network_map = _get_json(f"{tracker_url}/v1/network/map")
+            registered = network_map["nodes"][0]
+            assert registered["downloaded_models"] == [{
+                "model": "tiny-llama",
+                "shard_start": 0,
+                "shard_end": 15,
+                "path": str(tmp_path / "node-shards" / "tiny-llama"),
+                "file_count": 1,
+                "total_bytes": (tmp_path / "node-shards" / "tiny-llama" / "weights.json").stat().st_size,
+                "hf_repo": "org/tiny-llama-shards",
+            }]
+        finally:
             node.stop()
+    finally:
         tracker.stop()
 
 
+def test_downloaded_model_inventory_reports_local_model_percentage(tmp_path):
+    import meshnet_node.startup as startup_mod
+
+    model_dir = tmp_path / "models" / "tiny-llama"
+    model_dir.mkdir(parents=True)
+    (model_dir / "config.json").write_bytes(b"{}")
+    (model_dir / "weights-a.safetensors").write_bytes(b"a" * 3)
+
+    inventory = startup_mod._downloaded_model_inventory(
+        "tiny-llama",
+        0,
+        1,
+        model_dir,
+        hf_repo="org/tiny-llama",
+        model_sources=[{
+            "full_files": ["config.json", "weights-a.safetensors", "weights-b.safetensors"],
+            "file_sizes": {
+                "config.json": 2,
+                "weights-a.safetensors": 3,
+                "weights-b.safetensors": 5,
+            },
+        }],
+    )
+
+    assert inventory == [{
+        "model": "tiny-llama",
+        "shard_start": 0,
+        "shard_end": 1,
+        "path": str(model_dir),
+        "file_count": 2,
+        "total_bytes": 5,
+        "hf_repo": "org/tiny-llama",
+        "expected_file_count": 3,
+        "local_expected_file_count": 2,
+        "expected_bytes": 10,
+        "local_expected_bytes": 5,
+        "local_model_percentage": 50.0,
+    }]
+
+
 def test_network_assign_gap_found_field():
     """network/assign sets gap_found=True when a real gap exists, False when fully covered."""
     import json as _json
@@ -1387,3 +2016,152 @@ def test_startup_cpu_fallback(tmp_path, monkeypatch):
             node.stop()
     finally:
         tracker.stop()
+
+
+# --------------------------------------------------- layer detection (US: composite configs)
+
+
+def test_detect_num_layers_prefers_flattened_local_model_config(tmp_path, monkeypatch):
+    import meshnet_node.startup as startup_mod
+
+    model_dir = tmp_path / "Qwen3.6-35B-A3B"
+    model_dir.mkdir()
+    (model_dir / "config.json").write_text("{}")
+    calls = []
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id, cache_dir=None):
+            calls.append({"model_id": model_id, "cache_dir": cache_dir})
+            return types.SimpleNamespace(num_hidden_layers=37)
+
+    monkeypatch.setitem(
+        sys.modules,
+        "transformers",
+        types.SimpleNamespace(AutoConfig=AutoConfigStub),
+    )
+
+    assert startup_mod._detect_num_layers("unsloth/Qwen3.6-35B-A3B", cache_dir=tmp_path) == 37
+    assert calls == [{"model_id": str(model_dir), "cache_dir": None}]
+
+
+def test_layers_from_config_top_level():
+    from meshnet_node.model_catalog import layers_from_config
+
+    cfg = types.SimpleNamespace(num_hidden_layers=24)
+    assert layers_from_config(cfg) == 24
+
+
+def test_layers_from_config_nested_text_config():
+    """VLM/MoE composites (e.g. Qwen3.5-MoE) keep the layer count in text_config."""
+    from meshnet_node.model_catalog import layers_from_config
+
+    cfg = types.SimpleNamespace(text_config=types.SimpleNamespace(num_hidden_layers=40))
+    assert layers_from_config(cfg) == 40
+
+
+def test_layers_from_config_get_text_config_and_variants():
+    from meshnet_node.model_catalog import layers_from_config
+
+    inner = types.SimpleNamespace(n_layer=32)
+    cfg = types.SimpleNamespace(get_text_config=lambda: inner)
+    assert layers_from_config(cfg) == 32
+    assert layers_from_config(types.SimpleNamespace()) is None
+
+
+def test_download_dir_env_override(tmp_path, monkeypatch):
+    import importlib
+
+    from meshnet_node import config as config_mod
+
+    monkeypatch.chdir(tmp_path)
+    monkeypatch.setenv("MESHNET_DOWNLOAD_DIR", "/tmp/llm-store")
+    importlib.reload(config_mod)
+    assert config_mod.DEFAULTS["download_dir"] == "/tmp/llm-store"
+    monkeypatch.delenv("MESHNET_DOWNLOAD_DIR")
+    importlib.reload(config_mod)
+    assert config_mod.DEFAULTS["download_dir"].endswith("models")
+
+
+def test_cli_loads_local_env_before_config_defaults(tmp_path, monkeypatch):
+    import importlib
+
+    from meshnet_node import cli as cli_mod
+    from meshnet_node import config as config_mod
+
+    monkeypatch.delenv("MESHNET_DOWNLOAD_DIR", raising=False)
+    monkeypatch.chdir(tmp_path)
+    (tmp_path / ".env").write_text(
+        "MESHNET_DOWNLOAD_DIR=/run/media/popov/DATA/llm/safetensor/models\n"
+        "HF_TOKEN=hf_test_token\n"
+    )
+
+    cli_mod._load_env_defaults()
+    importlib.reload(config_mod)
+
+    assert config_mod.DEFAULTS["download_dir"] == "/run/media/popov/DATA/llm/safetensor/models"
+    assert os.environ["HF_TOKEN"] == "hf_test_token"
+
+
+def test_default_quantization_is_auto(monkeypatch):
+    import importlib
+
+    from meshnet_node import config as config_mod
+    from meshnet_node.model_backend import validate_quantization
+
+    monkeypatch.delenv("MESHNET_DOWNLOAD_DIR", raising=False)
+    importlib.reload(config_mod)
+
+    assert config_mod.DEFAULTS["quantization"] == "auto"
+    assert validate_quantization("auto") == "auto"
+
+
+def test_auto_quantization_uses_native_model_dtype_for_unquantized_config():
+    from meshnet_node.model_backend import _model_load_plan
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id, cache_dir=None):
+            assert model_id == "repo/model"
+            assert cache_dir is None
+            return types.SimpleNamespace(
+                text_config=types.SimpleNamespace(dtype="torch.bfloat16"),
+            )
+
+    torch_stub = types.SimpleNamespace(bfloat16="bf16", float16="fp16")
+
+    quant_config, dtype, uses_quantized_weights = _model_load_plan(
+        AutoConfigStub,
+        "repo/model",
+        "auto",
+        torch_stub,
+    )
+
+    assert quant_config is None
+    assert dtype == "bf16"
+    assert uses_quantized_weights is False
+
+
+def test_auto_quantization_preserves_native_quantized_config():
+    from meshnet_node.model_backend import _model_load_plan
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id, cache_dir=None):
+            return types.SimpleNamespace(
+                quantization_config={"quant_method": "gptq"},
+                torch_dtype="float16",
+            )
+
+    torch_stub = types.SimpleNamespace(bfloat16="bf16", float16="fp16")
+
+    quant_config, dtype, uses_quantized_weights = _model_load_plan(
+        AutoConfigStub,
+        "repo/model",
+        "auto",
+        torch_stub,
+    )
+
+    assert quant_config is None
+    assert dtype == "fp16"
+    assert uses_quantized_weights is True

tests/test_real_model_backend.py

@@ -4,6 +4,8 @@ import json
 import os
 from pathlib import Path
 import sys
+import threading
+import time
 import types
 import urllib.request
 
@@ -11,8 +13,12 @@ import pytest
 
 from meshnet_node.model_backend import (
     InsufficientVRAMError,
+    PartialModelLoadUnsupported,
     TensorPayload,
+    TorchModelShard,
     _call_layer,
+    _load_partial_model_from_snapshot,
+    _should_partial_materialize_shard,
     _decoder_attention_mask,
     _int_tensor_header,
     build_quantization_config,
@@ -94,7 +100,7 @@ class _FakePipelineHeadBackend(_FakeBackend):
     tokenizer = _FakeChatTokenizer()
 
     def encode_prompt(self, prompt: str) -> TensorPayload:
-        assert prompt == "debug prompt"
+        assert prompt.startswith("debug prompt")
         return TensorPayload(
             body=b"\x00" * (1 * 6 * 8 * 2),
             shape=[1, 6, 8],
@@ -113,6 +119,19 @@ class _FakePipelineTailBackend(_FakeTailBackend):
         return " token"
 
 
+class _BlockingStreamingTailBackend(_FakeTailBackend):
+    def __init__(self, second_token_release: threading.Event) -> None:
+        self._release = second_token_release
+        self.calls = 0
+
+    def forward_bytes(self, body, shape, attention_mask_header, position_ids_header, start_layer=None):
+        self.calls += 1
+        if self.calls == 1:
+            return " first"
+        self._release.wait(timeout=3.0)
+        return " second"
+
+
 def test_quantization_flag_validation():
     assert validate_quantization("bfloat16") == "bfloat16"
     assert validate_quantization("int8") == "int8"
@@ -299,6 +318,56 @@ def test_pipeline_hop_logs_are_enabled_with_debug(capsys):
     assert "  [node] pipeline hop 0 returned text=' token'" in out
 
 
+def test_split_shard_chat_streams_each_generated_token_incrementally():
+    release_second = threading.Event()
+    head = TorchNodeServer(backend=_FakePipelineHeadBackend(), tracker_mode=True)
+    tail = TorchNodeServer(backend=_BlockingStreamingTailBackend(release_second))
+    head_port = head.start()
+    tail_port = tail.start()
+    response = None
+    try:
+        payload = json.dumps({
+            "model": "fake-model",
+            "messages": [{"role": "user", "content": "hello"}],
+            "stream": True,
+            "max_tokens": 2,
+        }).encode()
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{head_port}/v1/chat/completions",
+            data=payload,
+            headers={
+                "Content-Type": "application/json",
+                "X-Meshnet-Route": json.dumps([
+                    {"endpoint": f"http://127.0.0.1:{tail_port}", "start_layer": 22},
+                ]),
+            },
+            method="POST",
+        )
+        response = urllib.request.urlopen(req, timeout=5)
+
+        first_token_line = ""
+        deadline = time.time() + 2.0
+        while time.time() < deadline:
+            line = response.readline().decode()
+            if '"content": " first"' in line:
+                first_token_line = line
+                break
+
+        assert first_token_line
+        assert not release_second.is_set()
+        release_second.set()
+        rest = response.read().decode()
+    finally:
+        release_second.set()
+        if response is not None:
+            response.close()
+        head.stop()
+        tail.stop()
+
+    assert '"content": " second"' in rest
+    assert "data: [DONE]" in rest
+
+
 def test_int_tensor_header_serializes_torch_tensors():
     torch = pytest.importorskip("torch")
 
@@ -334,6 +403,295 @@ def test_call_layer_passes_rotary_position_embeddings():
     ) == "hidden"
 
 
+def test_partial_materialize_guard_requires_local_non_full_non_quantized_snapshot(tmp_path):
+    snapshot_dir = tmp_path / "snapshot"
+    snapshot_dir.mkdir()
+    (snapshot_dir / "config.json").write_text("{}")
+    (snapshot_dir / "model.safetensors.index.json").write_text('{"weight_map": {}}')
+
+    assert _should_partial_materialize_shard(
+        str(snapshot_dir),
+        4,
+        7,
+        total_layers_hint=40,
+        uses_quantized_weights=False,
+    ) is True
+    assert _should_partial_materialize_shard(
+        str(snapshot_dir),
+        0,
+        39,
+        total_layers_hint=40,
+        uses_quantized_weights=False,
+    ) is False
+    assert _should_partial_materialize_shard(
+        str(snapshot_dir),
+        4,
+        7,
+        total_layers_hint=40,
+        uses_quantized_weights=True,
+    ) is False
+    assert _should_partial_materialize_shard(
+        "repo/model",
+        4,
+        7,
+        total_layers_hint=40,
+        uses_quantized_weights=False,
+    ) is False
+
+
+def test_partial_snapshot_loader_materializes_only_assigned_tensors(tmp_path):
+    snapshot_dir = tmp_path / "snapshot"
+    snapshot_dir.mkdir()
+    (snapshot_dir / "config.json").write_text("{}")
+    (snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({
+        "weight_map": {
+            "model.embed_tokens.weight": "shard-1.safetensors",
+            "model.layers.0.self_attn.q_proj.weight": "shard-1.safetensors",
+            "model.layers.1.self_attn.q_proj.weight": "shard-2.safetensors",
+            "model.layers.2.self_attn.q_proj.weight": "shard-3.safetensors",
+            "model.norm.weight": "shard-3.safetensors",
+            "lm_head.weight": "shard-3.safetensors",
+        }
+    }))
+    for rel in ("shard-1.safetensors", "shard-2.safetensors", "shard-3.safetensors"):
+        (snapshot_dir / rel).write_bytes(b"stub")
+
+    class FakeModule:
+        def __init__(self, name):
+            self.name = name
+            self.to_calls = []
+
+        def to(self, device):
+            self.to_calls.append(device)
+            return self
+
+    class FakeModel:
+        def __init__(self):
+            self.model = types.SimpleNamespace(
+                embed_tokens=FakeModule("embed"),
+                layers=[FakeModule("layer0"), FakeModule("layer1"), FakeModule("layer2")],
+                rotary_emb=FakeModule("rotary"),
+                norm=FakeModule("norm"),
+            )
+            self.lm_head = FakeModule("lm_head")
+            self.tie_weights_called = 0
+
+        def tie_weights(self):
+            self.tie_weights_called += 1
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id):
+            assert model_id == str(snapshot_dir)
+            return types.SimpleNamespace(num_hidden_layers=3)
+
+    class AutoModelStub:
+        @staticmethod
+        def from_config(cfg, torch_dtype=None):
+            assert cfg.num_hidden_layers == 3
+            assert torch_dtype == "bf16"
+            return FakeModel()
+
+    class EmptyWeights:
+        def __init__(self):
+            self.entered = 0
+            self.exited = 0
+
+        def __call__(self):
+            return self
+
+        def __enter__(self):
+            self.entered += 1
+            return None
+
+        def __exit__(self, exc_type, exc, tb):
+            self.exited += 1
+            return False
+
+    init_empty_weights = EmptyWeights()
+    set_calls = []
+
+    def fake_set_tensor(module, tensor_name, device, value=None, dtype=None):
+        set_calls.append((tensor_name, device, value, dtype))
+
+    tensors = {
+        "shard-1.safetensors": {
+            "model.embed_tokens.weight": "embed",
+            "model.layers.0.self_attn.q_proj.weight": "layer0",
+        },
+        "shard-2.safetensors": {
+            "model.layers.1.self_attn.q_proj.weight": "layer1",
+        },
+        "shard-3.safetensors": {
+            "model.layers.2.self_attn.q_proj.weight": "layer2",
+            "model.norm.weight": "norm",
+            "lm_head.weight": "lm_head",
+        },
+    }
+
+    class FakeSafeOpen:
+        def __init__(self, filename, framework, device):
+            assert framework == "pt"
+            assert device == "cpu"
+            self.filename = Path(filename).name
+
+        def __enter__(self):
+            return self
+
+        def __exit__(self, exc_type, exc, tb):
+            return False
+
+        def get_tensor(self, tensor_name):
+            return tensors[self.filename][tensor_name]
+
+    model = _load_partial_model_from_snapshot(
+        AutoConfigStub,
+        AutoModelStub,
+        types.SimpleNamespace(),
+        str(snapshot_dir),
+        1,
+        1,
+        "bf16",
+        "cpu:0",
+        init_empty_weights_fn=init_empty_weights,
+        set_tensor_fn=fake_set_tensor,
+        safe_open_fn=FakeSafeOpen,
+    )
+
+    assert init_empty_weights.entered == 1
+    assert init_empty_weights.exited == 1
+    assert model.tie_weights_called == 1
+    assert [call[0] for call in set_calls] == ["model.layers.1.self_attn.q_proj.weight"]
+    assert model.model.layers[1].to_calls == ["cpu:0"]
+    assert model.model.layers[0].to_calls == []
+    assert model.model.layers[2].to_calls == []
+    assert model.model.embed_tokens.to_calls == []
+    assert model.model.norm.to_calls == []
+    assert model.lm_head.to_calls == []
+    assert model.model.rotary_emb.to_calls == ["cpu:0"]
+
+
+def test_partial_snapshot_loader_requires_known_layer_count(tmp_path):
+    snapshot_dir = tmp_path / "snapshot"
+    snapshot_dir.mkdir()
+    (snapshot_dir / "config.json").write_text("{}")
+    (snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({
+        "weight_map": {"model.layers.0.self_attn.q_proj.weight": "shard.safetensors"}
+    }))
+    (snapshot_dir / "shard.safetensors").write_bytes(b"stub")
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id):
+            return types.SimpleNamespace()
+
+    class AutoModelStub:
+        @staticmethod
+        def from_config(cfg, torch_dtype=None):
+            raise AssertionError("from_config should not run without a known layer count")
+
+    class UnusedContext:
+        def __enter__(self):
+            return None
+
+        def __exit__(self, exc_type, exc, tb):
+            return False
+
+    with pytest.raises(PartialModelLoadUnsupported, match="num_hidden_layers"):
+        _load_partial_model_from_snapshot(
+            AutoConfigStub,
+            AutoModelStub,
+            types.SimpleNamespace(),
+            str(snapshot_dir),
+            0,
+            0,
+            "bf16",
+            "cpu:0",
+            init_empty_weights_fn=lambda: UnusedContext(),
+            set_tensor_fn=lambda *args, **kwargs: None,
+            safe_open_fn=lambda *args, **kwargs: None,
+        )
+
+
+def test_torch_model_shard_prefers_partial_loader_for_local_snapshot(tmp_path, monkeypatch):
+    import meshnet_node.model_backend as backend
+
+    snapshot_dir = tmp_path / "snapshot"
+    snapshot_dir.mkdir()
+    (snapshot_dir / "config.json").write_text("{}")
+    (snapshot_dir / "model.safetensors.index.json").write_text('{"weight_map": {}}')
+
+    class FakeModel:
+        def __init__(self):
+            self.model = types.SimpleNamespace(
+                layers=[object(), object(), object()],
+                embed_tokens=object(),
+            )
+            self.config = types.SimpleNamespace(hidden_size=8)
+            self.eval_called = 0
+
+        def eval(self):
+            self.eval_called += 1
+
+    fake_model = FakeModel()
+    partial_calls = []
+
+    class AutoConfigStub:
+        @staticmethod
+        def from_pretrained(model_id, cache_dir=None):
+            return types.SimpleNamespace(num_hidden_layers=3, text_config=types.SimpleNamespace(dtype="torch.bfloat16"))
+
+    class AutoModelStub:
+        @staticmethod
+        def from_pretrained(*args, **kwargs):
+            raise AssertionError("full model load should not run for partial local shards")
+
+    class AutoTokenizerStub:
+        @staticmethod
+        def from_pretrained(model_id, cache_dir=None):
+            assert model_id == str(snapshot_dir)
+            return types.SimpleNamespace()
+
+    monkeypatch.setitem(
+        sys.modules,
+        "torch",
+        types.SimpleNamespace(
+            cuda=types.SimpleNamespace(is_available=lambda: False),
+            device=lambda value: value,
+            bfloat16="bf16",
+        ),
+    )
+    monkeypatch.setitem(
+        sys.modules,
+        "transformers",
+        types.SimpleNamespace(
+            AutoConfig=AutoConfigStub,
+            AutoModelForCausalLM=AutoModelStub,
+            AutoTokenizer=AutoTokenizerStub,
+        ),
+    )
+    monkeypatch.setattr(
+        backend,
+        "_load_partial_model_from_snapshot",
+        lambda *args, **kwargs: partial_calls.append((args, kwargs)) or fake_model,
+    )
+
+    shard = TorchModelShard(
+        "repo/model",
+        1,
+        1,
+        quantization="auto",
+        cache_dir=snapshot_dir,
+    )
+
+    assert len(partial_calls) == 1
+    assert shard.model is fake_model
+    assert fake_model.eval_called == 1
+    assert shard.total_layers == 3
+    assert shard.is_head is False
+    assert shard.is_tail is False
+
+
 @pytest.mark.integration
 def test_two_node_gpt2_completion_is_deterministic():
     if os.environ.get("CI"):

tests/test_safetensors_selection.py

@@ -0,0 +1,86 @@
+"""Tests for layer-aware SafeTensors snapshot file selection."""
+
+import json
+
+import pytest
+
+from meshnet_node.safetensors_selection import select_safetensors_files_for_layers
+
+
+def _write_snapshot(tmp_path, *, config=None):
+    (tmp_path / "config.json").write_text(
+        json.dumps(config or {"num_hidden_layers": 5}),
+        encoding="utf-8",
+    )
+    (tmp_path / "tokenizer.json").write_text("{}", encoding="utf-8")
+    (tmp_path / "tokenizer_config.json").write_text("{}", encoding="utf-8")
+    (tmp_path / "README.md").write_text("not part of runtime snapshot", encoding="utf-8")
+    (tmp_path / "model.safetensors.index.json").write_text(
+        json.dumps({
+            "metadata": {"total_size": 123},
+            "weight_map": {
+                "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
+                "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+                "model.layers.1.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+                "model.layers.2.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+                "model.layers.3.mlp.down_proj.weight": "nested/model-00003-of-00004.safetensors",
+                "model.layers.4.self_attn.q_proj.weight": "nested/model-00003-of-00004.safetensors",
+                "model.norm.weight": "model-00004-of-00004.safetensors",
+                "lm_head.weight": "model-00004-of-00004.safetensors",
+            },
+        }),
+        encoding="utf-8",
+    )
+
+
+def test_selects_only_weight_shards_for_middle_layer_range(tmp_path):
+    _write_snapshot(tmp_path)
+
+    files = select_safetensors_files_for_layers(tmp_path, 2, 3)
+
+    assert files == [
+        "config.json",
+        "model-00002-of-00004.safetensors",
+        "model.safetensors.index.json",
+        "nested/model-00003-of-00004.safetensors",
+        "tokenizer.json",
+        "tokenizer_config.json",
+    ]
+
+
+def test_head_range_includes_embeddings(tmp_path):
+    _write_snapshot(tmp_path)
+
+    files = select_safetensors_files_for_layers(tmp_path, 0, 0)
+
+    assert "model-00001-of-00004.safetensors" in files
+    assert "model-00004-of-00004.safetensors" not in files
+
+
+def test_tail_range_includes_norm_and_lm_head_from_inferred_layer_count(tmp_path):
+    _write_snapshot(tmp_path, config={"text_config": {"num_hidden_layers": 5}})
+
+    files = select_safetensors_files_for_layers(tmp_path, 4, 4)
+
+    assert "nested/model-00003-of-00004.safetensors" in files
+    assert "model-00004-of-00004.safetensors" in files
+    assert "model-00001-of-00004.safetensors" not in files
+
+
+def test_tail_files_are_not_selected_without_total_layer_count(tmp_path):
+    _write_snapshot(tmp_path, config={"architectures": ["UnknownForTest"]})
+
+    files = select_safetensors_files_for_layers(tmp_path, 4, 4)
+
+    assert "nested/model-00003-of-00004.safetensors" in files
+    assert "model-00004-of-00004.safetensors" not in files
+
+
+def test_rejects_unsafe_weight_map_paths(tmp_path):
+    (tmp_path / "model.safetensors.index.json").write_text(
+        json.dumps({"weight_map": {"model.layers.0.weight": "../escape.safetensors"}}),
+        encoding="utf-8",
+    )
+
+    with pytest.raises(ValueError, match="unsafe relative file"):
+        select_safetensors_files_for_layers(tmp_path, 0, 0)

tests/test_tracker_routing.py

@@ -13,7 +13,13 @@ from meshnet_gateway.server import GatewayServer, _banned_route_wallet
 from meshnet_node.server import StubNodeServer
 from meshnet_contracts import LocalSolanaContracts
 from meshnet_tracker.auth import sign_hive_request
-from meshnet_tracker.server import TrackerServer, _NodeEntry, _registration_ban_error
+from meshnet_tracker.server import (
+    TrackerServer,
+    _NodeEntry,
+    _memory_pool_map,
+    _registration_ban_error,
+    _scale_demanded_models_locked,
+)
 
 _TEST_HIVE_SECRET = "test-hive-secret"
 
@@ -75,6 +81,13 @@ def test_tracker_exposes_registered_model_metadata():
                     "activated_parameters": "32B",
                     "context_length": 256000,
                 },
+                "downloaded_models": [{
+                    "model": "Kimi-K2.7-Code",
+                    "hf_repo": "unsloth/Kimi-K2.7-Code",
+                    "shard_start": 0,
+                    "shard_end": 60,
+                    "local_model_percentage": 100.0,
+                }],
             },
         )
 
@@ -91,6 +104,7 @@ def test_tracker_exposes_registered_model_metadata():
     registered = network_map["nodes"][0]
     assert registered["num_layers"] == 61
     assert registered["model_metadata"]["context_length"] == 256000
+    assert registered["downloaded_models"][0]["local_model_percentage"] == 100.0
 
 
 def test_tracker_lists_recommended_kimi_before_nodes_register():
@@ -154,6 +168,59 @@ def test_network_map_exposes_pool_size_and_speed_summary():
     assert pool["total_effective_throughput"] == 10.0
 
 
+def test_network_map_exposes_served_model_copy_count():
+    tracker = TrackerServer()
+    port = tracker.start()
+    url = f"http://127.0.0.1:{port}"
+    try:
+        _post_json(
+            f"{url}/v1/nodes/register",
+            {
+                "endpoint": "http://127.0.0.1:7201",
+                "model": "copy-count-test",
+                "hf_repo": "example/copy-count-test",
+                "num_layers": 37,
+                "shard_start": 0,
+                "shard_end": 21,
+                "hardware_profile": {},
+            },
+        )
+        network_map = _get_json(f"{url}/v1/network/map")
+        assert network_map["nodes"][0]["model_supply"]["served_model_copies"] == 0.59
+
+        _post_json(
+            f"{url}/v1/nodes/register",
+            {
+                "endpoint": "http://127.0.0.1:7202",
+                "model": "copy-count-test",
+                "hf_repo": "example/copy-count-test",
+                "num_layers": 37,
+                "shard_start": 22,
+                "shard_end": 36,
+                "hardware_profile": {},
+            },
+        )
+        network_map = _get_json(f"{url}/v1/network/map")
+        assert network_map["nodes"][0]["model_supply"]["served_model_copies"] == 1.0
+
+        _post_json(
+            f"{url}/v1/nodes/register",
+            {
+                "endpoint": "http://127.0.0.1:7203",
+                "model": "copy-count-test",
+                "hf_repo": "example/copy-count-test",
+                "num_layers": 37,
+                "shard_start": 0,
+                "shard_end": 36,
+                "hardware_profile": {},
+            },
+        )
+        network_map = _get_json(f"{url}/v1/network/map")
+        assert network_map["nodes"][0]["model_supply"]["served_model_copies"] == 2.0
+    finally:
+        tracker.stop()
+
+
 def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
     tracker = TrackerServer()
     port = tracker.start()
@@ -263,6 +330,177 @@ def test_tracker_serves_health_while_proxy_request_is_in_flight():
         slow_thread.join(timeout=1.0)
 
 
+def test_tracker_route_log_counts_proxy_inflight_requests():
+    entered = threading.Event()
+    release = threading.Event()
+
+    class SlowChatHandler(http.server.BaseHTTPRequestHandler):
+        def log_message(self, fmt, *args):
+            pass
+
+        def do_POST(self):
+            if self.path != "/v1/chat/completions":
+                self.send_response(404)
+                self.end_headers()
+                return
+            length = int(self.headers.get("Content-Length", 0))
+            self.rfile.read(length)
+            entered.set()
+            release.wait(timeout=3.0)
+            body = json.dumps({"choices": [{"message": {"content": "ok"}}]}).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+    slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
+    slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
+    slow_thread.start()
+    tracker = TrackerServer(heartbeat_timeout=60.0)
+    tracker_port = tracker.start()
+    errors = []
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": f"http://127.0.0.1:{slow_node.server_address[1]}",
+             "model": "burst-model", "num_layers": 1,
+             "shard_start": 0, "shard_end": 0,
+             "hardware_profile": {}, "score": 1.0},
+        )
+
+        def call_proxy():
+            try:
+                _post_json(
+                    f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+                    {"model": "burst-model", "messages": [{"role": "user", "content": "hi"}]},
+                )
+            except Exception as exc:
+                errors.append(exc)
+
+        first = threading.Thread(target=call_proxy)
+        second = threading.Thread(target=call_proxy)
+        first.start()
+        assert entered.wait(timeout=2.0)
+        second.start()
+
+        selected_events = []
+        deadline = time.time() + 2.0
+        while time.time() < deadline:
+            console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
+            selected_events = [
+                event for event in console["events"]
+                if event["message"] == "proxy route selected"
+            ]
+            if len(selected_events) >= 2:
+                break
+            time.sleep(0.05)
+
+        assert len(selected_events) >= 2
+        second_nodes = selected_events[-1]["fields"]["nodes"]
+        assert second_nodes[0]["queue_depth"] == 2
+        assert second_nodes[0]["proxy_inflight"] == 2
+    finally:
+        release.set()
+        first.join(timeout=3.0)
+        second.join(timeout=3.0)
+        tracker.stop()
+        slow_node.shutdown()
+        slow_node.server_close()
+        slow_thread.join(timeout=1.0)
+
+    assert not first.is_alive()
+    assert not second.is_alive()
+    assert not errors
+
+
+def test_tracker_logs_stream_progress_before_request_completes():
+    chunk_sent = threading.Event()
+    release = threading.Event()
+
+    class StreamingChatHandler(http.server.BaseHTTPRequestHandler):
+        def log_message(self, fmt, *args):
+            pass
+
+        def do_POST(self):
+            if self.path != "/v1/chat/completions":
+                self.send_response(404)
+                self.end_headers()
+                return
+            self.rfile.read(int(self.headers.get("Content-Length", 0)))
+            self.send_response(200)
+            self.send_header("Content-Type", "text/event-stream; charset=utf-8")
+            self.end_headers()
+            payload = json.dumps({
+                "choices": [{"delta": {"content": "hello world"}}],
+            }).encode()
+            self.wfile.write(b"data: " + payload + b"\n\n")
+            self.wfile.flush()
+            chunk_sent.set()
+            release.wait(timeout=3.0)
+            self.wfile.write(b"data: [DONE]\n\n")
+            self.wfile.flush()
+
+    node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
+    node_thread = threading.Thread(target=node.serve_forever, daemon=True)
+    node_thread.start()
+    tracker = TrackerServer(heartbeat_timeout=60.0)
+    tracker_port = tracker.start()
+    response = None
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": f"http://127.0.0.1:{node.server_address[1]}",
+             "model": "stream-progress-model", "num_layers": 1,
+             "shard_start": 0, "shard_end": 0,
+             "hardware_profile": {}, "score": 1.0},
+        )
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+            data=json.dumps({
+                "model": "stream-progress-model",
+                "stream": True,
+                "messages": [{"role": "user", "content": "hi"}],
+            }).encode(),
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        response = urllib.request.urlopen(req, timeout=3.0)
+        first_line = response.readline()
+        assert first_line.startswith(b"data:")
+        assert chunk_sent.wait(timeout=1.0)
+
+        progress_events = []
+        deadline = time.time() + 2.0
+        while time.time() < deadline:
+            console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
+            progress_events = [
+                event for event in console["events"]
+                if event["message"] == "proxy progress"
+            ]
+            if progress_events:
+                break
+            time.sleep(0.05)
+
+        assert progress_events
+        fields = progress_events[-1]["fields"]
+        assert fields["tokens"] == 2
+        assert fields["tokens_per_sec"] > 0
+        active = [
+            event for event in console["events"]
+            if event["message"] == "proxy route selected"
+        ]
+        assert active
+    finally:
+        release.set()
+        if response is not None:
+            response.close()
+        tracker.stop()
+        node.shutdown()
+        node.server_close()
+        node_thread.join(timeout=1.0)
+
+
 def test_tracker_routes_hf_model_alias_from_quickstart():
     """The documented qwen2.5-0.5b alias resolves a full HF repo registration."""
     tracker = TrackerServer()
@@ -347,6 +585,229 @@ def test_tracker_proxy_accepts_hf_model_alias_from_quickstart():
     assert response["choices"][0]["message"]["content"] == "56"
 
 
+def test_tracker_proxy_routes_split_preset_nodes_by_alias():
+    """The chat proxy must build routes from alias-matched preset nodes."""
+
+    class ChatHandler(http.server.BaseHTTPRequestHandler):
+        def log_message(self, fmt, *args):
+            pass
+
+        def do_POST(self):
+            if self.path != "/v1/chat/completions":
+                self.send_response(404)
+                self.end_headers()
+                return
+            length = int(self.headers.get("Content-Length", 0))
+            self.rfile.read(length)
+            route_header = self.headers.get("X-Meshnet-Route", "[]")
+            body = json.dumps({
+                "choices": [{"message": {"content": route_header}}],
+                "usage": {"prompt_tokens": 1, "completion_tokens": 1},
+            }).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+    head = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
+    head_thread = threading.Thread(target=head.serve_forever, daemon=True)
+    head_thread.start()
+    tracker = TrackerServer(model_presets={
+        "qwen3.6-35b-a3b": {
+            "layers_start": 0,
+            "layers_end": 39,
+            "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+            "aliases": ["Qwen3.6-35B-A3B"],
+        }
+    })
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": f"http://127.0.0.1:{head.server_address[1]}",
+             "model": "qwen3.6-35b-a3b",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 0,
+             "shard_end": 21,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9199",
+             "model": "Qwen3.6-35B-A3B",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 22,
+             "shard_end": 39,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+
+        response = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+            {"model": "Qwen3.6-35B-A3B",
+             "messages": [{"role": "user", "content": "hi"}]},
+        )
+        console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
+    finally:
+        tracker.stop()
+        head.shutdown()
+        head.server_close()
+        head_thread.join(timeout=1.0)
+
+    route = json.loads(response["choices"][0]["message"]["content"])
+    assert route == [{"endpoint": "http://127.0.0.1:9199", "start_layer": 22}]
+    complete = [event for event in console["events"] if event["message"] == "proxy complete"][-1]
+    assert complete["fields"]["tokens"] == 2
+    assert complete["fields"]["tokens_per_sec"] >= 0
+
+
+def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
+    tracker = TrackerServer(model_presets={
+        "qwen3.6-35b-a3b": {
+            "layers_start": 0,
+            "layers_end": 39,
+            "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+            "aliases": ["Qwen3.6-35B-A3B"],
+        }
+    })
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9101",
+             "model": "qwen3.6-35b-a3b",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 0,
+             "shard_end": 21,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9102",
+             "model": "Qwen3.6-35B-A3B",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 22,
+             "shard_end": 39,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+
+        response = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/route?model=Qwen3.6-35B-A3B"
+        )
+    finally:
+        tracker.stop()
+
+    assert response["route"] == ["http://127.0.0.1:9101", "http://127.0.0.1:9102"]
+    assert [node["start_layer"] for node in response["nodes"]] == [0, 22]
+
+
+def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
+    tracker = TrackerServer(model_presets={
+        "qwen3.6-35b-a3b": {
+            "layers_start": 0,
+            "layers_end": 39,
+            "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+            "aliases": ["Qwen3.6-35B-A3B"],
+        }
+    })
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9101",
+             "model": "qwen3.6-35b-a3b",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 0,
+             "shard_end": 39,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+
+        response = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/route?model=%20Qwen3.6-35B-A3B%20"
+        )
+    finally:
+        tracker.stop()
+
+    assert response["route"] == ["http://127.0.0.1:9101"]
+
+
+def test_tracker_proxy_ignores_model_case_and_outer_whitespace():
+    class ChatHandler(http.server.BaseHTTPRequestHandler):
+        def log_message(self, fmt, *args):
+            pass
+
+        def do_POST(self):
+            if self.path != "/v1/chat/completions":
+                self.send_response(404)
+                self.end_headers()
+                return
+            length = int(self.headers.get("Content-Length", 0))
+            request_body = json.loads(self.rfile.read(length) or b"{}")
+            body = json.dumps({
+                "model": request_body["model"],
+                "choices": [{"message": {"content": "ok"}}],
+            }).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+    node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
+    node_thread = threading.Thread(target=node.serve_forever, daemon=True)
+    node_thread.start()
+    tracker = TrackerServer(model_presets={
+        "qwen3.6-35b-a3b": {
+            "layers_start": 0,
+            "layers_end": 39,
+            "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+            "aliases": ["Qwen3.6-35B-A3B"],
+        }
+    })
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": f"http://127.0.0.1:{node.server_address[1]}",
+             "model": "qwen3.6-35b-a3b",
+             "hf_repo": "unsloth/Qwen3.6-35B-A3B",
+             "num_layers": 40,
+             "shard_start": 0,
+             "shard_end": 39,
+             "tracker_mode": True,
+             "hardware_profile": {},
+             "score": 1.0},
+        )
+
+        response = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+            {"model": "Qwen3.6-35B-A3B ",
+             "messages": [{"role": "user", "content": "hi"}]},
+        )
+    finally:
+        tracker.stop()
+        node.shutdown()
+        node.server_close()
+        node_thread.join(timeout=1.0)
+
+    assert response["choices"][0]["message"]["content"] == "ok"
+
+
 def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix():
     tracker = TrackerServer()
     tracker_port = tracker.start()
@@ -659,12 +1120,14 @@ def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
              "benchmark_tokens_per_sec": 3.0, "hardware_profile": {}, "score": 1.0},
         )
 
-        route_resp = _get_json(f"http://127.0.0.1:{tracker_port}/v1/route?model=tiny-model")
+        net = _get_json(f"http://127.0.0.1:{tracker_port}/v1/network/map")
         widths = {
             node["endpoint"]: node["shard_end"] - node["shard_start"] + 1
-            for node in route_resp["nodes"]
+            for node in net["nodes"]
         }
-        assert widths["http://127.0.0.1:9012"] > widths["http://127.0.0.1:9011"]
+        assert widths["http://127.0.0.1:9011"] == 12
+        assert widths["http://127.0.0.1:9012"] == 12
+        assert net["nodes"][0]["model_supply"]["served_model_copies"] == 2.0
     finally:
         tracker.stop()
 
@@ -692,7 +1155,8 @@ def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
         tracker.stop()
 
 
-def test_tracker_reassignment_emits_drop_before_load():
+def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
+    """A second managed node with capacity for the full model keeps the first copy intact."""
     tracker = TrackerServer(model_presets={
         "tiny-model": {
             "total_layers": 4,
@@ -701,21 +1165,69 @@ def test_tracker_reassignment_emits_drop_before_load():
     })
     tracker_port = tracker.start()
     try:
-        slow = _post_json(
+        first = _post_json(
             f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
             {"endpoint": "http://127.0.0.1:9015", "model": "tiny-model",
              "vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
              "benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
         )
-        _post_json(
+        second = _post_json(
             f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
             {"endpoint": "http://127.0.0.1:9016", "model": "tiny-model",
              "vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
              "benchmark_tokens_per_sec": 3.0, "hardware_profile": {}, "score": 1.0},
         )
 
-        hb = _post_json(f"http://127.0.0.1:{tracker_port}/v1/nodes/{slow['node_id']}/heartbeat", {})
-        assert [directive["action"] for directive in hb["directives"]] == ["DROP_SHARD", "LOAD_SHARD"]
+        net = _get_json(f"http://127.0.0.1:{tracker_port}/v1/network/map")
+        widths = {
+            node["endpoint"]: node["shard_end"] - node["shard_start"] + 1
+            for node in net["nodes"]
+        }
+        assert widths["http://127.0.0.1:9015"] == 4
+        assert widths["http://127.0.0.1:9016"] == 4
+        assert net["nodes"][0]["model_supply"]["served_model_copies"] == 2.0
+
+        hb = _post_json(f"http://127.0.0.1:{tracker_port}/v1/nodes/{first['node_id']}/heartbeat", {})
+        assert hb.get("directives", []) == []
+    finally:
+        tracker.stop()
+
+
+def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
+    """--model style joins with explicit shards add redundancy instead of reshuffling."""
+    tracker = TrackerServer(heartbeat_timeout=10.0)
+    tracker_port = tracker.start()
+    try:
+        base_reg = {
+            "model": "Qwen2.5-0.5B-Instruct",
+            "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+            "num_layers": 24,
+            "shard_start": 0,
+            "shard_end": 23,
+            "managed_assignment": True,
+            "vram_bytes": 2_000_000_000,
+            "hardware_profile": {},
+            "score": 1.0,
+        }
+        first = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {**base_reg, "endpoint": "http://127.0.0.1:9201"},
+        )
+        second = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {**base_reg, "endpoint": "http://127.0.0.1:9202"},
+        )
+
+        coverage = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/network/map"
+        )
+        assert coverage["nodes"][0]["model_supply"]["served_model_copies"] == 2.0
+
+        hb = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/{first['node_id']}/heartbeat", {}
+        )
+        assert hb.get("directives", []) == []
+        assert second["node_id"] in tracker._registry
     finally:
         tracker.stop()
 
@@ -753,22 +1265,20 @@ def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
         _post_json(
             f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
             {"endpoint": "http://127.0.0.1:9001", "model": "tiny-model",
-             "vram_bytes": 20_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
+             "vram_bytes": 5_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
              "benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
         )
         _post_json(
             f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
             {"endpoint": "http://127.0.0.1:9002", "model": "tiny-model",
-             "vram_bytes": 20_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
+             "vram_bytes": 5_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
              "benchmark_tokens_per_sec": 2.0, "hardware_profile": {}, "score": 1.0},
         )
 
-        route_resp = _get_json(f"http://127.0.0.1:{tracker_port}/v1/route?model=tiny-model")
-        widths = {
-            node["endpoint"]: node["shard_end"] - node["shard_start"] + 1
-            for node in route_resp["nodes"]
-        }
-        assert widths["http://127.0.0.1:9002"] > widths["http://127.0.0.1:9001"]
+        net = _get_json(f"http://127.0.0.1:{tracker_port}/v1/network/map")
+        heads = [node for node in net["nodes"] if node["shard_start"] == 0]
+        assert len(heads) == 1
+        assert heads[0]["endpoint"] == "http://127.0.0.1:9002"
     finally:
         tracker.stop()
 
@@ -975,6 +1485,30 @@ def test_tracker_route_rejects_non_extending_overlap():
         tracker.stop()
 
 
+def test_tracker_console_records_model_not_available():
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    try:
+        try:
+            _post_json(
+                f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+                {"model": "missing-model", "messages": [{"role": "user", "content": "hi"}]},
+            )
+            raise AssertionError("Expected 503")
+        except urllib.error.HTTPError as exc:
+            assert exc.code == 503
+
+        console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
+    finally:
+        tracker.stop()
+
+    assert any(
+        event["message"] == "no nodes available for model"
+        and event["fields"]["model"] == "missing-model"
+        for event in console["events"]
+    )
+
+
 def test_tracker_registration_rejects_invalid_payload():
     """Registration errors return a defined JSON 400 response."""
     tracker = TrackerServer()
@@ -1812,6 +2346,7 @@ def test_torch_node_applies_tracker_load_shard_directive(monkeypatch):
 
     assert loaded == [("Qwen/Qwen2.5-0.5B-Instruct", 0, 23, "bfloat16")]
     assert applied == {
+        "action": "LOAD_SHARD",
         "model": "Qwen/Qwen2.5-0.5B-Instruct",
         "shard_start": 0,
         "shard_end": 23,
@@ -1928,3 +2463,105 @@ def test_shard_heal_cycle_surviving_node_covers_dead_peers_gap(monkeypatch):
         )
     finally:
         tracker.stop()
+
+
+def test_network_map_exposes_memory_pool():
+    tracker = TrackerServer(model_presets={
+        "tiny-model": {
+            "total_layers": 8,
+            "bytes_per_layer": {"bfloat16": 1_000},
+            "hf_repo": "org/TinyModel",
+        },
+    })
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9050", "model": "tiny-model",
+             "hf_repo": "org/TinyModel", "num_layers": 8,
+             "shard_start": 0, "shard_end": 7, "max_loaded_shards": 2,
+             "vram_bytes": 20_000, "ram_bytes": 20_000, "quantizations": ["bfloat16"],
+             "benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
+        )
+        net = _get_json(f"http://127.0.0.1:{tracker_port}/v1/network/map")
+        pool = net["memory_pool"]
+        assert pool["total_spare_slots"] == 1
+        assert pool["hosts"][0]["loaded_slots"] == 1
+        assert pool["hosts"][0]["max_loaded_shards"] == 2
+        assert pool["hosts"][0]["memory_spare_bytes"] > 0
+    finally:
+        tracker.stop()
+
+
+def test_same_endpoint_can_register_multiple_models():
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    try:
+        base = {
+            "endpoint": "http://127.0.0.1:9055",
+            "num_layers": 24,
+            "shard_start": 0,
+            "shard_end": 23,
+            "hardware_profile": {},
+            "score": 1.0,
+            "max_loaded_shards": 2,
+            "vram_bytes": 50_000_000,
+            "ram_bytes": 50_000_000,
+        }
+        first = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {**base, "model": "Qwen2.5-0.5B-Instruct", "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct"},
+        )
+        second = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {**base, "model": "OtherModel", "hf_repo": "org/OtherModel"},
+        )
+        assert first["node_id"] != second["node_id"]
+        assert len(tracker._registry) == 2
+    finally:
+        tracker.stop()
+
+
+def test_scale_demanded_models_queues_add_shard_on_spare_host():
+    tracker = TrackerServer(model_presets={
+        "model-a": {
+            "total_layers": 4,
+            "bytes_per_layer": {"bfloat16": 1_000},
+            "hf_repo": "org/ModelA",
+        },
+        "model-b": {
+            "total_layers": 4,
+            "bytes_per_layer": {"bfloat16": 1_000},
+            "hf_repo": "org/ModelB",
+        },
+    })
+    tracker_port = tracker.start()
+    try:
+        reg_b = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9061", "model": "model-b",
+             "hf_repo": "org/ModelB", "num_layers": 4,
+             "shard_start": 0, "shard_end": 3, "max_loaded_shards": 2,
+             "vram_bytes": 20_000, "ram_bytes": 20_000, "quantizations": ["bfloat16"],
+             "benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
+        )
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9060", "model": "model-a",
+             "hf_repo": "org/ModelA", "num_layers": 4,
+             "shard_start": 0, "shard_end": 3, "max_loaded_shards": 1,
+             "vram_bytes": 20_000, "ram_bytes": 20_000, "quantizations": ["bfloat16"],
+             "benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
+        )
+        assert tracker._stats is not None
+        for _ in range(400):
+            tracker._stats.record_request("org/ModelA")
+        with tracker._lock:
+            _scale_demanded_models_locked(tracker._server)  # type: ignore[arg-type]
+        node_b = tracker._registry[reg_b["node_id"]]
+        assignment = node_b.pending_new_assignment
+        assert assignment is not None
+        assert assignment["action"] == "ADD_SHARD"
+        assert assignment["model"] == "org/ModelA"
+    finally:
+        tracker.stop()