new tasks, model pricing, auto quantisation, etc...

docs/issues/42-gguf-llamacpp-node-backend.md

@@ -0,0 +1,55 @@
+# US-042 — GGUF/llama.cpp node backend
+
+Status: planned
+Priority: High (unlocks big MoE models on volunteer hardware — the pool's core value)
+Stage: Draft design
+
+## Context
+
+The node backend is transformers-only (`model_backend.py` →
+`AutoModelForCausalLM`). For DeepSeek-V4-Flash (158B MoE, official weights FP8
+160 GB) the only quantizations that run on consumer hardware are GGUF
+(IQ2 87 GB → Q4_K_M-XL 175 GB) — llama.cpp format. The transformers-compatible
+quants (FP8, NVFP4, GPTQ W4A16) all need datacenter GPUs. Volunteer machines —
+including our own Strix Halo boxes (128 GB and 80 GB unified memory, GPU via
+Vulkan/ROCm, no FP8 support on RDNA3.5) — run these models today only under
+llama.cpp.
+
+## Design directions to evaluate (design-it-twice)
+
+**A. llama.cpp as a per-node shard executor.** Node loads a *layer range* of a
+GGUF via llama-cpp-python; our existing hop protocol (X-Meshnet-Route,
+activations over HTTP/relay) moves hidden states between nodes. Requires
+llama.cpp partial-layer loading and activation import/export — investigate
+feasibility first; this is the riskiest unknown.
+
+**B. llama.cpp RPC mode under tracker orchestration.** llama.cpp ships a
+native RPC backend that splits one model across machines. The tracker would
+provision/route to an llama.cpp RPC cluster rather than our own hop pipeline.
+Less code, but bypasses our billing/telemetry hop instrumentation and relay
+NAT path — needs a story for both.
+
+**C. Whole-model GGUF nodes (no sharding).** A node with enough memory serves
+a full GGUF (e.g. IQ2/IQ3 on a 128 GB box); the tracker routes whole requests
+to it (single-hop route). Smallest step, no cross-node activation work, and
+already useful: Strix Halo 128 GB serves DeepSeek-V4-Flash IQ3_XXS (114 GB)
+via llama.cpp Vulkan today.
+
+Recommended sequencing: C first (small, real value), then A/B investigation.
+
+## Also in scope
+
+- Model catalog: allow GGUF entries with quant selection; feature
+  `DeepSeek-V4-Flash` IQ4_XS/UD-Q4_K_XL as a curated/featured entry once at
+  least direction C works (a featured model nobody can load is an anti-feature)
+- Hardware detection: recognize Strix Halo/unified-memory APUs and Vulkan
+  (`hardware.py` currently reports "CPU mode" on these boxes)
+- `MESHNET_DOWNLOAD_DIR`/`--download-dir` applies to GGUF files as well
+
+## Acceptance criteria (phase C)
+
+- A node with `--gguf <repo-or-path> --quant IQ3_XXS` serves
+  `/v1/chat/completions` via llama.cpp with GPU offload where available
+- Tracker treats it as a full-coverage node (single-hop routes, billing works)
+- Streamed responses work through the tracker proxy and the relay (US-036)
+- `python -m pytest` passes from repo root (llama.cpp behind an optional extra)

docs/issues/43-dashboard-model-search-cards.md

@@ -0,0 +1,38 @@
+# US-043 — Dashboard model search and model cards
+
+Status: planned
+Priority: Medium (post-deploy polish)
+Stage: Idea
+
+## Context
+
+The dashboard shows nodes/routes/billing but nothing model-centric. Operators
+and testers should be able to search for a model and see, per model, a card
+with what the network knows about it.
+
+## Scope
+
+- **Search**: query box hitting a new tracker endpoint that proxies the HF Hub
+  search API (server-side, so the dashboard stays CSP-clean and unauthenticated
+  browsers aren't rate-limited) merged with the tracker's own model presets and
+  currently-served models.
+- **Model card** per result:
+  - name, architecture, params, layer count (reuse `model_metadata_for`,
+    which now handles nested `text_config` — US layer-detection fix)
+  - coverage on the network: which layer ranges are served, by how many nodes,
+    coverage gaps (the Coverage Map already exists on the tracker)
+  - price per 1K tokens, availability (routable now? single-hop or pipeline?)
+  - memory footprint per quantization where known (bf16 / GGUF sizes)
+  - action: "request this model" — flags demand so node operators (or
+    auto-shard assignment) know what to load next
+- Featured models section driven by the curated catalog (`CURATED_MODELS`),
+  including GGUF entries once US-042 lands.
+
+## Acceptance criteria
+
+- Searching a HF repo id or free text returns results without the browser
+  calling HF directly
+- A served model's card shows live coverage and a working "chat now" state
+- An unserved model's card shows the "request" action and estimated memory
+  per quant
+- `python -m pytest` passes from repo root

docs/issues/44-tracker-shard-source-partial-download.md

@@ -0,0 +1,65 @@
+# US-044 — Tracker as model-file source; nodes download only their shard
+
+Status: planned
+Priority: High (blocks multi-machine big-model serving; pairs with US-042)
+Stage: Designed (grill remaining decisions before build)
+
+## Context
+
+Common deployment: the tracker and the first node share a machine that already
+holds the model files (e.g. `/run/media/popov/DATA/llm/safetensor`). When a
+second node joins with no model selected, the tracker assigns it the uncovered
+layer range — and today that node then downloads the **entire snapshot from
+HuggingFace**, even for a 20-layer shard of a 160 GB model.
+
+What exists already (build on it, don't duplicate):
+
+- Nodes serve their shard dir as a tar at `GET /v1/shards/download` with
+  checksum verification; `download_shard` tries assignment-provided `peers`
+  before HF (`downloader.py`). But it only matches **identical layer ranges**,
+  and the HF fallback runs `snapshot_download` of the whole repo.
+- The torch path (`--model-id`) bypasses `download_shard` entirely:
+  `TorchModelShard` → `from_pretrained` downloads **and loads into RAM** the
+  full model, then executes only the assigned layers. Sharding currently saves
+  compute, not memory or bandwidth.
+
+## Design
+
+1. **Tracker `--models-dir PATH`** (env `MESHNET_MODELS_DIR`). When set, the
+   tracker indexes HF-layout snapshots under it and advertises itself as a
+   model-file source in `/v1/nodes/assign` responses.
+2. **Layer-aware file selection.** For safetensors models, read
+   `model.safetensors.index.json` and map the assigned layer range → the
+   subset of weight files containing those layers, plus the always-needed
+   files (config, tokenizer, index, embeddings/head files for head/tail
+   shards). Serve exactly that subset (tar stream, per-file checksums).
+   GGUF (US-042): single file or naive byte-range — phase 2.
+3. **Node download order**: exact-shard peer (existing) → tracker/peer file
+   subset (new) → HF `snapshot_download` with `allow_patterns` for the same
+   subset (new — stop downloading the whole repo even from HF) → full snapshot
+   (last resort).
+4. **Partial LOAD (the hard half).** Downloading a subset is wasted unless the
+   node stops instantiating the full model: build the model skeleton on the
+   `meta` device, materialize only assigned layers (+embeddings/norm/head as
+   role requires) from the local files, leave the rest on meta. Without this,
+   an 80 GB machine can never hold a shard of a 160 GB model regardless of
+   how the bytes arrive. This is the acceptance bar for the issue.
+
+## Open questions (grill before building)
+
+- Trust: joining nodes fetch weights from the tracker/peers — checksum against
+  what root of trust? (HF etag/sha vs tracker-signed manifest.)
+- Disk layout: partial snapshots must not corrupt the HF cache dir; probably
+  a meshnet-owned layout keyed by repo+revision.
+- Serving cost: a 100 GB tar stream per joining node on the tracker box —
+  rate-limit/queue? LAN-only heuristic?
+
+## Acceptance criteria
+
+- Two-machine test: machine A (tracker + node, holds full snapshot) serves
+  layers 0–k; machine B joins with no model and receives **only** the files
+  for its assigned range from A — nothing fetched from HF
+- Machine B's resident memory scales with its shard size, not model size
+- Checksums verified end-to-end; corrupted transfer falls back cleanly
+- Single-node/full-model flows unchanged
+- `python -m pytest` passes from repo root