From f102be10982b166d75923f729b5f48fe8400e2e6 Mon Sep 17 00:00:00 2001 From: Dobromir Popov Date: Tue, 14 Jul 2026 16:24:39 +0300 Subject: [PATCH] docs: retarget gguf epic to DeepSeek-V4-Flash --- docs/issues/42-gguf-llamacpp-node-backend.md | 21 +++++++++----------- 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/docs/issues/42-gguf-llamacpp-node-backend.md b/docs/issues/42-gguf-llamacpp-node-backend.md index dc26ac4..3a33c05 100644 --- a/docs/issues/42-gguf-llamacpp-node-backend.md +++ b/docs/issues/42-gguf-llamacpp-node-backend.md @@ -1,9 +1,16 @@ # US-042 — GGUF/llama.cpp node backend Status: planned -Priority: High (whole-model GGUF shortcut; distributed path in [ADR-0024](../adr/0024-distributed-gguf-runtime.md)) +Priority: High (unlocks DeepSeek-V4-Flash on volunteer hardware — the pool's core value) Stage: Draft design +## Goal + +Run **DeepSeek-V4-Flash** as the first real large-model target on volunteer +hardware via GGUF/llama.cpp. This epic is no longer GLM-oriented: the initial +objective is to prove that DeepSeek-V4-Flash can load and serve correctly on +consumer/unified-memory nodes, then expand from there. + ## Context The node backend is transformers-only (`model_backend.py` → @@ -35,17 +42,7 @@ to it (single-hop route). Smallest step, no cross-node activation work, and already useful: Strix Halo 128 GB serves DeepSeek-V4-Flash IQ3_XXS (114 GB) via llama.cpp Vulkan today. -Recommended sequencing: **C first** (US-042), then **ADR-0024 benchmark gate** (DGR-001), then distributed native worker (DGR-002+). Direction B (llama.cpp RPC) is rejected per ADR-0024. - -## Runtime sequencing - -| Stage | Track | Delivers | -|---|---|---| -| **C — Whole-model GGUF** | US-042 (this issue) | Single-hop llama.cpp, billing, relay streaming | -| **0 — Benchmark gate** | ADR-0024 DGR-001 | Safetensors vs GGUF measured contract | -| **1 — Distributed GGUF** | ADR-0024 `.scratch/distributed-gguf-runtime/` | gRPC C++ worker, layer-range GGUF | - -Phase C uses the existing tracker hop path (whole model, one node). ADR-0024 direction A (layer-range GGUF + activations) merges into the native worker track after the benchmark gate — not in parallel with phase C on the same backend without an integration plan. +Recommended sequencing: C first (small, real value), then A/B investigation. ## Also in scope