feature-gguf-distributed

.scratch/distributed-gguf-runtime/issues/01-local-llamacpp-gguf-backend.md

@@ -0,0 +1,29 @@
+# 01 — Local llama.cpp/GGUF backend
+
+Status: ready-for-agent
+
+## Goal
+
+Add a local full-model llama.cpp/GGUF backend to the node so a machine that can hold a GGUF model can serve it through the existing OpenAI-compatible node API.
+
+## Scope
+
+- Add backend selection for `llama.cpp` / GGUF.
+- Support launching or calling `llama-server` first; direct `libllama` bindings may come later.
+- Register model metadata and hardware profile with tracker.
+- Preserve current PyTorch path.
+- Add a local benchmark comparing PyTorch CPU vs llama.cpp/GGUF for the same supported small model.
+
+## Non-Goals
+
+- No distributed GGUF route yet.
+- No partial layer loading yet.
+- No torrent seeding yet.
+
+## Acceptance
+
+- A local GGUF model can answer `/v1/chat/completions`.
+- Startup output clearly says backend=`llama.cpp`.
+- Node registration includes backend and artifact metadata.
+- Test or smoke script verifies the backend wiring without requiring a huge model.
+