feature-gguf-distributed

.scratch/distributed-gguf-runtime/issues/07-glm52-ornith-support-audit.md

@@ -0,0 +1,29 @@
+# 07 — Large-model support audit
+
+Status: pending
+
+Depends on: 01, 05
+
+## Goal
+
+Determine which large target models can run through the distributed path and what upstream runtime work remains.
+
+The first serious large-model target is `deepseek-ai/DeepSeek-V4-Flash`. GLM-5.2 and Ornith remain follow-up targets.
+
+## Scope
+
+- Verify PyTorch/Transformers load semantics for DeepSeek-V4-Flash.
+- Verify vLLM/SGLang serving support for DeepSeek-V4-Flash.
+- Verify whether a GGUF/llama.cpp quantization path exists for DeepSeek-V4-Flash.
+- Estimate artifact size and 128K KV/cache memory by layer range for DeepSeek-V4-Flash.
+- Verify llama.cpp/GGUF support for `glm_moe_dsa`.
+- Verify cache accounting for GLM-5.2 DSA/MLA.
+- Verify Ornith/Qwen3.5-MoE hybrid attention support.
+- Identify smallest viable quantization for quality-first use.
+
+## Acceptance
+
+- Written compatibility matrix.
+- Clear "supported now / upstream needed / not viable" status per model.
+- DeepSeek-V4-Flash has a recommended first-runtime path: PyTorch, vLLM/SGLang, llama.cpp/GGUF, or blocked.
+- Runtime blockers converted into issues or upstream collaboration notes.