1.1 KiB
1.1 KiB
07 — Large-model support audit
Status: pending
Depends on: 01, 05
Goal
Determine which large target models can run through the distributed path and what upstream runtime work remains.
The first serious large-model target is deepseek-ai/DeepSeek-V4-Flash. GLM-5.2 and Ornith remain follow-up targets.
Scope
- Verify PyTorch/Transformers load semantics for DeepSeek-V4-Flash.
- Verify vLLM/SGLang serving support for DeepSeek-V4-Flash.
- Verify whether a GGUF/llama.cpp quantization path exists for DeepSeek-V4-Flash.
- Estimate artifact size and 128K KV/cache memory by layer range for DeepSeek-V4-Flash.
- Verify llama.cpp/GGUF support for
glm_moe_dsa. - Verify cache accounting for GLM-5.2 DSA/MLA.
- Verify Ornith/Qwen3.5-MoE hybrid attention support.
- Identify smallest viable quantization for quality-first use.
Acceptance
- Written compatibility matrix.
- Clear "supported now / upstream needed / not viable" status per model.
- DeepSeek-V4-Flash has a recommended first-runtime path: PyTorch, vLLM/SGLang, llama.cpp/GGUF, or blocked.
- Runtime blockers converted into issues or upstream collaboration notes.