30 lines
1.1 KiB
Markdown
30 lines
1.1 KiB
Markdown
# 07 — Large-model support audit
|
|
|
|
Status: pending
|
|
|
|
Depends on: 01, 05
|
|
|
|
## Goal
|
|
|
|
Determine which large target models can run through the distributed path and what upstream runtime work remains.
|
|
|
|
The first serious large-model target is `deepseek-ai/DeepSeek-V4-Flash`. GLM-5.2 and Ornith remain follow-up targets.
|
|
|
|
## Scope
|
|
|
|
- Verify PyTorch/Transformers load semantics for DeepSeek-V4-Flash.
|
|
- Verify vLLM/SGLang serving support for DeepSeek-V4-Flash.
|
|
- Verify whether a GGUF/llama.cpp quantization path exists for DeepSeek-V4-Flash.
|
|
- Estimate artifact size and 128K KV/cache memory by layer range for DeepSeek-V4-Flash.
|
|
- Verify llama.cpp/GGUF support for `glm_moe_dsa`.
|
|
- Verify cache accounting for GLM-5.2 DSA/MLA.
|
|
- Verify Ornith/Qwen3.5-MoE hybrid attention support.
|
|
- Identify smallest viable quantization for quality-first use.
|
|
|
|
## Acceptance
|
|
|
|
- Written compatibility matrix.
|
|
- Clear "supported now / upstream needed / not viable" status per model.
|
|
- DeepSeek-V4-Flash has a recommended first-runtime path: PyTorch, vLLM/SGLang, llama.cpp/GGUF, or blocked.
|
|
- Runtime blockers converted into issues or upstream collaboration notes.
|