feat: compare safetensors and gguf on cpu and gpu
This commit is contained in:
@@ -15,7 +15,12 @@ As a runtime engineer, I need a controlled baseline so that GGUF work proceeds f
|
||||
|
||||
## Baseline model target
|
||||
|
||||
Use the smallest *DeepSeek-family* GGUF that still points toward DeepSeek-V4-Flash. Current choice: **DeepSeek-V2-Lite GGUF Q2_K** (~6.5GB, `deepseek2` architecture). Reserve smaller non-DeepSeek fallback models only for loader plumbing smoke tests if needed; they do not count as the DGR-001 architecture-aligned baseline.
|
||||
Use the same model on both sides of the comparison, with the closest practical low-footprint precision pair:
|
||||
|
||||
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
|
||||
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K** (~6.5GB)
|
||||
|
||||
Keep the benchmark matrix explicit for **CPU** and **GPU** runs. Reserve smaller non-DeepSeek fallback models only for loader plumbing smoke tests if needed; they do not count as the DGR-001 architecture-aligned baseline.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user