feat: compare safetensors and gguf on cpu and gpu
This commit is contained in:
@@ -11,9 +11,11 @@
|
||||
|
||||
- Locks the DGR-001 benchmark contract in code.
|
||||
- Pins the architecture-aligned baseline to **DeepSeek-V2-Lite-Chat** (`deepseek2`).
|
||||
- Uses the smallest DeepSeek-family GGUF target selected for this story: **Q2_K** via `second-state/DeepSeek-V2-Lite-Chat-GGUF`.
|
||||
- Uses the same model on both sides of the comparison:
|
||||
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
|
||||
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K**
|
||||
- Exposes a machine-readable JSON contract with:
|
||||
- benchmark lanes for `transformers` safetensors and `llama.cpp` GGUF
|
||||
- benchmark lanes for `transformers` safetensors and `llama.cpp` GGUF on **CPU** and **GPU**
|
||||
- concurrency levels `1` and `4`
|
||||
- the required metrics list
|
||||
- an explicit stop condition for “no meaningful speed or fit benefit”
|
||||
|
||||
Reference in New Issue
Block a user