DGR-001 — performance contract baseline

Files changed

packages/node/meshnet_node/performance_contract.py
tests/test_performance_contract.py
.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md
.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json

What this slice does

Locks the DGR-001 benchmark contract in code.
Pins the architecture-aligned baseline to DeepSeek-V2-Lite-Chat (deepseek2).
Uses the smallest DeepSeek-family GGUF target selected for this story: Q2_K via second-state/DeepSeek-V2-Lite-Chat-GGUF.
Exposes a machine-readable JSON contract with:
- benchmark lanes for transformers safetensors and llama.cpp GGUF
- concurrency levels 1 and 4
- the required metrics list
- an explicit stop condition for “no meaningful speed or fit benefit”

Exact commands and real results

Targeted tests

pytest -q tests/test_performance_contract.py tests/test_route_session_benchmark.py

Result: 9 passed in 0.14s

Contract artifact generation

PYTHONPATH=packages/node python -m meshnet_node.performance_contract --json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json

Result: wrote .scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json

Python compile check

python -m compileall packages/node/meshnet_node/performance_contract.py tests/test_performance_contract.py

Result: passed

Limitations

This slice captures the DGR-001 contract and baseline selection only.
It does not download or run a real model yet.
Real safetensors vs GGUF execution, TTFT/prefill/decode measurements, RSS/VRAM capture, and output-drift comparison are still to be implemented against the contract.

Compatibility notes

The contract stays on the DeepSeek2 family to remain close to the DeepSeek-V4-Flash end goal.
A smaller non-DeepSeek model can still be used later for loader-plumbing smoke tests, but it does not replace this baseline.
Model artifacts must stay on the mounted drive and not under /home.

Dependent-story handoff

Next implementation work should attach to this contract and add the live benchmark runner that actually compares:

current Transformers/safetensors recipe
whole-model llama.cpp GGUF recipe

using the same model architecture/revision and the same prompt/context/concurrency settings.