docs: target GLM-5.2 Max for distributed alpha

This commit is contained in:
Dobromir Popov
2026-07-13 22:32:14 +03:00
parent ef2a9e67e8
commit 5ebce15d7a
16 changed files with 866 additions and 65 deletions

View File

@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
1. Read this file completely.
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
3. Read `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`, and the relevant part of `architecture.md`.
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
6. Read the evidence/handoff directories for every declared dependency.
@@ -35,6 +35,8 @@ If interrupted after code changes, record every changed file, command result and
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact served with `reasoning_effort=max` across physical consumer machines. Dense Llama is a structural fixture. Synthetic workers, dense-attention compatibility fallback, a smaller model, or a single host cannot satisfy target alpha. The immutable target contract and resource envelope are in `GLM-5.2-MAX-ALPHA-ROADMAP.md`.
A distributed demo is not success. The product must provide:
- Useful measured prefill and decode speed.
@@ -84,6 +86,8 @@ Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
16. Unified system RAM and integrated-GPU memory are one physical pool and must never be double-counted for admission.
17. Alpha requires native GLM-5.2 MoE, DSA, and IndexShare semantics; MTP/speculative decoding and 1M-context certification are post-alpha.
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
@@ -154,7 +158,7 @@ The project patch stack is limited to:
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
Dense Llama is first only as a cheap range/boundary fixture. GLM-5.2 is the explicit product adapter and alpha target immediately afterward. Qwen3/Qwen3-MoE is post-alpha. Do not generalize through unchecked tensor-name substitutions.
## Existing code seams to inspect first
@@ -222,6 +226,10 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
| DGR-017 | exact GLM-5.2/GGUF target manifest, resource planner, immutable alpha contract and upstream status; `evidence/DGR-017/` |
| DGR-018 | verified whole-model `UD-IQ1_S` oracle with native GLM semantic evidence; `evidence/DGR-018/` |
| DGR-019 | explicit range-owned GLM MoE/MLA/DSA/IndexShare adapter, fixtures and parity; `evidence/DGR-019/` |
| DGR-020 | real multi-node GLM-5.2 Max target evidence and immutable `alpha`/`stop` verdict; `evidence/DGR-020/` |
## Dependency handoff rule