docs: target GLM-5.2 Max for distributed alpha

This commit is contained in:
Dobromir Popov
2026-07-13 22:32:14 +03:00
parent ef2a9e67e8
commit 5ebce15d7a
16 changed files with 866 additions and 65 deletions

View File

@@ -7,6 +7,8 @@ Last updated: 2026-07-13
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
The alpha-release target is the exact `zai-org/GLM-5.2` model, pinned by revision and served with `reasoning_effort=max`, using the smallest published Unsloth `UD-IQ1_S` GGUF across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama remains a cheap structural fixture; Qwen expansion is post-alpha.
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
1. Models larger than one node's RAM/VRAM.
@@ -36,7 +38,7 @@ GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kerne
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
Before expensive native work, establish controlled lanes:
Before expensive native work, establish controlled lanes. DGR-001 remains immutable; DGR-017 adds a target-specific fit and semantics contract without rewriting DGR-001 evidence:
- Same model architecture and upstream revision.
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
@@ -134,6 +136,10 @@ Use as source and test donors only:
Do not adopt or fork their repositories wholesale.
### Mesh-LLM GLM branch: focused test/patch donor only
Use its GLM-5.2 branch to study DSA, IndexShare, stage-local KV, and sideband tests. Do not import its scheduler, discovery/control plane, package manager, or broad llama.cpp patch stack. Every adopted idea must be independently understood, minimized, attributed, and tested against our exact pin.
## Battle-proven transport decision
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
@@ -219,9 +225,9 @@ Two physical machines execute one model that uses both Shards. Synthetic activat
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
### Gate F: architecture expansion
### Gate F: exact GLM-5.2 alpha target
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
After the generic dense fixture proves range and boundary mechanics, certify explicit GLM-5.2 MoE, MLA KV, DSA, IndexShare, and NextN policy. Alpha requires the exact `UD-IQ1_S` target across physical consumer nodes, native Max-mode semantics, locked parity/usefulness/performance thresholds, and bounded failure cleanup. Qwen3/Qwen3-MoE is later architecture expansion.
## Scope discipline