docs: target GLM-5.2 Max for distributed alpha
This commit is contained in:
@@ -7,6 +7,8 @@ Last updated: 2026-07-13
|
||||
|
||||
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
|
||||
|
||||
The alpha-release target is the exact `zai-org/GLM-5.2` model, pinned by revision and served with `reasoning_effort=max`, using the smallest published Unsloth `UD-IQ1_S` GGUF across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama remains a cheap structural fixture; Qwen expansion is post-alpha.
|
||||
|
||||
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
|
||||
|
||||
1. Models larger than one node's RAM/VRAM.
|
||||
@@ -36,7 +38,7 @@ GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kerne
|
||||
|
||||
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
|
||||
|
||||
Before expensive native work, establish controlled lanes:
|
||||
Before expensive native work, establish controlled lanes. DGR-001 remains immutable; DGR-017 adds a target-specific fit and semantics contract without rewriting DGR-001 evidence:
|
||||
|
||||
- Same model architecture and upstream revision.
|
||||
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
|
||||
@@ -134,6 +136,10 @@ Use as source and test donors only:
|
||||
|
||||
Do not adopt or fork their repositories wholesale.
|
||||
|
||||
### Mesh-LLM GLM branch: focused test/patch donor only
|
||||
|
||||
Use its GLM-5.2 branch to study DSA, IndexShare, stage-local KV, and sideband tests. Do not import its scheduler, discovery/control plane, package manager, or broad llama.cpp patch stack. Every adopted idea must be independently understood, minimized, attributed, and tested against our exact pin.
|
||||
|
||||
## Battle-proven transport decision
|
||||
|
||||
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
|
||||
@@ -219,9 +225,9 @@ Two physical machines execute one model that uses both Shards. Synthetic activat
|
||||
|
||||
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
|
||||
|
||||
### Gate F: architecture expansion
|
||||
### Gate F: exact GLM-5.2 alpha target
|
||||
|
||||
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
|
||||
After the generic dense fixture proves range and boundary mechanics, certify explicit GLM-5.2 MoE, MLA KV, DSA, IndexShare, and NextN policy. Alpha requires the exact `UD-IQ1_S` target across physical consumer nodes, native Max-mode semantics, locked parity/usefulness/performance thresholds, and bounded failure cleanup. Qwen3/Qwen3-MoE is later architecture expansion.
|
||||
|
||||
## Scope discipline
|
||||
|
||||
|
||||
Reference in New Issue
Block a user