docs: target GLM-5.2 Max for distributed alpha
This commit is contained in:
@@ -6,6 +6,8 @@ Build one lean native GGUF execution path that lets an Inference Route combine c
|
||||
|
||||
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
|
||||
|
||||
The alpha target is now exact: `zai-org/GLM-5.2` at a pinned revision, the lowest published Unsloth `UD-IQ1_S` GGUF, and `reasoning_effort=max`, distributed across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a cheap structural fixture; Qwen is post-alpha expansion.
|
||||
|
||||
## Goals
|
||||
|
||||
- Execute one GGUF model across independently addressable contiguous Shards.
|
||||
@@ -15,6 +17,7 @@ The program is benchmark-gated. GGUF is not assumed faster merely because it is
|
||||
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
|
||||
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
|
||||
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
|
||||
- Pass an immutable GLM-5.2 Max `alpha`/`stop` gate with native MoE, DSA, IndexShare, parity, usefulness, speed, failure, and cleanup evidence.
|
||||
|
||||
## Quality Gates
|
||||
|
||||
@@ -213,6 +216,34 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||
|
||||
### DGR-017: Lock the GLM-5.2 Max target and alpha contract
|
||||
**Description:** Pin exact official/GGUF revisions, `UD-IQ1_S` files and hashes, Max-mode semantics, resource accounting, and immutable target thresholds before implementation results exist.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Produce machine-readable target, resource, upstream-status, and immutable acceptance contracts without downloading full weights.
|
||||
- [ ] Count unified RAM/VRAM once and test revision/shard/byte/threshold mutation failures.
|
||||
|
||||
### DGR-018: Certify whole-model GLM-5.2 runtime semantics
|
||||
**Description:** Establish a 224+ GiB usable-memory oracle for the exact target artifact before distributed parity work.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Verify the complete artifact and prove native MoE/shared expert, DSA, IndexShare, KV, NextN policy, and Max-template behavior.
|
||||
- [ ] Dense/replicated compatibility fallback cannot become the oracle merely because it emits text.
|
||||
|
||||
### DGR-019: Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||
**Description:** Add explicit target-model tensor, graph, boundary, sideband, and local-KV ownership after the generic dense seam.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Preserve MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared semantics across contiguous Shards.
|
||||
- [ ] Pass locked fixture/target parity and measured per-Shard memory ownership; never claim full-target parity from a reduced fixture.
|
||||
|
||||
### DGR-020: Pass real distributed GLM-5.2 Max alpha acceptance
|
||||
**Description:** Run the exact lowest-quant target through Meshnet on enough physical consumer nodes that no node can admit the whole recipe.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Pass the immutable identity, semantic, parity, Max-mode usefulness, 0.5 token/s, TTFT, reliability, and mounted-storage gates.
|
||||
- [ ] Preserve signed raw evidence and emit `alpha` only if every target criterion passes; otherwise emit `stop`.
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
|
||||
@@ -234,7 +265,7 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
- QUIC, WebRTC, or a custom socket protocol.
|
||||
- Automatic KV migration or mid-generation route repair in the first release.
|
||||
- Speculative decoding or disaggregated prefill before the core release gate.
|
||||
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
|
||||
- Supporting every GGUF architecture before the exact GLM-5.2 target; Qwen3-family certification is post-alpha.
|
||||
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
|
||||
|
||||
## Success Metrics
|
||||
@@ -247,6 +278,6 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
|
||||
- DGR-001 remains immutable. DGR-017 locks the exact GLM-5.2/`UD-IQ1_S` target and alpha thresholds without rewriting DGR-001 evidence.
|
||||
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
|
||||
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
|
||||
|
||||
Reference in New Issue
Block a user