popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: target GLM-5.2 Max for distributed alpha

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-13 22:32:14 +03:00
parent ef2a9e67e8
commit 5ebce15d7a
16 changed files with 866 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md
View File

@@ -39,7 +39,7 @@ As the Tracker, I need exact compatibility identity so that only numerically and
## Dependency handoff
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
- `DGR-002` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md
View File

@@ -43,7 +43,7 @@ As a maintainer, I need a small auditable fork boundary so that upstream updates
## Dependency handoff
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
- `DGR-001` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md
View File

@@ -42,7 +42,7 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
## Dependency handoff
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
- `DGR-006` and `DGR-019` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md
View File

@@ -44,7 +44,7 @@ As a client, I need failures to be bounded and explicit so that distributed spee
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
- DGR-012 continuous batching is post-alpha; this story must harden single-session failure/cancellation first and later extend the same invariants to batched sessions.
## Finish contract

60
.scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md Normal file
View File

@@ -0,0 +1,60 @@
# 17 — Lock the GLM-5.2 Max target and alpha contract
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-017` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code, upstream sources, and `git status`; historical text and previous agent claims are not evidence.
## Description
As a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.
## Expected durable outputs
- Machine-readable official/GGUF target manifest with revisions, filenames, sizes, and resolved hashes
- Architecture/config/chat-template snapshot and source links
- Deterministic memory/KV/network planner with unified-memory de-duplication
- Current llama.cpp and donor-support status report
- Immutable machine-readable alpha acceptance contract
- `evidence/DGR-017/README.md`
## Acceptance criteria
- [ ] Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.
- [ ] Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.
- [ ] Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.
- [ ] Generate deterministic minimum-node calculations from exact artifact bytes, configurable per-node reserve, KV dtype/context, scratch reserve, and endpoint/tensor imbalance.
- [ ] Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.
- [ ] Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.
- [ ] Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.
- [ ] Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.
- [ ] Targeted pytest tests pass.
- [ ] `python -m compileall packages tests` passes for Python changes.
- [ ] `git diff --check` passes.
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
- [ ] Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.
- [ ] Read and verify every dependency evidence README before relying on dependency behavior.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes.
## Dependency handoff
- `DGR-001` and `DGR-002` must have `passes: true`; read their evidence READMEs and verify referenced files/commands.
## Finish contract
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate source, benchmark, model, test, or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Ralph execution context](../RALPH-CONTEXT.md)
- [Current architecture](../architecture.md)

57
.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md Normal file
View File

@@ -0,0 +1,57 @@
# 18 — Certify whole-model GLM-5.2 runtime semantics
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-018` in [prd.json](../prd.json).
- Read and verify every dependency evidence README.
- Inspect current upstream behavior and `git status`; community claims that a model “works” are not correctness evidence.
## Description
As a runtime maintainer, I need a certified whole-model oracle for the exact lowest-quant GLM-5.2 artifact so that distributed parity is measured against correct MoE, DSA, IndexShare, cache, and Max-template semantics.
## Expected durable outputs
- Verified target artifact on mounted storage
- Stock-pin baseline and warning/tensor inventory
- Focused GLM runtime correctness tests and minimum patch series, if required
- Signed whole-model oracle recipe, deterministic outputs, metrics, and limitations
- `evidence/DGR-018/README.md`
## Acceptance criteria
- [ ] Preflight at least 224 GiB usable memory and approximately 250 GB free mounted storage; abort before download if storage would resolve under `/home` or requirements are unmet.
- [ ] Download/resume and verify all six DGR-017 `UD-IQ1_S` files against exact sizes and LFS SHA-256 values.
- [ ] Load the target with the unmodified DGR-004 pin first and retain full logs, tensor warnings, peak memory, context/KV allocation, TTFT, and decode timing.
- [ ] Prove from graph/runtime evidence whether 256-expert MoE/top-8/shared expert, DSA lightning indexer/sparse attention, IndexShare Full/Shared reuse, and `reasoning_effort=max` are active.
- [ ] Dense-attention or replicated-indexer compatibility fallback is labeled incomplete and cannot become the oracle solely because it emits plausible text.
- [ ] Audit focused upstream/Mesh-LLM donor tests and patches; adopt or rewrite only independently understood minimum changes and record provenance/rejection rationale.
- [ ] Handle the trailing NextN/MTP tensors explicitly; MTP may remain disabled for alpha, but ignored tensors and layer-count behavior must be understood and tested.
- [ ] Run deterministic prefill/greedy decode and fixed Max-mode coding, structured-output/tool-call, and reasoning sentinels; retain raw prompts, outputs, token IDs, and compared state/logit evidence.
- [ ] Bind the oracle to exact artifact, tokenizer/template, adapter, backend class, compute/activation/KV dtypes, context/RoPE, llama.cpp commit, and patch-series hash.
- [ ] Produce a signed `pass` or `stop` semantic-runtime verdict; no performance threshold is weakened after execution.
- [ ] Targeted native/unit tests and clean pinned rebuild pass.
- [ ] `git diff --check` passes.
- [ ] Default tests remain model-download-free/GPU-free; full-target execution is opt-in and never runs in default CI.
- [ ] Model artifacts and caches remain on configured mounted-drive storage and never under `/home`.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-018/README.md` with exact hardware, storage, source revisions, files, commands, raw-result paths, real results, limitations, and handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
## Dependency handoff
- `DGR-003`, `DGR-004`, and `DGR-017` must have `passes: true`; read and verify their evidence READMEs.
## Finish contract
- If 224+ GiB usable reference hardware is unavailable, write `evidence/DGR-018/BLOCKED.md` with exact preflight output; do not substitute a smaller model.
- Preserve real failures and blockers; never fabricate output.
- Emit `<promise>COMPLETE</promise>` only after the signed oracle evidence exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [llama.cpp GLM-5.2 issue](https://github.com/ggml-org/llama.cpp/issues/24730)

58
.scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md Normal file
View File

@@ -0,0 +1,58 @@
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-019` in [prd.json](../prd.json).
- Read and verify every dependency evidence README.
- Inspect current source and `git status`; do not generalize through tensor-name substitution.
## Description
As a target-model operator, I need explicit range-owned GLM-5.2 semantics so that contiguous consumer Shards preserve the whole-model MoE, DSA, IndexShare, and local-KV computation.
## Expected durable outputs
- Explicit GLM-5.2 architecture adapter and tensor-ownership rules
- IndexShare-aware Shard planner and named DSA sideband protocol mapping
- GLM architecture fixture and target parity evidence
- Memory/tensor/KV ownership report
- `evidence/DGR-019/README.md`
## Acceptance criteria
- [ ] Implement explicit ownership for 78 main layers, head embedding, tail norm/output head, routed/shared experts, DSA/indexer tensors, and the certified NextN/MTP policy.
- [ ] Each contiguous layer owner keeps all 256 experts and the shared-expert path for its layers local; no public cross-machine expert collectives are introduced.
- [ ] Define and fixture-test compressed MLA KV ownership for locally owned layers, keyed by Route Session/epoch, for DGR-007 to implement.
- [ ] Implement native DSA lightning indexer/top-2,048 and sparse-attention graph behavior matching DGR-018.
- [ ] Parse and validate artifact `indexer_types`; implement Full producer and Shared consumer behavior without fabricated/duplicated indexer tensors.
- [ ] Prefer Shard boundaries that preserve IndexShare ownership groups; when memory fit forces a split, carry a typed, bounded, validated top-k sideband in the DGR-006 named bundle.
- [ ] Reject missing, stale, wrong-width, wrong-position, or Shared-before-Full sideband/index state.
- [ ] Demonstrate mapped/resident weights and allocated KV scale with exact owned tensors/layers rather than the full model.
- [ ] Same-host two-stage F32 seam fixture produces 32 exact greedy tokens against the DGR-018 semantics; production seam meets the pre-locked token/similarity thresholds.
- [ ] If full-target same-host execution cannot fit, use a layer-reduced GLM architecture fixture only for graph parity and defer full-artifact parity to DGR-020; label it non-target evidence.
- [ ] Add deterministic fixture tests for MoE route/shared expert, DSA Full/Shared groups, internal group split, endpoint ownership, KV filtering, malformed metadata, and NextN policy.
- [ ] Targeted pytest/CTest/native tests pass; pinned patch stack applies and rebuilds cleanly.
- [ ] `python -m compileall packages tests` and `git diff --check` pass.
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
- [ ] Full deterministic `pytest -q` passes, or exact pre-existing unrelated failures are recorded.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-019/README.md` with files, commands, real results, raw evidence, limitations, and dependent-story handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
## Dependency handoff
- `DGR-005`, `DGR-006`, and `DGR-018` must have `passes: true`; read and verify their evidence READMEs.
- DGR-007 integration may be completed in parallel only after the adapter's cache contract is explicit.
## Finish contract
- Preserve real failures and blockers; never claim target parity from the architecture fixture.
- Emit `<promise>COMPLETE</promise>` only after evidence exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Current architecture](../architecture.md)

60
.scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md Normal file
View File

@@ -0,0 +1,60 @@
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-020` in [prd.json](../prd.json).
- Read and verify every dependency evidence README and the immutable DGR-017 contract.
- Inspect live source, hardware, network, storage, and `git status`; synthetic or layer-reduced evidence cannot satisfy this story.
## Description
As an alpha user, I need the exact lowest-quant GLM-5.2 model to run in Max reasoning mode across consumer machines through Meshnet so that the project has proven its end goal on real hardware.
## Expected durable outputs
- Exact multi-node hardware/network/storage/runtime manifest
- Signed target identity, ownership, parity, quality, performance, and reliability reports
- Raw worker/tracker/API logs, metrics, prompts, outputs, and cleanup evidence
- Explicit immutable `alpha` or `stop` verdict
- `evidence/DGR-020/README.md`
## Acceptance criteria
- [ ] Use the exact DGR-017 `UD-IQ1_S` artifact and certified DGR-018/DGR-019 runtime recipe with all source and patch hashes verified.
- [ ] Use at least two physical consumer machines with at least 224 GiB aggregate usable admitted memory; no participating node may individually admit the complete recipe and unified memory is counted once.
- [ ] Tracker selects disjoint contiguous Shards whose exact required tensor inventory has complete union and no unintended overlap.
- [ ] Every stage reports real CPU/GPU compute, owned tensor bytes/layers, local KV, backend, queue, and seam telemetry; synthetic, passthrough, or zero-layer workers fail acceptance.
- [ ] Runtime evidence proves native GLM MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared paths are active; dense fallback fails acceptance.
- [ ] OpenAI-compatible API applies and records `reasoning_effort=max`, stable model ID, finish reason, and token usage.
- [ ] Configure 16,384 context; complete the fixed 4,096-token prefill lane and generate at least 512 output tokens or valid natural EOS after at least 128 tokens.
- [ ] Production seam meets at least 0.90 greedy token agreement and 0.999 mean compared-state/logit cosine similarity against DGR-018 on the fixed corpus, with no non-finite or malformed tensors.
- [ ] Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable relevant outputs retained for human review.
- [ ] After warm-up, fixed Max lane median decode is at least 0.5 token/s, 4,096-token-prompt TTFT is at most 10 minutes, and no unexplained stall exceeds 60 seconds without progress telemetry.
- [ ] Record per-stage/seam p50/p95 latency, bytes, compute time, peak RSS/VRAM, KV pressure, network properties, errors, and total energy when available.
- [ ] Two consecutive cold starts load, generate, release, and exit without leaked processes, mapped weights, queues, or KV leases.
- [ ] Cancellation in prefill and decode releases all stages; one worker loss aborts the route; retry starts from token zero on a newly compatible route; stale epochs and duplicate steps fail closed.
- [ ] Model artifacts/caches remain on configured mounted-drive storage and never under `/home`; secret scan passes.
- [ ] Run targeted/full deterministic tests and clean native rebuild in addition to target execution; `git diff --check` passes.
- [ ] Preserve all raw failures and do not weaken DGR-017 thresholds after results.
- [ ] Write a signed `alpha` verdict only if every criterion passes; otherwise write `stop` with the measured bottleneck and next narrow action.
- [ ] Write `evidence/DGR-020/README.md` with exact commands, manifests, raw-result paths, outputs, limitations, and post-alpha handoff.
- [ ] Update only this story issue to `Status: done` after the immutable verdict and all evidence exist.
## Dependency handoff
- `DGR-007`, `DGR-008`, `DGR-009`, `DGR-011`, `DGR-013`, `DGR-017`, `DGR-018`, and `DGR-019` must have `passes: true`; read and verify their evidence READMEs.
- DGR-012 continuous batching and DGR-014 final comparative release gate are post-alpha and do not block one target session.
## Finish contract
- If required physical nodes/storage are unavailable, write `evidence/DGR-020/BLOCKED.md`; do not substitute a smaller model, API, synthetic worker, or single host.
- Preserve real failures and blockers; never fabricate target output or telemetry.
- Emit `<promise>COMPLETE</promise>` only after the signed verdict exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Ralph execution context](../RALPH-CONTEXT.md)