chore: triage maintenance review and close completed stories
This commit is contained in:
Dobromir Popov
@@ -44,6 +44,102 @@ Real-model/hardware stories must:
|
||||
|
||||
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
||||
|
||||
## Dependency Graph and Status
|
||||
|
||||
Status as of 2026-07-14 (MAINT-003). Authoritative per-story status is
|
||||
`passes` in [prd.json](prd.json); closed issues live in
|
||||
`docs/issues/distributed-gguf-runtime/`, open and blocked issues in
|
||||
[issues/](issues/).
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef done fill:#c8e6c9,stroke:#2e7d32;
|
||||
classDef blocked fill:#ffcdd2,stroke:#c62828;
|
||||
|
||||
DGR001[DGR-001 perf contract]:::done
|
||||
DGR002[DGR-002 gRPC Shard protocol]:::done
|
||||
DGR003[DGR-003 artifact/recipe identity]:::done
|
||||
DGR004[DGR-004 pinned llama.cpp patch stack]:::done
|
||||
DGR005[DGR-005 dense-Llama range ownership]:::done
|
||||
DGR006[DGR-006 boundary input/output]:::done
|
||||
DGR017[DGR-017 GLM-5.2 target/alpha contract]:::done
|
||||
DGR018[DGR-018 whole-model GLM oracle]:::blocked
|
||||
DGR019[DGR-019 GLM range/DSA/IndexShare]:::blocked
|
||||
DGR020[DGR-020 distributed GLM alpha]:::blocked
|
||||
DGR007[DGR-007 Hot KV State]
|
||||
DGR008[DGR-008 C++ gRPC worker]
|
||||
DGR009[DGR-009 Meshnet integration]
|
||||
DGR010[DGR-010 local two-process acceptance]
|
||||
DGR011[DGR-011 two-machine route]
|
||||
DGR012[DGR-012 continuous batching]
|
||||
DGR013[DGR-013 failure/cancel/restart]
|
||||
DGR014[DGR-014 release gate]
|
||||
DGR015[DGR-015 Qwen3 adapter]
|
||||
DGR016[DGR-016 upstream package]
|
||||
|
||||
DGR002 --> DGR003
|
||||
DGR017 --> DGR003
|
||||
DGR001 --> DGR004
|
||||
DGR017 --> DGR004
|
||||
DGR003 --> DGR005
|
||||
DGR004 --> DGR005
|
||||
DGR002 --> DGR006
|
||||
DGR005 --> DGR006
|
||||
DGR001 --> DGR017
|
||||
DGR002 --> DGR017
|
||||
DGR003 --> DGR018
|
||||
DGR004 --> DGR018
|
||||
DGR017 --> DGR018
|
||||
DGR005 --> DGR019
|
||||
DGR006 --> DGR019
|
||||
DGR018 --> DGR019
|
||||
DGR006 --> DGR007
|
||||
DGR019 --> DGR007
|
||||
DGR002 --> DGR008
|
||||
DGR003 --> DGR008
|
||||
DGR004 --> DGR008
|
||||
DGR006 --> DGR008
|
||||
DGR007 --> DGR008
|
||||
DGR003 --> DGR009
|
||||
DGR008 --> DGR009
|
||||
DGR009 --> DGR010
|
||||
DGR010 --> DGR011
|
||||
DGR007 --> DGR012
|
||||
DGR009 --> DGR012
|
||||
DGR010 --> DGR012
|
||||
DGR008 --> DGR013
|
||||
DGR009 --> DGR013
|
||||
DGR001 --> DGR014
|
||||
DGR011 --> DGR014
|
||||
DGR012 --> DGR014
|
||||
DGR013 --> DGR014
|
||||
DGR014 --> DGR015
|
||||
DGR010 --> DGR016
|
||||
DGR007 --> DGR020
|
||||
DGR008 --> DGR020
|
||||
DGR009 --> DGR020
|
||||
DGR011 --> DGR020
|
||||
DGR013 --> DGR020
|
||||
DGR017 --> DGR020
|
||||
DGR018 --> DGR020
|
||||
DGR019 --> DGR020
|
||||
```
|
||||
|
||||
- **Done (`passes: true`):** DGR-001, DGR-002, DGR-003, DGR-004, DGR-005,
|
||||
DGR-006, DGR-017.
|
||||
- **Blocked on hardware:** DGR-018 requires a 256-GiB-class host with at least
|
||||
224 GiB runtime-accessible memory and 250 GB free storage outside `/home`;
|
||||
no such host is currently available (development host: 124.9 GiB MemTotal).
|
||||
Exact preflight output: [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md).
|
||||
DGR-019 (needs the DGR-018 oracle) and DGR-020 (needs DGR-018/DGR-019 plus
|
||||
multiple physical consumer nodes) are blocked transitively.
|
||||
- **Consequence of the graph as written:** DGR-007 depends on DGR-019, so every
|
||||
remaining story (DGR-007 through DGR-016) is transitively blocked on the
|
||||
256-GiB host. Unblocking the generic dense pipeline without that host would
|
||||
require an explicit re-planning decision to relax the DGR-007 → DGR-019
|
||||
dependency; that decision is out of scope for maintenance and has not been
|
||||
made.
|
||||
|
||||
## User Stories
|
||||
|
||||
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# DGR-018 — BLOCKED: no 256-GiB-class oracle host
|
||||
|
||||
Recorded: 2026-07-14 (MAINT-003). Preflight scripts preserved at commit
|
||||
`a0f28b5` ("chore: preserve DGR-018 preflight scripts (postponed)").
|
||||
|
||||
## Blocker
|
||||
|
||||
DGR-018 requires a 256-GiB-class host with at least **224 GiB
|
||||
runtime-accessible memory** (the DGR-017 experimental hard-fit floor for the
|
||||
whole-model `UD-IQ1_S` oracle) and **250 GB free storage** on one filesystem
|
||||
outside `/home` (216.715 GB artifact plus resume/temp headroom). The available
|
||||
development host fails both gates, so the whole-model oracle cannot be
|
||||
established. Per the issue's finish contract, no smaller model may be
|
||||
substituted.
|
||||
|
||||
DGR-019 (needs the DGR-018 oracle for parity certification) and DGR-020
|
||||
(needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single
|
||||
node admits the whole recipe) are blocked transitively.
|
||||
|
||||
## Exact preflight output
|
||||
|
||||
Command (offline; resolves everything from the pinned target manifest and
|
||||
never contacts the network):
|
||||
|
||||
```
|
||||
$ python scripts/glm_whole_model_preflight.py
|
||||
target: UD-IQ1_S 216.715 GB, 6 shards @ abc55e725277
|
||||
[FAIL] storage: need >= 250 GB free on one filesystem outside ['/home']; observed no eligible filesystem
|
||||
[FAIL] memory: need >= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor); observed 124.9 GiB MemTotal
|
||||
destination: NONE — no filesystem outside ['/home'] has 250 GB free
|
||||
- /run/media/popov/DATA (ext4): 74.2 GB free
|
||||
- / (ext4): 51.1 GB free
|
||||
- /run/media/popov/Windows (fuseblk): 26.0 GB free
|
||||
- /run/media/popov/d (fuseblk): 5.1 GB free
|
||||
verdict: fail
|
||||
$ echo $?
|
||||
1
|
||||
```
|
||||
|
||||
Host: Linux 7.0.14-101.fc43.x86_64 x86_64, `MemTotal: 130997376 kB`
|
||||
(124.9 GiB). The full machine-readable report (including the ordered
|
||||
download/verify plan against revision `abc55e72527792c6e77069c99b4cb7de16fa9f23`,
|
||||
manifest SHA-256 `0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`)
|
||||
is in [preflight.json](preflight.json).
|
||||
|
||||
## How to resume
|
||||
|
||||
1. On a qualifying host, run `python scripts/glm_whole_model_preflight.py`
|
||||
(optionally `--dest DIR`); it must exit 0 with `verdict: pass`.
|
||||
2. Download shards in the preflight's ordered plan; verify each with
|
||||
`python scripts/verify_glm_shards.py` before the next transfer starts.
|
||||
3. Proceed with the DGR-018 issue
|
||||
(`.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md`).
|
||||
@@ -0,0 +1,140 @@
|
||||
{
|
||||
"generated_by": "scripts/glm_whole_model_preflight.py",
|
||||
"target": {
|
||||
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"quantization": "UD-IQ1_S",
|
||||
"shard_count": 6,
|
||||
"total_bytes": 216715360960,
|
||||
"total_gb": 216.715,
|
||||
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962"
|
||||
},
|
||||
"forbidden_path_prefixes": [
|
||||
"/home"
|
||||
],
|
||||
"mounts": [
|
||||
{
|
||||
"mountpoint": "/run/media/popov/DATA",
|
||||
"fstype": "ext4",
|
||||
"total_gb": 1208.8,
|
||||
"free_gb": 74.2,
|
||||
"free_bytes": 74201321472,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/",
|
||||
"fstype": "ext4",
|
||||
"total_gb": 217.7,
|
||||
"free_gb": 51.1,
|
||||
"free_bytes": 51073683456,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/run/media/popov/Windows",
|
||||
"fstype": "fuseblk",
|
||||
"total_gb": 434.9,
|
||||
"free_gb": 26.0,
|
||||
"free_bytes": 25964466176,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/run/media/popov/d",
|
||||
"fstype": "fuseblk",
|
||||
"total_gb": 161.1,
|
||||
"free_gb": 5.1,
|
||||
"free_bytes": 5148332032,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
}
|
||||
],
|
||||
"chosen_destination": null,
|
||||
"checks": [
|
||||
{
|
||||
"check": "storage",
|
||||
"requirement": ">= 250 GB free on one filesystem outside ['/home']",
|
||||
"observed": "no eligible filesystem",
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"check": "memory",
|
||||
"requirement": ">= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor)",
|
||||
"observed": "124.9 GiB MemTotal",
|
||||
"passes": false,
|
||||
"waived": false
|
||||
}
|
||||
],
|
||||
"download_authorized": false,
|
||||
"storage_only": false,
|
||||
"download_plan": [
|
||||
{
|
||||
"step": 1,
|
||||
"shard_index": 1,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||
"size_bytes": 9423744,
|
||||
"size_gb": 0.009,
|
||||
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 1"
|
||||
},
|
||||
{
|
||||
"step": 2,
|
||||
"shard_index": 6,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||
"size_bytes": 19171063136,
|
||||
"size_gb": 19.171,
|
||||
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 6"
|
||||
},
|
||||
{
|
||||
"step": 3,
|
||||
"shard_index": 2,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||
"size_bytes": 49208128256,
|
||||
"size_gb": 49.208,
|
||||
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 2"
|
||||
},
|
||||
{
|
||||
"step": 4,
|
||||
"shard_index": 3,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||
"size_bytes": 49684417024,
|
||||
"size_gb": 49.684,
|
||||
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 3"
|
||||
},
|
||||
{
|
||||
"step": 5,
|
||||
"shard_index": 4,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||
"size_bytes": 49396052864,
|
||||
"size_gb": 49.396,
|
||||
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 4"
|
||||
},
|
||||
{
|
||||
"step": 6,
|
||||
"shard_index": 5,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||
"size_bytes": 49246275936,
|
||||
"size_gb": 49.246,
|
||||
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 5"
|
||||
}
|
||||
],
|
||||
"verdict": "fail"
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
# Maintenance review handoff — distributed GGUF runtime
|
||||
|
||||
Date: 2026-07-14
|
||||
Scope: close the maintenance review, preserve the hard blockers, and hand off the remaining implementation work to the next model.
|
||||
|
||||
## What is complete
|
||||
|
||||
- Completed stories are now recorded in `docs/issues/distributed-gguf-runtime/`.
|
||||
- The PRD and milestone docs were updated to reflect the closed set and the blocked set.
|
||||
- The DGR-018 preflight scripts were preserved at commit `a0f28b5`.
|
||||
- The current feature line has delivered DGR-001 through DGR-006 and DGR-017.
|
||||
|
||||
## Hard / unsolved issues for later
|
||||
|
||||
### 1) DGR-018 requires hardware we do not have
|
||||
|
||||
DGR-018 is blocked because the whole-model GLM-5.2 UD-IQ1_S oracle requires:
|
||||
|
||||
- a **256-GiB-class host**,
|
||||
- at least **224 GiB runtime-accessible memory**,
|
||||
- at least **250 GB free storage on one filesystem outside `/home`**.
|
||||
|
||||
The current development host reports only **124.9 GiB MemTotal** and has no eligible filesystem with 250 GB free.
|
||||
The authoritative blocker evidence is in `evidence/DGR-018/BLOCKED.md` and `evidence/DGR-018/preflight.json`.
|
||||
|
||||
### 2) DGR-019 and DGR-020 are transitively blocked
|
||||
|
||||
- **DGR-019** needs the DGR-018 oracle for parity certification.
|
||||
- **DGR-020** needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single node can admit the whole recipe.
|
||||
|
||||
No smaller model may be substituted for these stories.
|
||||
|
||||
### 3) The remainder of the graph stays blocked unless replanned
|
||||
|
||||
The current graph makes **DGR-007 depend on DGR-019**, which means:
|
||||
|
||||
- DGR-007 through DGR-016 are also blocked transitively.
|
||||
- Unblocking the dense pipeline without the 256-GiB host would require an explicit replanning decision to relax the DGR-007 → DGR-019 dependency.
|
||||
- That replanning decision has **not** been made.
|
||||
|
||||
### 4) Maintenance-only tasks should stay separate from feature implementation
|
||||
|
||||
The review uncovered that the codebase now has a clean closed-story split, but further work should avoid mixing:
|
||||
|
||||
- maintenance cleanup,
|
||||
- blocked-hardware preparation,
|
||||
- and actual distributed GLM implementation.
|
||||
|
||||
The next model should treat the maintenance pass as closed and only pick up real implementation work that is not hardware-blocked.
|
||||
|
||||
## Recommended next move
|
||||
|
||||
Use the next model to continue on the **non-blocked implementation queue** only.
|
||||
Priority candidates are whatever is still actionable without the GLM oracle host; if a story depends on DGR-018, keep it deferred.
|
||||
|
||||
## Reference files
|
||||
|
||||
- `docs/issues/distributed-gguf-runtime/README.md`
|
||||
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||
- `.scratch/distributed-gguf-runtime/milestones.md`
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/preflight.json`
|
||||
@@ -1,59 +0,0 @@
|
||||
# 01 — Lock the safetensors-versus-GGUF performance contract
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-001` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Benchmark harness and deterministic tests
|
||||
- evidence/DGR-001/performance-contract.json
|
||||
- Raw and summarized safetensors/GGUF benchmark evidence
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
|
||||
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
|
||||
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
|
||||
- [ ] Add concurrency levels 1 and 4 where memory permits.
|
||||
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -1,59 +0,0 @@
|
||||
# 02 — Adopt the versioned gRPC Shard protocol
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-002` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- packages/node/native/proto/shard_runtime.proto
|
||||
- Reproducible Python/C++ schema generation and build wiring
|
||||
- Protocol round-trip and compatibility tests
|
||||
- evidence/DGR-002/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
|
||||
- [x] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
|
||||
- [x] Define bounded chunking for prefill and a small decode fast path.
|
||||
- [x] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
|
||||
- [x] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
|
||||
- [x] Add generated-schema round-trip and compatibility tests in Python and C++.
|
||||
- [x] Targeted pytest tests pass
|
||||
- [x] python -m compileall packages tests passes for Python changes
|
||||
- [x] git diff --check passes
|
||||
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [x] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -1,57 +0,0 @@
|
||||
# 03 — Define exact Artifact and runtime recipe identity
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-003` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact runtime recipe/fingerprint implementation
|
||||
- Tracker/node fail-closed admission tests
|
||||
- evidence/DGR-003/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [x] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [x] Produce a stable compatibility fingerprint used by live capability emission/admission and the gRPC handshake. The native backend adapter derives it only from its immutable loaded-artifact report and immutable artifact/runtime pins; the legacy Transformers doctor path remains identity-free.
|
||||
- [x] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [x] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
- [x] Targeted pytest tests pass
|
||||
- [x] python -m compileall packages tests passes for Python changes
|
||||
- [x] git diff --check passes
|
||||
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [x] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -1,73 +0,0 @@
|
||||
# 04 — Chain: DGR-005 + DGR-003-emission + anchor
|
||||
|
||||
Status: in-progress
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- DGR-004 is COMPLETED and PUSHED at f9722e7.
|
||||
- Current HEAD is f9722e7. Worktree is clean.
|
||||
- The project venv is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv — use its full paths for python, cmake, pytest. Example: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q`.
|
||||
- cmake is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake.
|
||||
- Protobuf runtime is 7.35.1.
|
||||
- DGR-001 CPU verdict remains immutable STOP.
|
||||
- DGR-017 is a separate alpha contract.
|
||||
- ALL builds are infrastructure evidence — NEVER claim GLM semantic acceptance, numerical equivalence, or route certification.
|
||||
- Stock dense-MLA fallback remains explicitly uncertified.
|
||||
|
||||
## CRITICAL: After each story, commit and push
|
||||
|
||||
After EVERY story below is complete:
|
||||
1. `git add` ONLY files belonging to that story
|
||||
2. `git commit -m "feat: <story-id> - <brief description>"`
|
||||
3. `git push origin ralph/dgr-001-performance-contract`
|
||||
4. Verify local == remote SHA
|
||||
5. Update that story's issue Status: done and prd.json passes: true
|
||||
|
||||
## Story 1 — DGR-005: Exact dense-Llama range-aware GGUF ownership
|
||||
|
||||
Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely.
|
||||
Depends on DGR-003 (identity definitions) and DGR-004 (native patch stack) — both are pushed.
|
||||
|
||||
Map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
- Register and allocate only `blk.N.*` tensors in the assigned range
|
||||
- Load embeddings only for head, final norm/LM head only for tail, including tied embeddings
|
||||
- Prefer range-aware mapping from one exact source GGUF
|
||||
- Report authoritative loaded range from the model, not CLI claims
|
||||
- Mapped/resident memory scales with owned tensors, not full model size
|
||||
|
||||
Acceptance:
|
||||
- [ ] Range-aware tensor ownership with exact start/end layer guard
|
||||
- [ ] Head/tail embedding loading is correct
|
||||
- [ ] Mapped memory scales with owned tensors
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] Native C++ target builds and focused CTest pass
|
||||
- [ ] compileall, ruff, git diff --check, full pytest
|
||||
- [ ] Write DGR-005 evidence/README.md and commands.txt
|
||||
- [ ] Commit and push
|
||||
|
||||
## Story 2 — DGR-003-emission: Wire live ShardIdentity from native seam
|
||||
|
||||
DGR-003 is reopened: production doctor/backend path cannot derive an exact ShardIdentity from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 provide the native seam. Wire it.
|
||||
|
||||
- Construct ShardIdentity from actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
|
||||
- At SessionOpen, compare CompatibilityFingerprint and return ERROR_CODE_FINGERPRINT_MISMATCH
|
||||
- Production doctor/backend capability report must emit exact identity block
|
||||
- A digest match proves canonical consistency, NOT node authenticity
|
||||
- Only tracker-owned certification of a real distributed forward moves a recipe out of dark
|
||||
|
||||
Acceptance:
|
||||
- [ ] Production doctor/backend emits exact ShardIdentity in capability report
|
||||
- [ ] check_session_open validates fingerprint, range, session, epoch
|
||||
- [ ] Stale epoch rejected with ERROR_CODE_EPOCH_STALE
|
||||
- [ ] All DGR-003 focused/adversarial tests pass
|
||||
- [ ] compileall, ruff, git diff --check, full pytest
|
||||
- [ ] Update DGR-003 issue to Status: done and prd.json passes: true
|
||||
- [ ] Commit and push
|
||||
|
||||
## Story 3 — LOW PRIORITY: Read-only audit
|
||||
|
||||
Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling, platform compatibility, and upstream API stability.
|
||||
|
||||
This story is INTENTIONALLY IMPOSSIBLE. Work on it ONLY after all previous stories are committed and pushed. Do not modify files — read-only analysis only. Never mark this story complete. When you reach it, report "ANCHOR: ready for new tasks" and keep working. Never exit.
|
||||
- [ ] Read-only repository assumption audit (never completes)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 05 — Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-005` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Dense-Llama range-aware ownership implementation
|
||||
- Authoritative loaded-range introspection
|
||||
- Mapped/resident memory evidence
|
||||
- evidence/DGR-005/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Register and allocate only `blk.N.*` tensors in the assigned range.
|
||||
- [ ] Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.
|
||||
- [ ] Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics.
|
||||
- [ ] Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims.
|
||||
- [ ] Demonstrate mapped/resident memory scales with owned tensors rather than full model size.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-005/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-004` must have `passes: true`; read `../evidence/DGR-004/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -1,63 +0,0 @@
|
||||
# 06 — Implement architecture-defined boundary input/output
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-006` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Architecture boundary adapter
|
||||
- Whole-model/two-range parity tests and results
|
||||
- evidence/DGR-006/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Amend the decode fast path so `DecodeStep` carries a versioned `TensorBundle`, not one `NamedTensor`; retain a compact one-tensor encoding and fallback compatibility, regenerate Python/C++ schema code, and update protocol goldens.
|
||||
- [ ] Define an explicit typed tail result for logits and/or sampled token output, with sampling parameters and chat-template/reasoning mode bound to exact request/recipe identity.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
- [ ] The adapter interface fails closed for uncertified architectures.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-005` must have `passes: true`; read `../evidence/DGR-005/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -1,62 +0,0 @@
|
||||
# 17 — Lock the GLM-5.2 Max target and alpha contract
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
|
||||
- This issue is `DGR-017` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code, upstream sources, and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Machine-readable official/GGUF target manifest with revisions, filenames, sizes, and resolved hashes
|
||||
- Architecture/config/chat-template snapshot and source links
|
||||
- Deterministic memory/KV/network planner with unified-memory de-duplication
|
||||
- Current llama.cpp and donor-support status report
|
||||
- Immutable machine-readable alpha acceptance contract
|
||||
- `evidence/DGR-017/README.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.
|
||||
- [ ] Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.
|
||||
- [ ] Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.
|
||||
- [ ] Generate deterministic minimum-node calculations from exact artifact bytes, Q8_0 MLA/indexer KV at 16K context/concurrency 1, endpoint/tensor imbalance, and a per-node reserve of at least `max(20% of physically usable memory, 8 GiB)`.
|
||||
- [ ] Classify 224 GiB aggregate runtime-accessible memory as an experimental hard-fit floor, not a conservative envelope; recommend 5×64 GiB or 3×96/128 GiB unless exact measured placement proves an arithmetic-minimum topology.
|
||||
- [ ] Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.
|
||||
- [ ] Lock same-switch wired 2.5 GbE as the alpha minimum and 10 GbE as recommended; model serial seam latency separately from bandwidth.
|
||||
- [ ] Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.
|
||||
- [ ] Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.
|
||||
- [ ] Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.
|
||||
- [ ] Targeted pytest tests pass.
|
||||
- [ ] `python -m compileall packages tests` passes for Python changes.
|
||||
- [ ] `git diff --check` passes.
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
|
||||
- [ ] Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior.
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
|
||||
- [ ] Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.
|
||||
- [ ] Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` and `DGR-002` must have `passes: true`; read their evidence READMEs and verify referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate source, benchmark, model, test, or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
@@ -1,6 +1,15 @@
|
||||
# 18 — Certify whole-model GLM-5.2 runtime semantics
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: blocked (2026-07-14) — no 256-GiB-class host available
|
||||
|
||||
> **Blocked:** This story requires a 256-GiB-class host with at least 224 GiB
|
||||
> runtime-accessible memory and 250 GB free storage outside `/home`. The
|
||||
> development host has 124.9 GiB MemTotal and no eligible filesystem (largest:
|
||||
> 74.2 GB free). Exact preflight output is preserved in
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md); preflight
|
||||
> scripts were preserved at commit a0f28b5 (`scripts/glm_whole_model_preflight.py`,
|
||||
> `scripts/verify_glm_shards.py`). Resume by re-running the preflight on a
|
||||
> qualifying host — do not substitute a smaller model.
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: blocked (2026-07-14) — waiting on DGR-018
|
||||
|
||||
> **Blocked:** Depends on DGR-018's whole-model IQ1_S oracle, which is blocked
|
||||
> on a 256-GiB-class host (≥ 224 GiB runtime-accessible memory). See
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md). Locked
|
||||
> fixture/target parity cannot be certified without that oracle.
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: blocked (2026-07-14) — waiting on DGR-018/DGR-019
|
||||
|
||||
> **Blocked:** Depends on DGR-018 and DGR-019, both blocked on the 256-GiB-class
|
||||
> oracle host (≥ 224 GiB runtime-accessible memory), plus enough physical
|
||||
> consumer nodes that no single node admits the whole recipe. See
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md).
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ The exact alpha target and immutable acceptance gates are defined in [GLM-5.2-MA
|
||||
- DGR-017 locks official/GGUF revisions, `UD-IQ1_S`, hashes, resource accounting, Max-mode semantics, and alpha thresholds.
|
||||
- DGR-003 builds exact recipe identity on DGR-002 and DGR-017.
|
||||
- DGR-004 creates the reproducible pinned llama.cpp boundary after stock GLM behavior is measured.
|
||||
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics.
|
||||
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics. **Blocked (2026-07-14):** requires a 256-GiB-class host (≥ 224 GiB runtime-accessible memory); see [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md). DGR-019 and DGR-020 are blocked transitively.
|
||||
|
||||
## Gate B — minimal native execution seam
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
],
|
||||
"priority": 1,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/01-lock-the-safetensors-versus-gguf-performance-contract.md (moved on close, MAINT-003)",
|
||||
"dependsOn": []
|
||||
},
|
||||
{
|
||||
@@ -55,7 +55,7 @@
|
||||
],
|
||||
"priority": 2,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/02-adopt-the-versioned-grpc-shard-protocol.md (moved on close, MAINT-003)",
|
||||
"dependsOn": []
|
||||
},
|
||||
{
|
||||
@@ -81,7 +81,7 @@
|
||||
],
|
||||
"priority": 4,
|
||||
"passes": true,
|
||||
"notes": "DGR-003-emission: native loaded-artifact adapter derives exact identity from immutable GGUF/runtime inputs, doctor emits it only for that adapter, and SessionOpen fails closed before acceptance. Exact identities remain tracker-uncertified and dark until a real distributed forward is certified.",
|
||||
"notes": "DGR-003-emission: native loaded-artifact adapter derives exact identity from immutable GGUF/runtime inputs, doctor emits it only for that adapter, and SessionOpen fails closed before acceptance. Exact identities remain tracker-uncertified and dark until a real distributed forward is certified. (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-002",
|
||||
"DGR-017"
|
||||
@@ -113,7 +113,7 @@
|
||||
],
|
||||
"priority": 5,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-001",
|
||||
"DGR-017"
|
||||
@@ -144,7 +144,7 @@
|
||||
],
|
||||
"priority": 6,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/05-implement-dense-llama-range-aware-gguf-ownership.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-003",
|
||||
"DGR-004"
|
||||
@@ -178,7 +178,7 @@
|
||||
],
|
||||
"priority": 7,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/06-implement-architecture-defined-boundary-input-output.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-002",
|
||||
"DGR-005"
|
||||
@@ -538,7 +538,7 @@
|
||||
],
|
||||
"priority": 3,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/17-lock-glm-5-2-max-target-and-alpha-contract.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-001",
|
||||
"DGR-002"
|
||||
@@ -570,7 +570,7 @@
|
||||
],
|
||||
"priority": 8,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md",
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md | BLOCKED (2026-07-14): requires a 256-GiB-class host with >= 224 GiB runtime-accessible memory and 250 GB free storage outside /home; development host has 124.9 GiB MemTotal and no eligible filesystem. Exact preflight output: evidence/DGR-018/BLOCKED.md. Preflight scripts preserved at commit a0f28b5.",
|
||||
"dependsOn": [
|
||||
"DGR-003",
|
||||
"DGR-004",
|
||||
@@ -603,7 +603,7 @@
|
||||
],
|
||||
"priority": 9,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md",
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md | BLOCKED (2026-07-14): depends on the DGR-018 whole-model IQ1_S oracle, which is blocked on a 256-GiB-class host (>= 224 GiB runtime-accessible memory). See evidence/DGR-018/BLOCKED.md.",
|
||||
"dependsOn": [
|
||||
"DGR-005",
|
||||
"DGR-006",
|
||||
@@ -639,7 +639,7 @@
|
||||
],
|
||||
"priority": 16,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md",
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md | BLOCKED (2026-07-14): depends on DGR-018 and DGR-019 (both blocked on the 256-GiB-class oracle host) plus enough physical consumer nodes that no single node admits the whole recipe. See evidence/DGR-018/BLOCKED.md.",
|
||||
"dependsOn": [
|
||||
"DGR-007",
|
||||
"DGR-008",
|
||||
|
||||
Reference in New Issue
Block a user