Compare commits
26 Commits
ralph/dist
...
ralph/dgr-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cae7c2b171 | ||
|
|
64f83d4392 | ||
|
|
454a681a50 | ||
|
|
a0f28b5631 | ||
|
|
7925e5253d | ||
|
|
91c450840d | ||
|
|
d6b808dcf9 | ||
|
|
31065c0e12 | ||
|
|
ec36290863 | ||
|
|
f844ae6567 | ||
|
|
252d131e7d | ||
|
|
3d8f93f4aa | ||
|
|
f9722e7b57 | ||
|
|
7b8e467c6b | ||
|
|
7364ed6731 | ||
|
|
ad2d17541c | ||
|
|
e7c780a623 | ||
|
|
9580ed643e | ||
|
|
5ebce15d7a | ||
|
|
ef2a9e67e8 | ||
|
|
b1c9deeb01 | ||
|
|
9e67b829e3 | ||
|
|
e24db7854f | ||
|
|
59f2486bf2 | ||
|
|
d904c40f66 | ||
|
|
30dcf953fe |
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: ask-matt
|
name: ask-matt
|
||||||
description: Ask which skill or flow fits your situation. A router over the skills in this repo.
|
description: Ask which skill or flow fits your situation. A router over the user-invoked skills in this repo.
|
||||||
disable-model-invocation: true
|
disable-model-invocation: true
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -8,28 +8,26 @@ disable-model-invocation: true
|
|||||||
|
|
||||||
You don't remember every skill, so ask.
|
You don't remember every skill, so ask.
|
||||||
|
|
||||||
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone, or a vocabulary layer that runs underneath.
|
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone.
|
||||||
|
|
||||||
## The main flow: idea → ship
|
## The main flow: idea → ship
|
||||||
|
|
||||||
The route most work travels. You have an idea and want it built.
|
The route most work travels. You have an idea and want it built.
|
||||||
|
|
||||||
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone. Both run the same `/grilling` primitive; `grill-with-docs` is the one that leaves a paper trail.)
|
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone.)
|
||||||
2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (see Crossing sessions):
|
2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (see Crossing sessions):
|
||||||
- **`/handoff`** out, then open a fresh session against that file,
|
- **`/handoff`** out, then open a fresh session against that file,
|
||||||
- **`/prototype`** to answer the question with throwaway code,
|
- **`/prototype`** to answer the question with throwaway code,
|
||||||
- **`/handoff`** back what you learned, and reference it from the original idea thread.
|
- **`/handoff`** back what you learned, and reference it from the original idea thread.
|
||||||
3. **Branch — is this a multi-session build?**
|
3. **Branch — is this a multi-session build?**
|
||||||
- **Yes** → **`/to-spec`** (turn the thread into a spec), then **`/to-tickets`** to split it into tracer-bullet tickets, each declaring its **blocking edges**. On a local tracker that's one file per ticket under `.scratch/<feature>/issues/`, worked blockers-first by hand; on a real tracker the edges become native blocking links, so any ticket whose blockers are done can be grabbed — kick off **`/implement`** per ticket, **clearing context between each one**.
|
- **Yes** → **`/to-prd`** (turn the thread into a PRD) → **`/to-issues`** (split the PRD into independently-grabbable issues). Because the issues are independent, **clear context between each one**: start a fresh session per issue and kick off **`/implement`** by passing it the PRD and the single issue to work on.
|
||||||
- **No** → **`/implement`** right here, in the same context window.
|
- **No** → **`/implement`** right here, in the same context window.
|
||||||
|
|
||||||
Either way, **`/implement`** builds each issue by driving **`/tdd`** internally — one red-green slice at a time — then closes out by running **`/code-review`**, a two-axis review (Standards + Spec) of the diff, before committing. Reach for **`/tdd`** on its own when you just want to build a concrete behaviour test-first without a full spec, and **`/code-review`** on its own whenever you want to review a branch or PR against a fixed point.
|
|
||||||
|
|
||||||
### Context hygiene
|
### Context hygiene
|
||||||
|
|
||||||
Keep steps 1–3 in **one unbroken context window** — don't compact or clear until after `/to-tickets` — so the grilling, spec, and tickets all build on the same thinking. Each `/implement` then starts fresh, working from the ticket.
|
Keep steps 1–3 in **one unbroken context window** — don't compact or clear until after `/to-issues` — so the grilling, PRD, and issues all build on the same thinking. Each `/implement` then starts fresh, working from the issue.
|
||||||
|
|
||||||
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-tickets`, don't push on degraded — `/handoff` and continue in a fresh thread.
|
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-issues`, don't push on degraded — `/handoff` and continue in a fresh thread.
|
||||||
|
|
||||||
## On-ramps
|
## On-ramps
|
||||||
|
|
||||||
@@ -37,26 +35,13 @@ A starting situation that generates work, then merges onto the main flow.
|
|||||||
|
|
||||||
- **Bugs and requests piling up** → **`/triage`**. It moves issues through triage roles and produces agent-ready issues, which **`/implement`** later picks up.
|
- **Bugs and requests piling up** → **`/triage`**. It moves issues through triage roles and produces agent-ready issues, which **`/implement`** later picks up.
|
||||||
|
|
||||||
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Tickets that `/to-tickets` produced are already agent-ready, so **don't triage them**.
|
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Issues that `/to-issues` produced are already agent-ready, so **don't triage them**.
|
||||||
|
|
||||||
- **Something's broken** → **`/diagnosing-bugs`**. For the hard ones: the bug that resists a first glance, the intermittent flake, the regression that crept in between two known-good states. It refuses to theorise until it has a **tight feedback loop** — one command that already goes red on *this* bug — then fixes with a regression test. Its post-mortem hands off to **`/improve-codebase-architecture`** when the real finding is that there's no good seam to lock the bug down.
|
|
||||||
|
|
||||||
- **A huge, foggy effort — a greenfield project or a huge feature build, too big for one session** → **`/wayfinder`**, the most cognitively demanding flow here. When the way from here to the destination isn't visible yet, it charts a **shared map** of **decision tickets** on the issue tracker and resolves them one at a time — producing **decisions, not deliverables** — until the fog is pushed back and the way is clear. Where **`/grill-with-docs`** sharpens an idea you can hold in one session, wayfinder is for the idea you can't — and it's slower and denser, so save it for exactly that, never a well-scoped feature.
|
|
||||||
|
|
||||||
When the map clears, **it hands off, it doesn't build**: merge onto the main flow at **`/to-spec`**, which collapses the map's linked decisions into a buildable plan, then `/to-tickets` and `/implement` as usual. Looping the map straight into `/implement` skips that collapse and throws the linked detail away — go straight to `/implement` only when the effort turned out genuinely small.
|
|
||||||
|
|
||||||
## Codebase health
|
## Codebase health
|
||||||
|
|
||||||
Not feature work — upkeep.
|
Not feature work — upkeep.
|
||||||
|
|
||||||
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces **deepening opportunities**; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`. It's the survey that finds the candidates; **`/codebase-design`** (below) is the bench you design the chosen one on.
|
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces deepening opportunities; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`.
|
||||||
|
|
||||||
## Vocabulary underneath
|
|
||||||
|
|
||||||
Two model-invoked references that run *beneath* the other skills — each the single source of truth for its vocabulary. Reach for them directly when the **words**, not the process, are the problem; or let the skills above pull them in.
|
|
||||||
|
|
||||||
- **`/domain-modeling`** — sharpen the project's *domain* language: challenge a fuzzy term, resolve an overloaded word ("account" doing three jobs), record a hard-to-reverse decision as an ADR. It's the active discipline `/grill-with-docs` drives to keep `CONTEXT.md` a clean glossary.
|
|
||||||
- **`/codebase-design`** — the deep-module vocabulary (module, interface, depth, seam, adapter, leverage, locality) for designing a module's *shape*: a lot of behaviour behind a small interface at a clean seam. `/tdd` and `/improve-codebase-architecture` both speak it.
|
|
||||||
|
|
||||||
## Crossing sessions
|
## Crossing sessions
|
||||||
|
|
||||||
@@ -68,8 +53,6 @@ Two model-invoked references that run *beneath* the other skills — each the si
|
|||||||
Off the main flow entirely.
|
Off the main flow entirely.
|
||||||
|
|
||||||
- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but for when you have **no codebase**. Stateless: it saves nothing locally, builds no `CONTEXT.md`. Reach for it to sharpen any plan or design that doesn't live in a repo.
|
- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but for when you have **no codebase**. Stateless: it saves nothing locally, builds no `CONTEXT.md`. Reach for it to sharpen any plan or design that doesn't live in a repo.
|
||||||
- **`/prototype`** — a small, throwaway program that answers one design question: does this state model feel right, or what should this UI look like. Throwaway from day one — keep the answer, delete the code. It's the detour in step 2 of the main flow, but reach for it any time a design question is hard to settle on paper.
|
|
||||||
- **`/research`** — delegate reading legwork to a **background agent**: it investigates a question against **primary sources**, then leaves a cited Markdown file in the repo. Keep working while it reads. The file it produces is something to take *into* the main flow at `/grill-with-docs` — research feeds the thinking, it doesn't replace it.
|
|
||||||
- **`/teach`** — learn a concept over multiple sessions, using the current directory as a stateful workspace.
|
- **`/teach`** — learn a concept over multiple sessions, using the current directory as a stateful workspace.
|
||||||
- **`/writing-great-skills`** — reference for writing and editing skills well.
|
- **`/writing-great-skills`** — reference for writing and editing skills well.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Ask Matt"
|
|
||||||
short_description: "Find the right skill or workflow"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -1,12 +1,10 @@
|
|||||||
---
|
---
|
||||||
name: grilling
|
name: grilling
|
||||||
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
|
description: Interview the user relentlessly about a plan or design. Use when the user wants to stress-test a plan before building, or uses any 'grill' trigger phrases.
|
||||||
---
|
---
|
||||||
|
|
||||||
Interview me relentlessly about every aspect of this until we reach a shared understanding. Walk down each branch of the decision tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
|
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
|
||||||
|
|
||||||
Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering.
|
Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering.
|
||||||
|
|
||||||
If a *fact* can be found by exploring the environment (filesystem, tools, etc.), look it up rather than asking me. The *decisions*, though, are mine — put each one to me and wait for my answer.
|
If a question can be answered by exploring the codebase, explore the codebase instead.
|
||||||
|
|
||||||
Do not act on it until I confirm we have reached a shared understanding.
|
|
||||||
|
|||||||
@@ -1,3 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Grilling"
|
|
||||||
short_description: "Stress-test thinking one question at a time"
|
|
||||||
@@ -9,7 +9,7 @@ Write a handoff document summarising the current conversation so a fresh agent c
|
|||||||
|
|
||||||
Include a "suggested skills" section in the document, which suggests skills that the agent should invoke.
|
Include a "suggested skills" section in the document, which suggests skills that the agent should invoke.
|
||||||
|
|
||||||
Do not duplicate content already captured in other artifacts (specs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
Do not duplicate content already captured in other artifacts (PRDs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
||||||
|
|
||||||
Redact any sensitive information, such as API keys, passwords, or personally identifiable information.
|
Redact any sensitive information, such as API keys, passwords, or personally identifiable information.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Handoff"
|
|
||||||
short_description: "Compact a conversation into a handoff"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -1,15 +1,15 @@
|
|||||||
---
|
---
|
||||||
name: implement
|
name: implement
|
||||||
description: "Implement a piece of work based on a spec or set of tickets."
|
description: "Implement a piece of work based on a PRD or set of issues."
|
||||||
disable-model-invocation: true
|
disable-model-invocation: true
|
||||||
---
|
---
|
||||||
|
|
||||||
Implement the work described by the user in the spec or tickets.
|
Implement the work described by the user in the PRD or issues.
|
||||||
|
|
||||||
Use /tdd where possible, at pre-agreed seams.
|
Use /tdd where possible, at pre-agreed seams.
|
||||||
|
|
||||||
Run typechecking regularly, single test files regularly, and the full test suite once at the end.
|
Run typechecking regularly, single test files regularly, and the full test suite once at the end.
|
||||||
|
|
||||||
Once done, use /code-review to review the work.
|
Once done, use /review to review the work.
|
||||||
|
|
||||||
Commit your work to the current branch.
|
Commit your work to the current branch.
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Implement"
|
|
||||||
short_description: "Build work from a spec or tickets"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -17,11 +17,6 @@ This command is _informed_ by the project's domain model and built on a shared d
|
|||||||
|
|
||||||
### 1. Explore
|
### 1. Explore
|
||||||
|
|
||||||
**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look:
|
|
||||||
|
|
||||||
- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below.
|
|
||||||
- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net.
|
|
||||||
|
|
||||||
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
|
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
|
||||||
|
|
||||||
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
|
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
|
||||||
@@ -61,7 +56,7 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o
|
|||||||
|
|
||||||
### 3. Grilling loop
|
### 3. Grilling loop
|
||||||
|
|
||||||
Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
|
Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
|
||||||
|
|
||||||
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
|
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Improve Codebase Architecture"
|
|
||||||
short_description: "Find and grill architecture improvements"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -36,7 +36,7 @@ The right shape depends on the question:
|
|||||||
|
|
||||||
Pick whichever shape best fits the question being asked, *not* whichever is easiest to wire to a TUI. Keep it pure: no I/O, no terminal code, no `console.log` for control flow. The TUI imports it and calls into it; nothing flows the other direction.
|
Pick whichever shape best fits the question being asked, *not* whichever is easiest to wire to a TUI. Keep it pure: no I/O, no terminal code, no `console.log` for control flow. The TUI imports it and calls into it; nothing flows the other direction.
|
||||||
|
|
||||||
This is what makes the prototype useful past its own lifetime: when the question's been answered, the validated reducer / machine / function set can be lifted into the real module on its own.
|
This is what makes the prototype useful past its own lifetime. When the question's been answered, the validated reducer / machine / function set can be lifted into the real module — the TUI shell gets deleted.
|
||||||
|
|
||||||
### 4. Build the smallest TUI that exposes the state
|
### 4. Build the smallest TUI that exposes the state
|
||||||
|
|
||||||
@@ -66,9 +66,9 @@ If the host project has no task runner, just put the command at the top of the p
|
|||||||
|
|
||||||
Give the user the run command. They'll drive it themselves; the interesting moments are when they say "wait, that shouldn't be possible" or "huh, I assumed X would be different" — those are the bugs in the _idea_, which is the whole point. If they want new actions added, add them. Prototypes evolve.
|
Give the user the run command. They'll drive it themselves; the interesting moments are when they say "wait, that shouldn't be possible" or "huh, I assumed X would be different" — those are the bugs in the _idea_, which is the whole point. If they want new actions added, add them. Prototypes evolve.
|
||||||
|
|
||||||
### 7. Capture the answer and the prototype
|
### 7. Capture the answer
|
||||||
|
|
||||||
Once the prototype has answered its question, capture the answer, then capture the prototype the way the [SKILL](SKILL.md) describes. The logic-specific mapping: the validated reducer / machine / function set lifts into the real module (the decision, absorbed); the TUI shell rides along to the throwaway branch that keeps the prototype as a primary source.
|
When the prototype has done its job, the answer to the question is the only thing worth keeping. If the user is around, ask what it taught them. If not, leave a `NOTES.md` next to the prototype so the answer can be filled in (or filled in by you, if you've watched the session) before the prototype gets deleted.
|
||||||
|
|
||||||
## Anti-patterns
|
## Anti-patterns
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
---
|
---
|
||||||
name: prototype
|
name: prototype
|
||||||
description: Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
|
description: Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
|
||||||
|
disable-model-invocation: true
|
||||||
---
|
---
|
||||||
|
|
||||||
# Prototype
|
# Prototype
|
||||||
@@ -21,6 +22,10 @@ The two branches produce very different artifacts — getting this wrong wastes
|
|||||||
1. **Throwaway from day one, and clearly marked as such.** Locate the prototype code close to where it will actually be used (next to the module or page it's prototyping for) so context is obvious — but name it so a casual reader can see it's a prototype, not production. For throwaway UI routes, obey whatever routing convention the project already uses; don't invent a new top-level structure.
|
1. **Throwaway from day one, and clearly marked as such.** Locate the prototype code close to where it will actually be used (next to the module or page it's prototyping for) so context is obvious — but name it so a casual reader can see it's a prototype, not production. For throwaway UI routes, obey whatever routing convention the project already uses; don't invent a new top-level structure.
|
||||||
2. **One command to run.** Whatever the project's existing task runner supports — `pnpm <name>`, `python <path>`, `bun <path>`, etc. The user must be able to start it without thinking.
|
2. **One command to run.** Whatever the project's existing task runner supports — `pnpm <name>`, `python <path>`, `bun <path>`, etc. The user must be able to start it without thinking.
|
||||||
3. **No persistence by default.** State lives in memory. Persistence is the thing the prototype is _checking_, not something it should depend on. If the question explicitly involves a database, hit a scratch DB or a local file with a clear "PROTOTYPE — wipe me" name.
|
3. **No persistence by default.** State lives in memory. Persistence is the thing the prototype is _checking_, not something it should depend on. If the question explicitly involves a database, hit a scratch DB or a local file with a clear "PROTOTYPE — wipe me" name.
|
||||||
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast.
|
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast and then delete it.
|
||||||
5. **Surface the state.** After every action (logic) or on every variant switch (UI), print or render the full relevant state so the user can see what changed.
|
5. **Surface the state.** After every action (logic) or on every variant switch (UI), print or render the full relevant state so the user can see what changed.
|
||||||
6. **Capture it when done.** Fold any validated decision into the real code, then capture the prototype itself as a **primary source**: commit it to a throwaway branch, out of main, and leave a context pointer to that branch on the implementation issue. Capture the answer too — the verdict and the question it settled — in the issue or a commit. The main branch keeps only the validated decision.
|
6. **Delete or absorb when done.** When the prototype has answered its question, either delete it or fold the validated decision into the real code — don't leave it rotting in the repo.
|
||||||
|
|
||||||
|
## When done
|
||||||
|
|
||||||
|
The _answer_ is the only thing worth keeping from a prototype. Capture it somewhere durable (commit message, ADR, issue, or a `NOTES.md` next to the prototype) along with the question it was answering. If the user is around, that capture is a quick conversation; if not, leave the placeholder so they (or you, on the next pass) can fill in the verdict before deleting the prototype.
|
||||||
|
|||||||
@@ -97,12 +97,12 @@ Surface the URL (and the `?variant=` keys). The user will flip through whenever
|
|||||||
|
|
||||||
### 6. Capture the answer and clean up
|
### 6. Capture the answer and clean up
|
||||||
|
|
||||||
Once a variant has won, capture the answer — which variant and why — then capture the prototype the way the [SKILL](SKILL.md) describes. Fold the winner into the real code and move the rest onto the throwaway branch, not into main:
|
Once a variant has won, write down which one and why (commit message, ADR, issue, or a `NOTES.md` next to the prototype if running AFK and the user hasn't responded yet). Then:
|
||||||
|
|
||||||
- **Sub-shape A** — fold the winner into the existing page; drop the losing variants and the switcher from main.
|
- **Sub-shape A** — delete the losing variants and the switcher; fold the winner into the existing page.
|
||||||
- **Sub-shape B** — promote the winning variant to a real route; drop the throwaway route and the switcher from main.
|
- **Sub-shape B** — promote the winning variant to a real route, delete the throwaway route and the switcher.
|
||||||
|
|
||||||
The full set of variants is the primary source, so it lands on the throwaway branch, not the bin — variant components and the switcher left in the main branch rot fast and confuse the next reader.
|
Don't leave variant components or the switcher lying around. They rot fast and confuse the next reader.
|
||||||
|
|
||||||
## Anti-patterns
|
## Anti-patterns
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Prototype"
|
|
||||||
short_description: "Prototype to answer a design question"
|
|
||||||
@@ -26,18 +26,16 @@ Look at the current repo to understand its starting state. Read whatever exists;
|
|||||||
- `docs/adr/` and any `src/*/docs/adr/` directories
|
- `docs/adr/` and any `src/*/docs/adr/` directories
|
||||||
- `docs/agents/` — does this skill's prior output already exist?
|
- `docs/agents/` — does this skill's prior output already exist?
|
||||||
- `.scratch/` — sign that a local-markdown issue tracker convention is already in use
|
- `.scratch/` — sign that a local-markdown issue tracker convention is already in use
|
||||||
- Is the `triage` skill installed? (a `triage` skill folder alongside this one, or `triage` in your available skills.) This decides whether Section B runs at all.
|
|
||||||
- Monorepo signals — a `pnpm-workspace.yaml`, a `workspaces` field in `package.json`, or a populated `packages/*` with its own `src/`. Present only in a genuinely large multi-package repo; their absence means single-context, which is almost every repo.
|
|
||||||
|
|
||||||
### 2. Present findings and ask
|
### 2. Present findings and ask
|
||||||
|
|
||||||
Summarise what's present and what's missing. Then take the sections in order — one section, one answer, then the next.
|
Summarise what's present and what's missing. Then walk the user through the three decisions **one at a time** — present a section, get the user's answer, then move to the next. Don't dump all three at once.
|
||||||
|
|
||||||
Lead each section with the recommended answer so the user can accept it in a word. Give a one-line explainer only when the choice genuinely branches; skip the section entirely when exploration already settled it (Section B when `triage` isn't installed, Section C when there's no monorepo).
|
Assume the user does not know what these terms mean. Each section starts with a short explainer (what it is, why these skills need it, what changes if they pick differently). Then show the choices and the default.
|
||||||
|
|
||||||
**Section A — Issue tracker.**
|
**Section A — Issue tracker.**
|
||||||
|
|
||||||
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-tickets`, `triage`, `to-spec`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
|
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-issues`, `triage`, `to-prd`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
|
||||||
|
|
||||||
Default posture: these skills were designed for GitHub. If a `git remote` points at GitHub, propose that. If a `git remote` points at GitLab (`gitlab.com` or a self-hosted host), propose GitLab. Otherwise (or if the user prefers), offer:
|
Default posture: these skills were designed for GitHub. If a `git remote` points at GitHub, propose that. If a `git remote` points at GitLab (`gitlab.com` or a self-hosted host), propose GitLab. Otherwise (or if the user prefers), offer:
|
||||||
|
|
||||||
@@ -46,26 +44,41 @@ Default posture: these skills were designed for GitHub. If a `git remote` points
|
|||||||
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
|
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
|
||||||
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
|
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
|
||||||
|
|
||||||
Record the choice in `docs/agents/issue-tracker.md`. The GitHub and GitLab templates carry a "PRs as a request surface" flag, defaulted **off** — leave it off and don't raise it; a user who wants external PRs in the triage queue can flip the flag in the file later.
|
If — and only if — the user picked **GitHub** or **GitLab**, ask one follow-up:
|
||||||
|
|
||||||
**Section B — Triage label vocabulary.** Skip this section entirely if the `triage` skill isn't installed (exploration told you) — an uninstalled skill needs no labels.
|
> Explainer: Open-source repos often receive feature requests as pull requests, not just issues — a PR is an issue with attached code. If you turn this on, `/triage` pulls *external* PRs into the same queue and runs them through the same labels and states as issues (collaborators' in-flight PRs are left alone). Leave it off if PRs aren't a request surface for you.
|
||||||
|
|
||||||
If it is installed, ask exactly one question:
|
- **PRs as a request surface** — yes / no (default: no). Record the answer in `docs/agents/issue-tracker.md`. For local-markdown and other trackers, skip this question — there are no PRs.
|
||||||
|
|
||||||
> Do you want to keep the default triage labels? (recommended: **yes**)
|
**Section B — Triage label vocabulary.**
|
||||||
|
|
||||||
The defaults are the five canonical roles, each label string equal to its name: `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`. On **yes**, write them as-is. Only if the user says no — usually because their tracker already uses other names (e.g. `bug:triage` for `needs-triage`) — collect the overrides so `triage` applies existing labels instead of creating duplicates.
|
> Explainer: When the `triage` skill processes an incoming issue, it moves it through a state machine — needs evaluation, waiting on reporter, ready for an AFK agent to pick up, ready for a human, or won't fix. To do that, it needs to apply labels (or the equivalent in your issue tracker) that match strings *you've actually configured*. If your repo already uses different label names (e.g. `bug:triage` instead of `needs-triage`), map them here so the skill applies the right ones instead of creating duplicates.
|
||||||
|
|
||||||
**Section C — Domain docs.** Default to **single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. This fits almost every repo; write it without asking.
|
The five canonical roles:
|
||||||
|
|
||||||
Offer **multi-context** — a root `CONTEXT-MAP.md` pointing to per-context `CONTEXT.md` files — only when exploration found monorepo signals. Then confirm which layout they want.
|
- `needs-triage` — maintainer needs to evaluate
|
||||||
|
- `needs-info` — waiting on reporter
|
||||||
|
- `ready-for-agent` — fully specified, AFK-ready (an agent can pick it up with no human context)
|
||||||
|
- `ready-for-human` — needs human implementation
|
||||||
|
- `wontfix` — will not be actioned
|
||||||
|
|
||||||
|
Default: each role's string equals its name. Ask the user if they want to override any. If their issue tracker has no existing labels, the defaults are fine.
|
||||||
|
|
||||||
|
**Section C — Domain docs.**
|
||||||
|
|
||||||
|
> Explainer: Some skills (`improve-codebase-architecture`, `diagnosing-bugs`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
|
||||||
|
|
||||||
|
Confirm the layout:
|
||||||
|
|
||||||
|
- **Single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. Most repos are this.
|
||||||
|
- **Multi-context** — `CONTEXT-MAP.md` at the root pointing to per-context `CONTEXT.md` files (typically a monorepo).
|
||||||
|
|
||||||
### 3. Confirm and edit
|
### 3. Confirm and edit
|
||||||
|
|
||||||
Show the user a draft of:
|
Show the user a draft of:
|
||||||
|
|
||||||
- The `## Agent skills` block to add to whichever of `CLAUDE.md` / `AGENTS.md` is being edited (see step 4 for selection rules)
|
- The `## Agent skills` block to add to whichever of `CLAUDE.md` / `AGENTS.md` is being edited (see step 4 for selection rules)
|
||||||
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/domain.md`, and `docs/agents/triage-labels.md` (the last only when `triage` is installed)
|
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/triage-labels.md`, `docs/agents/domain.md`
|
||||||
|
|
||||||
Let them edit before writing.
|
Let them edit before writing.
|
||||||
|
|
||||||
@@ -88,7 +101,7 @@ The block:
|
|||||||
|
|
||||||
### Issue tracker
|
### Issue tracker
|
||||||
|
|
||||||
[one-line summary of where issues are tracked]. See `docs/agents/issue-tracker.md`.
|
[one-line summary of where issues are tracked, plus whether external PRs are a triage surface]. See `docs/agents/issue-tracker.md`.
|
||||||
|
|
||||||
### Triage labels
|
### Triage labels
|
||||||
|
|
||||||
@@ -99,14 +112,12 @@ The block:
|
|||||||
[one-line summary of layout — "single-context" or "multi-context"]. See `docs/agents/domain.md`.
|
[one-line summary of layout — "single-context" or "multi-context"]. See `docs/agents/domain.md`.
|
||||||
```
|
```
|
||||||
|
|
||||||
Include the `### Triage labels` sub-block, and write `docs/agents/triage-labels.md`, only when `triage` is installed and Section B ran. When it isn't, both are omitted.
|
Then write the three docs files using the seed templates in this skill folder as a starting point:
|
||||||
|
|
||||||
Then write the docs files using the seed templates in this skill folder as a starting point:
|
|
||||||
|
|
||||||
- [issue-tracker-github.md](./issue-tracker-github.md) — GitHub issue tracker
|
- [issue-tracker-github.md](./issue-tracker-github.md) — GitHub issue tracker
|
||||||
- [issue-tracker-gitlab.md](./issue-tracker-gitlab.md) — GitLab issue tracker
|
- [issue-tracker-gitlab.md](./issue-tracker-gitlab.md) — GitLab issue tracker
|
||||||
- [issue-tracker-local.md](./issue-tracker-local.md) — local-markdown issue tracker
|
- [issue-tracker-local.md](./issue-tracker-local.md) — local-markdown issue tracker
|
||||||
- [triage-labels.md](./triage-labels.md) — label mapping (only if `triage` is installed)
|
- [triage-labels.md](./triage-labels.md) — label mapping
|
||||||
- [domain.md](./domain.md) — domain doc consumer rules + layout
|
- [domain.md](./domain.md) — domain doc consumer rules + layout
|
||||||
|
|
||||||
For "other" issue trackers, write `docs/agents/issue-tracker.md` from scratch using the user's description.
|
For "other" issue trackers, write `docs/agents/issue-tracker.md` from scratch using the user's description.
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Setup Matt Pocock Skills"
|
|
||||||
short_description: "Configure a repo for the skills"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -32,14 +32,3 @@ Create a GitHub issue.
|
|||||||
## When a skill says "fetch the relevant ticket"
|
## When a skill says "fetch the relevant ticket"
|
||||||
|
|
||||||
Run `gh issue view <number> --comments`.
|
Run `gh issue view <number> --comments`.
|
||||||
|
|
||||||
## Wayfinding operations
|
|
||||||
|
|
||||||
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
|
|
||||||
|
|
||||||
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `gh issue create --label wayfinder:map`.
|
|
||||||
- **Child ticket**: an issue linked to the map as a GitHub sub-issue (`gh api` on the sub-issues endpoint). Where sub-issues aren't enabled, add the child to a task list in the map body and put `Part of #<map>` at the top of the child body. Labels: `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
|
|
||||||
- **Blocking**: GitHub's **native issue dependencies** — the canonical, UI-visible representation. Add an edge with `gh api --method POST repos/<owner>/<repo>/issues/<child>/dependencies/blocked_by -F issue_id=<blocker-db-id>`, where `<blocker-db-id>` is the blocker's numeric **database id** (`gh api repos/<owner>/<repo>/issues/<n> --jq .id`, _not_ the `#number` or `node_id`). GitHub reports `issue_dependencies_summary.blocked_by` (open blockers only — the live gate). Where dependencies aren't available, fall back to a `Blocked by: #<n>, #<n>` line at the top of the child body. A ticket is unblocked when every blocker is closed.
|
|
||||||
- **Frontier query**: list the map's open children (`gh issue list --state open`, scoped to the map's sub-issues / task list), drop any with an open blocker (`issue_dependencies_summary.blocked_by > 0`, or an open issue in the `Blocked by` line) or an assignee; first in map order wins.
|
|
||||||
- **Claim**: `gh issue edit <n> --add-assignee @me` — the session's first write.
|
|
||||||
- **Resolve**: `gh issue comment <n> --body "<answer>"`, then `gh issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.
|
|
||||||
|
|||||||
@@ -33,14 +33,3 @@ Create a GitLab issue.
|
|||||||
## When a skill says "fetch the relevant ticket"
|
## When a skill says "fetch the relevant ticket"
|
||||||
|
|
||||||
Run `glab issue view <number> --comments`.
|
Run `glab issue view <number> --comments`.
|
||||||
|
|
||||||
## Wayfinding operations
|
|
||||||
|
|
||||||
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
|
|
||||||
|
|
||||||
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `glab issue create --label wayfinder:map`. (On GitLab tiers with native epics, an epic may hold the map instead; a labelled issue works everywhere.)
|
|
||||||
- **Child ticket**: an issue carrying `Part of #<map>` at the top of its description and labels `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
|
|
||||||
- **Blocking**: GitLab's **native blocking link** — the canonical, UI-visible representation. Add it with the `/blocked_by #<n>` quick action, posted as a note (`glab issue note <child> --message "/blocked_by #<blocker>"`). Native blocking links are a Premium/Ultimate feature; on the free tier (or where unavailable) fall back to a `Blocked by: #<n>, #<n>` line at the top of the description. A ticket is unblocked when every blocker is closed.
|
|
||||||
- **Frontier query**: `glab issue list -F json` scoped to the map's children, drop any with an open blocker — a native `blocked_by` link to an open issue (`glab api projects/:id/issues/:iid/links`), or an open issue in the `Blocked by` line — or an assignee; first in map order wins.
|
|
||||||
- **Claim**: `glab issue update <n> --assignee @me` — the session's first write.
|
|
||||||
- **Resolve**: `glab issue note <n> --message "<answer>"`, then `glab issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.
|
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
# Issue tracker: Local Markdown
|
# Issue tracker: Local Markdown
|
||||||
|
|
||||||
Issues and specs (you may know a spec as a PRD) for this repo live as markdown files in `.scratch/`.
|
Issues and PRDs for this repo live as markdown files in `.scratch/`.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- One feature per directory: `.scratch/<feature-slug>/`
|
- One feature per directory: `.scratch/<feature-slug>/`
|
||||||
- The spec is `.scratch/<feature-slug>/spec.md`
|
- The PRD is `.scratch/<feature-slug>/PRD.md`
|
||||||
- Implementation issues are one file per ticket at `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` — never a single combined tickets file
|
- Implementation issues are `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01`
|
||||||
- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings)
|
- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings)
|
||||||
- Comments and conversation history append to the bottom of the file under a `## Comments` heading
|
- Comments and conversation history append to the bottom of the file under a `## Comments` heading
|
||||||
|
|
||||||
@@ -17,14 +17,3 @@ Create a new file under `.scratch/<feature-slug>/` (creating the directory if ne
|
|||||||
## When a skill says "fetch the relevant ticket"
|
## When a skill says "fetch the relevant ticket"
|
||||||
|
|
||||||
Read the file at the referenced path. The user will normally pass the path or the issue number directly.
|
Read the file at the referenced path. The user will normally pass the path or the issue number directly.
|
||||||
|
|
||||||
## Wayfinding operations
|
|
||||||
|
|
||||||
Used by `/wayfinder`. The **map** is a file with one **child** file per ticket.
|
|
||||||
|
|
||||||
- **Map**: `.scratch/<effort>/map.md` — the Notes / Decisions-so-far / Fog body.
|
|
||||||
- **Child ticket**: `.scratch/<effort>/issues/NN-<slug>.md`, numbered from `01`, with the question in the body. A `Type:` line records the ticket type (`research`/`prototype`/`grilling`/`task`); a `Status:` line records `claimed`/`resolved`.
|
|
||||||
- **Blocking**: a `Blocked by: NN, NN` line near the top. A ticket is unblocked when every file it lists is `resolved`.
|
|
||||||
- **Frontier**: scan `.scratch/<effort>/issues/` for files that are open, unblocked, and unclaimed; first by number wins.
|
|
||||||
- **Claim**: set `Status: claimed` and save before any work.
|
|
||||||
- **Resolve**: append the answer under an `## Answer` heading, set `Status: resolved`, then append a context pointer (gist + link) to the map's Decisions-so-far in `map.md`.
|
|
||||||
|
|||||||
@@ -1,102 +0,0 @@
|
|||||||
---
|
|
||||||
name: setup-ts-deep-modules
|
|
||||||
description: Wire dependency-cruiser into a TypeScript repo so each package is a deep module — implementation hidden in subfolders, reachable only through its entry-point files. User-invoked.
|
|
||||||
disable-model-invocation: true
|
|
||||||
---
|
|
||||||
|
|
||||||
# Setup TS Deep Modules
|
|
||||||
|
|
||||||
Make every package in this repo a **deep module**: a lot of behaviour behind a small interface. A package's public surface is its **entry points** — the files at the package root — and everything in its subfolders is hidden. This skill installs [dependency-cruiser](https://github.com/sverweij/dependency-cruiser) and the rules that make the entry points the only way in, then proves the rules bite.
|
|
||||||
|
|
||||||
For the vocabulary (deep module, interface, seam, depth), run the `/codebase-design` skill — use its language throughout.
|
|
||||||
|
|
||||||
## The shape this enforces
|
|
||||||
|
|
||||||
```
|
|
||||||
src/packages/
|
|
||||||
<name>/
|
|
||||||
index.ts ← an entry point (public). Import this from outside.
|
|
||||||
client.ts ← another entry point. Packages may expose SEVERAL.
|
|
||||||
lib/ ← implementation: hidden from outside, free to import each other.
|
|
||||||
tests/ ← co-located tests + fixtures (a subfolder, so private).
|
|
||||||
```
|
|
||||||
|
|
||||||
The public surface is the package's **root files** — not one designated `index.ts`. By convention implementation lives in `lib/` and tests in `tests/`, giving every package the same two-folder shape. The rule itself is general, though: *anything* in *any* subfolder is private, so you never extend the config to add a folder.
|
|
||||||
|
|
||||||
Four rules, all `error`:
|
|
||||||
|
|
||||||
1. **Entry-point boundary** — code outside a package (app code or another package) may import only that package's entry points (its root files), never anything in its subfolders.
|
|
||||||
2. **Intra-package freedom** — a package's own files import each other freely.
|
|
||||||
3. **Tests through the entry points** — files under `<pkg>/tests/` may import any package's entry points and their own `tests/` fixtures, but never any package's subfolder internals (not even their own). Integration tests across packages are fine; deep imports are not.
|
|
||||||
4. **No cycles** — no dependency cycles.
|
|
||||||
|
|
||||||
**Entry points, not a barrel.** Because the public surface is *every* root file, a package can expose several small entry points (`index.ts`, `client.ts`, `server.ts`) instead of funnelling everything through one giant `index.ts`. Barrel files that re-export a whole subtree are discouraged — keep entry points small and hide implementation in subfolders.
|
|
||||||
|
|
||||||
Layering (which packages may depend on which) is a *different* concern and is left as a commented stub in the config for this repo to fill in.
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Detect the environment
|
|
||||||
|
|
||||||
- **Package manager** — `pnpm-lock.yaml` → pnpm, `yarn.lock` → yarn, `bun.lockb` → bun, else npm. Use it for every command below (`pnpm`/`yarn`/`npm run`/`bunx`).
|
|
||||||
- **Packages root** — if `src/` exists use `src/packages`, else `packages`. Confirm the choice with the user if the repo already has a different obvious convention.
|
|
||||||
- **Existing config** — check for a `.dependency-cruiser.*` file. If one exists, do **not** overwrite it: merge the four rules and the options in, and tell the user what you added.
|
|
||||||
|
|
||||||
**Done when:** package manager, packages root, and existing-config status are all known.
|
|
||||||
|
|
||||||
### 2. Install dependency-cruiser
|
|
||||||
|
|
||||||
Install `dependency-cruiser` as a devDependency with the detected package manager.
|
|
||||||
|
|
||||||
**Done when:** `dependency-cruiser` is in `devDependencies`.
|
|
||||||
|
|
||||||
### 3. Write the config
|
|
||||||
|
|
||||||
Copy [`dependency-cruiser.config.cjs`](./dependency-cruiser.config.cjs) to the repo root as `.dependency-cruiser.cjs`. Set `PACKAGES_ROOT` to the root detected in step 1. The rules are path-depth based and extension-agnostic, so nothing else needs adapting.
|
|
||||||
|
|
||||||
**Done when:** `.dependency-cruiser.cjs` exists with the correct `PACKAGES_ROOT`, and the four forbidden rules are present.
|
|
||||||
|
|
||||||
### 4. Wire it into the checks
|
|
||||||
|
|
||||||
- Add a `lint:boundaries` script: `depcruise <packages-root>` (or `depcruise src`).
|
|
||||||
- Fold it into the repo's umbrella check command — the one that already runs typecheck (e.g. a `check` / `ci` / `validate` script). Do **not** touch `tsconfig` or add path aliases.
|
|
||||||
- If there is no umbrella script, add `lint:boundaries` and tell the user to include it in CI.
|
|
||||||
|
|
||||||
**Done when:** `lint:boundaries` exists and runs as part of the same command as typecheck.
|
|
||||||
|
|
||||||
### 5. Scaffold the example package
|
|
||||||
|
|
||||||
Create a committed `<packages-root>/example/` as a copy-me template:
|
|
||||||
|
|
||||||
- `index.ts` — an entry point. Export one function that delegates to an internal file (so the package is visibly *deep*, not a pass-through).
|
|
||||||
- `lib/impl.ts` — an internal file in a **subfolder**, imported by `index.ts`, not reachable from outside.
|
|
||||||
- `tests/example.test.ts` — imports **only** `../index` (an entry point), and asserts against the public function.
|
|
||||||
|
|
||||||
Tell the user this is a starter template to copy or delete.
|
|
||||||
|
|
||||||
**Done when:** the example package exists, exposes its behaviour through a root entry point, and hides `impl` in a subfolder.
|
|
||||||
|
|
||||||
### 6. Prove the rules bite
|
|
||||||
|
|
||||||
This is the completion criterion for the whole skill — a config that doesn't fail on a violation is worthless.
|
|
||||||
|
|
||||||
1. Run `lint:boundaries`. It must **pass** on the clean example.
|
|
||||||
2. Temporarily add a deep import to `tests/example.test.ts` (e.g. `import { thing } from "../lib/impl"`). Run `lint:boundaries` again — it must **fail** with `tests-through-entrypoints`.
|
|
||||||
3. Revert the deep import. Run once more — it must **pass**.
|
|
||||||
|
|
||||||
**Done when:** you have observed a pass, then a fail on the deep import, then a pass again. If step 2 does not fail, the rules are not wired correctly — fix before finishing.
|
|
||||||
|
|
||||||
### 7. Document the convention
|
|
||||||
|
|
||||||
Write a `README.md` **in the packages folder** (`<packages-root>/README.md`) — next to the packages it governs — covering: the `src/packages/<name>/` layout (entry points at the root, `lib/` for implementation, `tests/` for tests), "import only through a package's entry points (its root files)", and how to run `lint:boundaries`. **Discourage barrel files** explicitly — expose several small entry points instead of re-exporting a whole subtree through one index. Keep it to the copy-me snippet plus the four rules in one paragraph each.
|
|
||||||
|
|
||||||
Then add a **context pointer** to it from the repo's agent-instructions file — `CLAUDE.md` if present, else `AGENTS.md` (create `AGENTS.md` if neither exists). One line is enough, e.g. `Packages are deep modules — see [src/packages/README.md](./src/packages/README.md) before adding or importing one.` This is what makes an agent discover the boundary rule instead of tripping over it.
|
|
||||||
|
|
||||||
**Done when:** `<packages-root>/README.md` exists and discourages barrels, and the repo's `CLAUDE.md`/`AGENTS.md` links to it.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- The config's `$1` back-references (dependency-cruiser's group matching) are what let a package reach its own internals while outsiders can't — don't flatten them into separate per-package rules.
|
|
||||||
- Public vs private is decided by **depth**: a package's root files are entry points; anything in a subfolder is private. The conventional subfolders are `lib/` (implementation) and `tests/`, but the rule doesn't hardcode them — any subfolder is private, so a new folder never needs a config change. Adding an entry point is just adding a root file — no barrel.
|
|
||||||
- Packages are **flat**: one tier of immediate children under the root. A package's internals may nest as deep as you like; a package may not contain another package.
|
|
||||||
- Use `.cjs` (not `.js`) so the config's `module.exports` works even in `"type": "module"` repos.
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Setup TS Deep Modules"
|
|
||||||
short_description: "Enforce deep TypeScript modules"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -1,95 +0,0 @@
|
|||||||
// @ts-check
|
|
||||||
// Deep-module enforcement for dependency-cruiser.
|
|
||||||
//
|
|
||||||
// Each package under the packages root is a DEEP MODULE: a lot of behaviour
|
|
||||||
// behind a small interface. A package's PUBLIC SURFACE is its ENTRY POINTS —
|
|
||||||
// the files at the package root. Implementation lives in SUBFOLDERS and is
|
|
||||||
// private — by convention `lib/` for implementation and `tests/` for tests,
|
|
||||||
// though any subfolder is private. A package may expose several small entry
|
|
||||||
// points (index.ts, client.ts, server.ts, …) — prefer that over one giant
|
|
||||||
// barrel index.
|
|
||||||
//
|
|
||||||
// The only thing you should ever need to edit here is PACKAGES_ROOT.
|
|
||||||
|
|
||||||
/** Where packages live. One immediate child dir per package (flat, no nesting). */
|
|
||||||
const PACKAGES_ROOT = "src/packages";
|
|
||||||
|
|
||||||
// --- derived patterns (no need to edit) -------------------------------------
|
|
||||||
const R = PACKAGES_ROOT;
|
|
||||||
/**
|
|
||||||
* A package's private internals: anything nested inside a package subfolder.
|
|
||||||
* The package's root files are its entry points and are NOT matched here —
|
|
||||||
* they stay importable from outside.
|
|
||||||
*/
|
|
||||||
const PACKAGE_INTERNALS = `^${R}/[^/]+/[^/]+/`;
|
|
||||||
|
|
||||||
/** @type {import('dependency-cruiser').IConfiguration} */
|
|
||||||
module.exports = {
|
|
||||||
forbidden: [
|
|
||||||
{
|
|
||||||
name: "entrypoint-boundary-from-app",
|
|
||||||
comment:
|
|
||||||
"App/root code may import a package's entry points (its root files), but nothing inside its subfolders.",
|
|
||||||
severity: "error",
|
|
||||||
from: { pathNot: `^${R}/` }, // importer is NOT inside any package
|
|
||||||
to: { path: PACKAGE_INTERNALS },
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "entrypoint-boundary-across-packages",
|
|
||||||
comment:
|
|
||||||
"A package's own files import each other freely, but may reach OTHER packages only through their entry points — never their internals.",
|
|
||||||
severity: "error",
|
|
||||||
// importer is inside a package ($1), but is not a test file
|
|
||||||
from: { path: `^${R}/([^/]+)/`, pathNot: `^${R}/[^/]+/tests/` },
|
|
||||||
to: {
|
|
||||||
path: PACKAGE_INTERNALS,
|
|
||||||
pathNot: `^${R}/$1/`, // same package → intra-package freedom
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "tests-through-entrypoints",
|
|
||||||
comment:
|
|
||||||
"A package's tests exercise it through its entry points like everyone else: they may import any package's entry points and their own tests/ fixtures, but never any package's internals — not even their own.",
|
|
||||||
severity: "error",
|
|
||||||
from: { path: `^${R}/([^/]+)/tests/` }, // a test file, in package $1
|
|
||||||
to: {
|
|
||||||
path: PACKAGE_INTERNALS,
|
|
||||||
pathNot: `^${R}/$1/tests/`, // own tests/ fixtures → allowed
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "tests-folder-is-private",
|
|
||||||
comment:
|
|
||||||
"A package's tests/ folder is reachable only from tests — nothing else may import fixtures.",
|
|
||||||
severity: "error",
|
|
||||||
from: { pathNot: `^${R}/[^/]+/tests/` }, // importer is not itself a test
|
|
||||||
to: { path: `^${R}/[^/]+/tests/` },
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "no-circular",
|
|
||||||
comment: "No dependency cycles. Scope to `^${R}/` if you want to allow cycles outside packages.",
|
|
||||||
severity: "error",
|
|
||||||
from: {},
|
|
||||||
to: { circular: true },
|
|
||||||
},
|
|
||||||
|
|
||||||
// --- Layering (optional, off by default) ----------------------------------
|
|
||||||
// Interface-hiding controls HOW you import (through the entry points).
|
|
||||||
// Layering controls WHICH packages may depend on which. Add your own rules
|
|
||||||
// here, e.g.:
|
|
||||||
//
|
|
||||||
// {
|
|
||||||
// name: "ui-may-not-depend-on-billing",
|
|
||||||
// severity: "error",
|
|
||||||
// from: { path: `^${R}/ui/` },
|
|
||||||
// to: { path: `^${R}/billing/` },
|
|
||||||
// },
|
|
||||||
],
|
|
||||||
options: {
|
|
||||||
doNotFollow: { path: "node_modules" },
|
|
||||||
tsConfig: { fileName: "tsconfig.json" },
|
|
||||||
enhancedResolveOptions: {
|
|
||||||
extensions: [".ts", ".tsx", ".js", ".jsx", ".json"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
@@ -5,32 +5,104 @@ description: Test-driven development. Use when the user wants to build features
|
|||||||
|
|
||||||
# Test-Driven Development
|
# Test-Driven Development
|
||||||
|
|
||||||
TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle — consult them before and during the loop, not after.
|
## Philosophy
|
||||||
|
|
||||||
When exploring the codebase, read `CONTEXT.md` (if it exists) so test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
|
**Core principle**: Tests should verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't.
|
||||||
|
|
||||||
## What a good test is
|
**Good tests** are integration-style: they exercise real code paths through public APIs. They describe _what_ the system does, not _how_ it does it. A good test reads like a specification - "user can checkout with valid cart" tells you exactly what capability exists. These tests survive refactors because they don't care about internal structure.
|
||||||
|
|
||||||
Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification — "user can checkout with valid cart" tells you exactly what capability exists — and survives refactors because it doesn't care about internal structure.
|
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
|
||||||
|
|
||||||
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
|
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
|
||||||
|
|
||||||
## Seams — where tests go
|
## Anti-Pattern: Horizontal Slices
|
||||||
|
|
||||||
A **seam** is the public boundary you test at: the interface where you observe behavior without reaching inside. Tests live at seams, never against internals.
|
**DO NOT write all tests first, then all implementation.** This is "horizontal slicing" - treating RED as "write all tests" and GREEN as "write all code."
|
||||||
|
|
||||||
**Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. No test is written at an unconfirmed seam. You can't test everything — agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case.
|
This produces **crap tests**:
|
||||||
|
|
||||||
Ask: "What's the public interface, and which seams should we test?"
|
- Tests written in bulk test _imagined_ behavior, not _actual_ behavior
|
||||||
|
- You end up testing the _shape_ of things (data structures, function signatures) rather than user-facing behavior
|
||||||
|
- Tests become insensitive to real changes - they pass when behavior breaks, fail when behavior is fine
|
||||||
|
- You outrun your headlights, committing to test structure before understanding the implementation
|
||||||
|
|
||||||
## Anti-patterns
|
**Correct approach**: Vertical slices via tracer bullets. One test → one implementation → repeat. Each test responds to what you learned from the previous cycle. Because you just wrote the code, you know exactly what behavior matters and how to verify it.
|
||||||
|
|
||||||
- **Implementation-coupled** — mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
|
```
|
||||||
- **Tautological** — the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth — a known-good literal, a worked example, the spec.
|
WRONG (horizontal):
|
||||||
- **Horizontal slicing** — writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead — one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.
|
RED: test1, test2, test3, test4, test5
|
||||||
|
GREEN: impl1, impl2, impl3, impl4, impl5
|
||||||
|
|
||||||
## Rules of the loop
|
RIGHT (vertical):
|
||||||
|
RED→GREEN: test1→impl1
|
||||||
|
RED→GREEN: test2→impl2
|
||||||
|
RED→GREEN: test3→impl3
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
- **Red before green.** Write the failing test first, then only enough code to pass it. Don't anticipate future tests or add speculative features.
|
## Workflow
|
||||||
- **One slice at a time.** One seam, one test, one minimal implementation per cycle.
|
|
||||||
- **Refactoring is not part of the loop.** It belongs to the review stage (see the `code-review` skill), not the red → green implementation cycle.
|
### 1. Planning
|
||||||
|
|
||||||
|
When exploring the codebase, read `CONTEXT.md` (if it exists) so that test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
|
||||||
|
|
||||||
|
Before writing any code:
|
||||||
|
|
||||||
|
- [ ] Confirm with user what interface changes are needed
|
||||||
|
- [ ] Confirm with user which behaviors to test (prioritize)
|
||||||
|
- [ ] Identify opportunities for deep modules (small interface, deep implementation) — run the `/codebase-design` skill for the vocabulary and the testability checks
|
||||||
|
- [ ] List the behaviors to test (not implementation steps)
|
||||||
|
- [ ] Get user approval on the plan
|
||||||
|
|
||||||
|
Ask: "What should the public interface look like? Which behaviors are most important to test?"
|
||||||
|
|
||||||
|
**You can't test everything.** Confirm with the user exactly which behaviors matter most. Focus testing effort on critical paths and complex logic, not every possible edge case.
|
||||||
|
|
||||||
|
### 2. Tracer Bullet
|
||||||
|
|
||||||
|
Write ONE test that confirms ONE thing about the system:
|
||||||
|
|
||||||
|
```
|
||||||
|
RED: Write test for first behavior → test fails
|
||||||
|
GREEN: Write minimal code to pass → test passes
|
||||||
|
```
|
||||||
|
|
||||||
|
This is your tracer bullet - proves the path works end-to-end.
|
||||||
|
|
||||||
|
### 3. Incremental Loop
|
||||||
|
|
||||||
|
For each remaining behavior:
|
||||||
|
|
||||||
|
```
|
||||||
|
RED: Write next test → fails
|
||||||
|
GREEN: Minimal code to pass → passes
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- One test at a time
|
||||||
|
- Only enough code to pass current test
|
||||||
|
- Don't anticipate future tests
|
||||||
|
- Keep tests focused on observable behavior
|
||||||
|
|
||||||
|
### 4. Refactor
|
||||||
|
|
||||||
|
After all tests pass, look for [refactor candidates](refactoring.md):
|
||||||
|
|
||||||
|
- [ ] Extract duplication
|
||||||
|
- [ ] Deepen modules (move complexity behind simple interfaces)
|
||||||
|
- [ ] Apply SOLID principles where natural
|
||||||
|
- [ ] Consider what new code reveals about existing code
|
||||||
|
- [ ] Run tests after each refactor step
|
||||||
|
|
||||||
|
**Never refactor while RED.** Get to GREEN first.
|
||||||
|
|
||||||
|
## Checklist Per Cycle
|
||||||
|
|
||||||
|
```
|
||||||
|
[ ] Test describes behavior, not implementation
|
||||||
|
[ ] Test uses public interface only
|
||||||
|
[ ] Test would survive internal refactor
|
||||||
|
[ ] Code is minimal for this test
|
||||||
|
[ ] No speculative features added
|
||||||
|
```
|
||||||
|
|||||||
@@ -1,3 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "TDD"
|
|
||||||
short_description: "Test-driven red-green-refactor"
|
|
||||||
@@ -59,19 +59,3 @@ test("createUser makes user retrievable", async () => {
|
|||||||
expect(retrieved.name).toBe("Alice");
|
expect(retrieved.name).toBe("Alice");
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// BAD: Expected value is recomputed the way the code computes it
|
|
||||||
test("calculateTotal sums line items", () => {
|
|
||||||
const items = [{ price: 10 }, { price: 5 }];
|
|
||||||
const expected = items.reduce((sum, i) => sum + i.price, 0);
|
|
||||||
expect(calculateTotal(items)).toBe(expected);
|
|
||||||
});
|
|
||||||
|
|
||||||
// GOOD: Expected value is an independent, known literal
|
|
||||||
test("calculateTotal sums line items", () => {
|
|
||||||
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -1,75 +0,0 @@
|
|||||||
---
|
|
||||||
name: to-spec
|
|
||||||
description: Turn the current conversation into a spec and publish it to the project issue tracker — no interview, just synthesis of what you've already discussed.
|
|
||||||
disable-model-invocation: true
|
|
||||||
---
|
|
||||||
|
|
||||||
This skill takes the current conversation context and codebase understanding and produces a spec (you may know this document as a PRD). Do NOT interview the user — just synthesize what you already know.
|
|
||||||
|
|
||||||
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
|
|
||||||
|
|
||||||
## Process
|
|
||||||
|
|
||||||
1. Explore the repo to understand the current state of the codebase, if you haven't already. Use the project's domain glossary vocabulary throughout the spec, and respect any ADRs in the area you're touching.
|
|
||||||
|
|
||||||
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can. The fewer seams across the codebase, the better - the ideal number is one.
|
|
||||||
|
|
||||||
Check with the user that these seams match their expectations.
|
|
||||||
|
|
||||||
3. Write the spec using the template below, then publish it to the project issue tracker. Apply the `ready-for-agent` triage label - no need for additional triage.
|
|
||||||
|
|
||||||
<spec-template>
|
|
||||||
|
|
||||||
## Problem Statement
|
|
||||||
|
|
||||||
The problem that the user is facing, from the user's perspective.
|
|
||||||
|
|
||||||
## Solution
|
|
||||||
|
|
||||||
The solution to the problem, from the user's perspective.
|
|
||||||
|
|
||||||
## User Stories
|
|
||||||
|
|
||||||
A LONG, numbered list of user stories. Each user story should be in the format of:
|
|
||||||
|
|
||||||
1. As an <actor>, I want a <feature>, so that <benefit>
|
|
||||||
|
|
||||||
<user-story-example>
|
|
||||||
1. As a mobile bank customer, I want to see balance on my accounts, so that I can make better informed decisions about my spending
|
|
||||||
</user-story-example>
|
|
||||||
|
|
||||||
This list of user stories should be extremely extensive and cover all aspects of the feature.
|
|
||||||
|
|
||||||
## Implementation Decisions
|
|
||||||
|
|
||||||
A list of implementation decisions that were made. This can include:
|
|
||||||
|
|
||||||
- The modules that will be built/modified
|
|
||||||
- The interfaces of those modules that will be modified
|
|
||||||
- Technical clarifications from the developer
|
|
||||||
- Architectural decisions
|
|
||||||
- Schema changes
|
|
||||||
- API contracts
|
|
||||||
- Specific interactions
|
|
||||||
|
|
||||||
Do NOT include specific file paths or code snippets. They may end up being outdated very quickly.
|
|
||||||
|
|
||||||
Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it within the relevant decision and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
|
|
||||||
|
|
||||||
## Testing Decisions
|
|
||||||
|
|
||||||
A list of testing decisions that were made. Include:
|
|
||||||
|
|
||||||
- A description of what makes a good test (only test external behavior, not implementation details)
|
|
||||||
- Which modules will be tested
|
|
||||||
- Prior art for the tests (i.e. similar types of tests in the codebase)
|
|
||||||
|
|
||||||
## Out of Scope
|
|
||||||
|
|
||||||
A description of the things that are out of scope for this spec.
|
|
||||||
|
|
||||||
## Further Notes
|
|
||||||
|
|
||||||
Any further notes about the feature.
|
|
||||||
|
|
||||||
</spec-template>
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "To Spec"
|
|
||||||
short_description: "Turn a conversation into a spec"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
---
|
|
||||||
name: to-tickets
|
|
||||||
description: Break a plan, spec, or the current conversation into a set of tracer-bullet tickets, each declaring its blocking edges, published to the configured tracker — edges as text in one file per ticket locally, or native blocking links on a real tracker.
|
|
||||||
disable-model-invocation: true
|
|
||||||
---
|
|
||||||
|
|
||||||
# To Tickets
|
|
||||||
|
|
||||||
Break a plan, spec, or conversation into a set of **tickets** — tracer-bullet vertical slices, each declaring the tickets that **block** it.
|
|
||||||
|
|
||||||
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
|
|
||||||
|
|
||||||
## Process
|
|
||||||
|
|
||||||
### 1. Gather context
|
|
||||||
|
|
||||||
Work from whatever is already in the conversation context. If the user passes a reference (a spec path, an issue number or URL) as an argument, fetch it and read its full body and comments.
|
|
||||||
|
|
||||||
### 2. Explore the codebase (optional)
|
|
||||||
|
|
||||||
If you have not already explored the codebase, do so to understand the current state of the code. Ticket titles and descriptions should use the project's domain glossary vocabulary, and respect ADRs in the area you're touching.
|
|
||||||
|
|
||||||
Look for opportunities to prefactor the code to make the implementation easier. "Make the change easy, then make the easy change."
|
|
||||||
|
|
||||||
### 3. Draft vertical slices
|
|
||||||
|
|
||||||
Break the work into **tracer bullet** tickets.
|
|
||||||
|
|
||||||
<vertical-slice-rules>
|
|
||||||
|
|
||||||
- Each slice cuts a narrow but COMPLETE path through every layer (schema, API, UI, tests) — vertical, NOT a horizontal slice of one layer
|
|
||||||
- A completed slice is demoable or verifiable on its own
|
|
||||||
- Each slice is sized to fit in a single fresh context window
|
|
||||||
- Any prefactoring should be done first
|
|
||||||
|
|
||||||
</vertical-slice-rules>
|
|
||||||
|
|
||||||
Give each ticket its **blocking edges** — the other tickets that must complete before it can start. A ticket with no blockers can start immediately.
|
|
||||||
|
|
||||||
**Wide refactors are the exception to vertical slicing.** A **wide refactor** is one mechanical change — rename a column, retype a shared symbol — whose **blast radius** fans across the whole codebase, so a single edit breaks thousands of call sites at once and no vertical slice can land green. Don't force it into a tracer bullet; sequence it as **expand–contract**. First expand: add the new form beside the old so nothing breaks. Then migrate the call sites over in batches sized by blast radius (per package, per directory), each batch its own ticket blocked by the expand, keeping CI green batch to batch because the old form still exists. Finally contract: delete the old form once no caller remains, in a ticket blocked by every migrate batch. When even the batches can't stay green alone, keep the sequence but let them share an integration branch that all block a final integrate-and-verify ticket — green is promised only there.
|
|
||||||
|
|
||||||
### 4. Quiz the user
|
|
||||||
|
|
||||||
Present the proposed breakdown as a numbered list. For each ticket, show:
|
|
||||||
|
|
||||||
- **Title**: short descriptive name
|
|
||||||
- **Blocked by**: which other tickets (if any) must complete first
|
|
||||||
- **What it delivers**: the end-to-end behaviour this ticket makes work
|
|
||||||
|
|
||||||
Ask the user:
|
|
||||||
|
|
||||||
- Does the granularity feel right? (too coarse / too fine)
|
|
||||||
- Are the blocking edges correct — does each ticket only depend on tickets that genuinely gate it?
|
|
||||||
- Should any tickets be merged or split further?
|
|
||||||
|
|
||||||
Iterate until the user approves the breakdown.
|
|
||||||
|
|
||||||
### 5. Publish the tickets to the configured tracker
|
|
||||||
|
|
||||||
Publish the approved tickets. **How** depends on the tracker `/setup-matt-pocock-skills` configured — the tickets are the same either way, only the shape of the blocking edges changes:
|
|
||||||
|
|
||||||
- **Local files** → write one file per ticket under `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` in dependency order (blockers first). Each file's "Blocked by" lists the numbers/titles it depends on. Use the per-ticket file template below — one ticket per file, never a single combined file.
|
|
||||||
- **A real issue tracker (GitHub, Linear, …)** → publish one issue per ticket in dependency order (blockers first) so each ticket's blocking edges can reference real identifiers. Use the platform's native blocking / sub-issue relationship where it has one; otherwise set each ticket's "Blocked by" to the blocking issues. Apply the `ready-for-agent` triage label unless instructed otherwise — the tickets are agent-grabbable by construction.
|
|
||||||
|
|
||||||
Work the **frontier**: any ticket whose blockers are all done. For a purely linear chain that means top to bottom.
|
|
||||||
|
|
||||||
Do NOT close or modify any parent issue.
|
|
||||||
|
|
||||||
<local-ticket-template>
|
|
||||||
|
|
||||||
# <NN> — <Ticket title>
|
|
||||||
|
|
||||||
**What to build:** the end-to-end behaviour this ticket makes work, from the user's perspective — not a layer-by-layer implementation list.
|
|
||||||
|
|
||||||
**Blocked by:** the numbers/titles of the tickets that gate this one, or "None — can start immediately".
|
|
||||||
|
|
||||||
**Status:** ready-for-agent
|
|
||||||
|
|
||||||
- [ ] Acceptance criterion 1
|
|
||||||
- [ ] Acceptance criterion 2
|
|
||||||
|
|
||||||
</local-ticket-template>
|
|
||||||
|
|
||||||
<issue-template>
|
|
||||||
|
|
||||||
## Parent
|
|
||||||
|
|
||||||
A reference to the parent issue on the tracker (if the source was an existing issue, otherwise omit this section).
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The end-to-end behaviour this ticket makes work, from the user's perspective — not layer-by-layer implementation.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [ ] Criterion 1
|
|
||||||
- [ ] Criterion 2
|
|
||||||
|
|
||||||
## Blocked by
|
|
||||||
|
|
||||||
- A reference to each blocking ticket, or "None — can start immediately".
|
|
||||||
|
|
||||||
</issue-template>
|
|
||||||
|
|
||||||
In either form, avoid specific file paths or code snippets — they go stale fast. Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
|
|
||||||
|
|
||||||
Work the frontier one ticket at a time with `/implement`, clearing context between tickets.
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "To Tickets"
|
|
||||||
short_description: "Split a plan into tracer-bullet tickets"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -1,16 +1,9 @@
|
|||||||
---
|
---
|
||||||
name: wayfinder
|
name: wayfinder
|
||||||
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of decision tickets on your issue tracker, and resolve them one at a time until the way to the destination is clear.
|
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of investigation tickets on your issue tracker, and resolve them one at a time until the way to the goal is clear.
|
||||||
disable-model-invocation: true
|
|
||||||
---
|
---
|
||||||
|
|
||||||
A loose idea has arrived — too big for one agent session, and wrapped in fog: the way from here to the **destination** isn't visible yet. Wayfinding is about finding that way, not charging at the destination. This skill charts the way as a **shared map** on the repo's issue tracker, then works its **decision tickets** — questions whose resolution is a decision, not slices of a build to execute — one at a time until the route is clear.
|
A loose idea has arrived — too big for one agent session, and wrapped in fog: the route from here to a plan isn't visible yet. This skill charts it as a **shared map** on the repo's issue tracker, then works its tickets one at a time. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
|
||||||
|
|
||||||
The destination varies per effort, and naming it is the first act of charting — it shapes every ticket. It might be a spec to hand off and iterate on, a decision to lock before planning starts, or a change made in place like a data-structure migration. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
|
|
||||||
|
|
||||||
## Plan, don't do
|
|
||||||
|
|
||||||
Wayfinder is **planning** by default: each ticket resolves a decision, and the map is done when the way is clear — nothing left to decide before someone goes and does the thing. The pull to just do the work is usually the signal you've reached the edge of the map and it's time to hand off. An effort can override this in its **Notes** — carrying execution into the map itself — but absent that, produce decisions, not deliverables.
|
|
||||||
|
|
||||||
## Refer by name
|
## Refer by name
|
||||||
|
|
||||||
@@ -22,17 +15,13 @@ The map is a single issue on this repo's issue tracker, labelled `wayfinder:map`
|
|||||||
|
|
||||||
The map is an **index**, not a store. It lists the decisions made and points at the tickets that hold their detail; a decision lives in exactly one place — its ticket — so the map never restates it, only gists it and links.
|
The map is an **index**, not a store. It lists the decisions made and points at the tickets that hold their detail; a decision lives in exactly one place — its ticket — so the map never restates it, only gists it and links.
|
||||||
|
|
||||||
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** The issue tracker should have been provided to you — run `/setup-matt-pocock-skills` if not. Consult the tracker doc's "Wayfinding operations" section for how _this_ repo expresses them. If no tracker has been provided, default to the local-markdown tracker.
|
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** Consult `docs/agents/issue-tracker.md` (the "Wayfinding operations" section) for how _this_ repo expresses them. If that doc is absent, default to the local-markdown tracker.
|
||||||
|
|
||||||
### The map body
|
### The map body
|
||||||
|
|
||||||
The whole map at low resolution, loaded once per session. Open tickets are **not** listed — they are open child issues, found by query.
|
The whole map at low resolution, loaded once per session. Open tickets are **not** listed — they are open child issues, found by query.
|
||||||
|
|
||||||
```markdown
|
```markdown
|
||||||
## Destination
|
|
||||||
|
|
||||||
<what reaching the end of this map looks like — the spec, decision, or change this effort is finding its way to. One or two lines; every session orients to it before choosing a ticket.>
|
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
<domain; skills every session should consult; standing preferences for this effort>
|
<domain; skills every session should consult; standing preferences for this effort>
|
||||||
@@ -43,13 +32,9 @@ The whole map at low resolution, loaded once per session. Open tickets are **not
|
|||||||
|
|
||||||
- [<closed ticket title>](link) — <one-line gist of the answer>
|
- [<closed ticket title>](link) — <one-line gist of the answer>
|
||||||
|
|
||||||
## Not yet specified
|
## Fog
|
||||||
|
|
||||||
<!-- see "Fog of war": in-scope fog you can't ticket yet; graduates as the frontier advances -->
|
<!-- see "Fog of war" for what belongs here -->
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
<!-- see "Out of scope": work ruled beyond the destination; closed, never graduates -->
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Tickets
|
### Tickets
|
||||||
@@ -72,48 +57,36 @@ The answer isn't part of the body — it's recorded on resolution (see [Work thr
|
|||||||
|
|
||||||
## Ticket Types
|
## Ticket Types
|
||||||
|
|
||||||
Every ticket is either **HITL** — human in the loop, worked *with* a human who speaks for themselves — or **AFK**, driven by the agent alone. A HITL ticket only resolves through that live exchange; the agent never stands in for the human's side of it (a grilling agent that answers its own questions has broken this).
|
- **Research**: Reading documentation, third-party APIs, or local resources like knowledge bases. Creates a markdown summary as a linked asset. Use when knowledge outside the current working directory is required.
|
||||||
|
- **Prototype**: Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
|
||||||
- **Research** (AFK): Reading documentation, third-party APIs, or local resources like knowledge bases to surface a fact a decision waits on. Resolved by a `/research` **subagent**. Use when knowledge outside the current working directory is required.
|
- **Grilling**: Conversation with the agent. Uses the /grilling and /domain-modeling skills. Asks one question at a time. The default case.
|
||||||
- **Prototype** (HITL): Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
|
- **Task**: Literal manual work that must be done before the discussion can move forward — nothing to decide, prototype, or research. Moving data, signing up for a service, provisioning access. The agent automates it where it can; otherwise it hands the human a precise checklist. Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
|
||||||
- **Grilling** (HITL): Conversation via the /grilling and /domain-modeling skills, one question at a time. The default case.
|
|
||||||
- **Task** (HITL or AFK): Manual work that must happen before a *decision* can be made — nothing to decide, prototype, or research, but the discussion is blocked until it's done. Signing up for a service so its API can be judged, provisioning access, moving data so its shape can be seen. This is the one type that *does* rather than decides — and it earns its place by unblocking a decision, not by delivering the destination. The agent drives it alone where it can (AFK); otherwise it hands the human a precise checklist (HITL). Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
|
|
||||||
|
|
||||||
## Fog of war
|
## Fog of war
|
||||||
|
|
||||||
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the live tickets lies the **fog of war** — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the destination is clear and no tickets remain.
|
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the tickets lies fog — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the goal is clear and no tickets remain.
|
||||||
|
|
||||||
The map's **Not yet specified** section is where that dim view is written down: the suspected question, the area to revisit later. It's the undiscovered frontier _toward_ the destination — everything here is in scope, just not sharp enough to ticket. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
|
The map's **Fog** section is where that dim view is written down: the suspected question, the area to revisit later, the risk you're deferring. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
|
||||||
|
|
||||||
**Fog or ticket?** The test is whether you can state the question precisely now — _not_ whether you can answer it now.
|
**Fog or ticket?** The test is whether you can state the question precisely now — _not_ whether you can answer it now.
|
||||||
|
|
||||||
- **Ticket when** the question is already sharp — even if it's blocked and you can't act on it yet.
|
- **Ticket when** the question is already sharp — even if it's blocked and you can't act on it yet.
|
||||||
- **Not yet specified when** you can't yet phrase it that sharply. Don't pre-slice the fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
|
- **Fog when** you can't yet phrase it that sharply. Don't pre-slice fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
|
||||||
|
|
||||||
**Not yet specified** excludes what's already decided (Decisions so far), what's already a live ticket, and what's out of scope (the next section).
|
Fog excludes only what's already decided (that's Decisions so far) and what's already a ticket.
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
Fog only ever gathers _toward_ the destination. The destination fixes the scope, so work beyond it is **out of scope** — it isn't fog, and it doesn't belong in **Not yet specified**. It gets its own **Out of scope** section on the map: work you've consciously ruled out of _this_ effort. Scope, not sharpness, lands it here.
|
|
||||||
|
|
||||||
Out-of-scope work never graduates — the frontier stops at the destination — so it returns only if the destination is redrawn, and then as a fresh effort, not a resumption.
|
|
||||||
|
|
||||||
Ruling something out of scope is a scoping act, not a step on the route. When a ticket that already exists turns out to sit past the destination — mis-scoped in while charting, or exposed by a resolution — **close it** (a closed ticket is unambiguously off the frontier) and leave one line in the **Out of scope** section: the gist plus why it's out of scope, linking the closed ticket. It stays out of **Decisions so far**, which records the route actually walked — a scope boundary isn't a step on it.
|
|
||||||
|
|
||||||
## Invocation
|
## Invocation
|
||||||
|
|
||||||
Two modes. Either way, **never resolve more than one ticket per session** — with the exception of research tickets.
|
Two modes. Either way, **never resolve more than one ticket per session.**
|
||||||
|
|
||||||
### Chart the map
|
### Chart the map
|
||||||
|
|
||||||
User invokes with a loose idea.
|
User invokes with a loose idea.
|
||||||
|
|
||||||
1. **Name the destination.** Run a `/grilling` and `/domain-modeling` session to pin down what this map is finding its way to — the spec, decision, or change. The destination fixes the scope, so it's settled first.
|
1. Run a `/grilling` and `/domain-modeling` session to surface the open decisions.
|
||||||
2. **Map the frontier.** Grill again, **breadth-first** this time: fan out across the whole space rather than deep on any one thread, surfacing the open decisions and the first steps takeable now. **If this surfaces no fog** — the way to the destination is already clear, the whole journey small enough for one session — you don't need a map. Stop and ask the user how they'd like to proceed.
|
2. **Create the map** (label `wayfinder:map`): Notes filled in, Decisions-so-far empty, Fog sketched.
|
||||||
3. **Create the map** (label `wayfinder:map`): Destination and Notes filled in, Decisions-so-far empty, the fog sketched into **Not yet specified**.
|
3. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the Fog.
|
||||||
4. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the fog — the **Not yet specified** section.
|
4. Stop — charting the map is one session's work; do not also resolve tickets.
|
||||||
5. **Fire the research subagents.** For each `research` ticket you just created, spin up a `/research` subagent to resolve it in parallel, capturing its findings on a throwaway `research/<name>` branch with a context pointer from the ticket.
|
|
||||||
6. Stop — charting is one session's work; it hand-resolves nothing.
|
|
||||||
|
|
||||||
### Work through the map
|
### Work through the map
|
||||||
|
|
||||||
@@ -123,6 +96,6 @@ User invokes with a map (URL or number). A ticket is **optional** — without on
|
|||||||
2. Choose the ticket. If the user named one, use it. Otherwise take the first frontier ticket in order. **Claim it**: assign it to yourself before any work.
|
2. Choose the ticket. If the user named one, use it. Otherwise take the first frontier ticket in order. **Claim it**: assign it to yourself before any work.
|
||||||
3. Resolve it — **zoom as needed**: fetch the full body of any related or closed ticket on demand; invoke the skills the `## Notes` block names. If in doubt, use `/grilling` and `/domain-modeling`.
|
3. Resolve it — **zoom as needed**: fetch the full body of any related or closed ticket on demand; invoke the skills the `## Notes` block names. If in doubt, use `/grilling` and `/domain-modeling`.
|
||||||
4. Record the resolution: post the answer as a **resolution comment**, **close** the issue, and **append a context pointer** to the map's Decisions-so-far.
|
4. Record the resolution: post the answer as a **resolution comment**, **close** the issue, and **append a context pointer** to the map's Decisions-so-far.
|
||||||
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from **Not yet specified** so it lives only as its new ticket. If the answer reveals a ticket — this one or another — sits beyond the destination, **rule it out of scope** rather than resolving it on the route. If the decision invalidates other parts of the map, update or delete those tickets.
|
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from the Fog so it lives only as its new ticket. If the decision invalidates other parts of the map, update or delete those tickets.
|
||||||
|
|
||||||
The user may run unblocked tickets in parallel, so expect other sessions to be editing the tracker concurrently.
|
The user may run unblocked tickets in parallel, so expect other sessions to be editing the tracker concurrently.
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Wayfinder"
|
|
||||||
short_description: "Map a large effort as decision tickets"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -158,12 +158,6 @@ _Failure mode._ Ending the current step before it is genuinely done, because the
|
|||||||
|
|
||||||
_Avoid_: premature closure, the rush, rushing, shortcutting
|
_Avoid_: premature closure, the rush, rushing, shortcutting
|
||||||
|
|
||||||
### Negation
|
|
||||||
|
|
||||||
_Failure mode._ Steering by prohibition — telling the agent what _not_ to do — which drags the forbidden behaviour into context and makes it _more_ available, not less. _Don't think of an elephant_, and the elephant is all there is; _never write verbose comments_, and verbosity is the pattern the agent has just read. The negation is a weak modifier the strongly-activated concept overruns, so the ban half-reads as an instruction to do the thing. Its **leading word** is the _elephant_: whatever a prohibition names into the frame. Cure: prompt the **positive** — describe the target behaviour ("write one-line comments") so the banned one is never spoken. A prohibition earns its place only as a hard guardrail on a behaviour you cannot phrase positively; even then, pair it with the positive target so attention lands on what to do.
|
|
||||||
|
|
||||||
_Avoid_: ironic rebound, don't-prompting, the pink elephant
|
|
||||||
|
|
||||||
## Pruning
|
## Pruning
|
||||||
|
|
||||||
Keeping a skill lean — each remedy paired with the failure it cures.
|
Keeping a skill lean — each remedy paired with the failure it cures.
|
||||||
|
|||||||
@@ -80,4 +80,3 @@ Use these to diagnose issues the user may be having with the skill.
|
|||||||
- **Sediment** — stale layers that settle because adding feels safe and removing feels risky. The default fate of any skill without a pruning discipline.
|
- **Sediment** — stale layers that settle because adding feels safe and removing feels risky. The default fate of any skill without a pruning discipline.
|
||||||
- **Sprawl** — a skill simply too long, even when every line is live and unique. Hurts readability and maintainability and wastes tokens. The cure is the ladder: disclose **reference** behind pointers, and split by **branch** or sequence so each path carries only what it needs.
|
- **Sprawl** — a skill simply too long, even when every line is live and unique. Hurts readability and maintainability and wastes tokens. The cure is the ladder: disclose **reference** behind pointers, and split by **branch** or sequence so each path carries only what it needs.
|
||||||
- **No-op** — a line the model already obeys by default, so you pay load to say nothing. The test: does it change behaviour versus the default? A weak leading word (_be thorough_ when the agent is already thorough-ish) is a no-op; the fix is a stronger word (_relentless_), not a different technique.
|
- **No-op** — a line the model already obeys by default, so you pay load to say nothing. The test: does it change behaviour versus the default? A weak leading word (_be thorough_ when the agent is already thorough-ish) is a no-op; the fix is a stronger word (_relentless_), not a different technique.
|
||||||
- **Negation** — steering by prohibition backfires: _don't think of an elephant_ names the elephant and makes it more available, not less. Prompt the **positive** — state the target behaviour so the banned one is never spoken; keep a prohibition only as a hard guardrail you can't phrase positively, and even then pair it with what to do instead.
|
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
interface:
|
|
||||||
display_name: "Writing Great Skills"
|
|
||||||
short_description: "Principles for predictable skills"
|
|
||||||
policy:
|
|
||||||
allow_implicit_invocation: false
|
|
||||||
@@ -2,9 +2,11 @@
|
|||||||
|
|
||||||
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
||||||
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
||||||
- [Project status](project-status.md) — US-001…US-035 done; US-036…US-050 in docs/prd.json; alpha hardening + scratch features next
|
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
|
||||||
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
|
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
|
||||||
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
|
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
|
||||||
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan; [ADR-0023](../../docs/adr/0023-model-agnostic-node-capability-admission.md), [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md))
|
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
|
||||||
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
||||||
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
||||||
|
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.
|
||||||
|
- **DGR-004 llama.cpp boundary** — `packages/node/native/llama/` locks `e920c523e3b8a0163fe498af5bf90df35ff51d25`, with a one-patch CMake marker and fail-closed clean materialize/apply/build/smoke harness. This is infrastructure only; stock GLM dense fallback remains uncertified.
|
||||||
|
|||||||
@@ -20,13 +20,13 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path.
|
|||||||
|
|
||||||
**Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above.
|
**Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above.
|
||||||
|
|
||||||
**Two new issues from this session:**
|
**Two new issues from this session, both `ready-for-agent`:**
|
||||||
- **21 — Honest-noise calibration corpus** — `Status: ready-for-human` (engineering done 2026-07-06; blocked on human fleet calibration run before mainnet launch).
|
- **21 — Honest-noise calibration corpus** (`.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`) rescoped from "prod gate" to a **hard alpha-release blocker**. Confirmed by code read: `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns bool only, no raw divergence value; fleet-dispatch exists but wrong shape (`server.py:2998-3104`, pinned routes + latency, not full-fleet + TOPLOC divergence); storage wrong shape (`registry_events` has no divergence/hardware columns). Three-part build: (1) surface raw TOPLOC distance from audit.py, (2) extend dispatch to hit every registered node with fixed prompt/seed, (3) new SQLite table keyed by node+GPU+dtype. Small-fleet exception granted (N = actual hired-VPS fleet size). Hired VPS hosts stay stake-free until this closes.
|
||||||
- **23 — Dynamic HF-benchmarked pricing** — `Status: done` (see `23-dynamic-hf-pricing_completed.md`).
|
- **23 — Dynamic HF-benchmarked pricing** (`.scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md`), high priority but not a release blocker. Pricing today is 100% static (`DEFAULT_PRICE_PER_1K_TOKENS = 0.02`, `billing.py:21`; `model_presets.json` has no per-model price). Target: 80% of cheapest comparable provider on `https://huggingface.co/inference/models` (per-provider-per-model marketplace, `?search=` query param works, no confirmed JSON API — plain scrape attempted first, escalate to headless browser only if the table isn't in raw HTML). Human-verified `hf_aliases` + `hf_verified_match_note` (params/quantization) per model, not auto-discovered matching. Reuses the `_settlement_loop` daemon-thread pattern for a daily refresh; falls back silently to the static default on any failure.
|
||||||
|
|
||||||
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
|
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
|
||||||
|
|
||||||
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (US-001…US-047; base 35/35 done, friends-test arc 36–47 open/in-progress). Alpha hardening uses `.scratch/alpha-hardening/prd.json` (AH-001…AH-023). Point Ralph at the prd.json for the branch you're running.
|
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (35/35 done) and does NOT see `.scratch/alpha-hardening/issues/`. No ralph loop is running and no `.ralph-tui/` state exists. `.scratch/alpha-hardening/prd.json` now has 23 stories (AH-001…AH-023); point Ralph at that file for the alpha-hardening branch. Do NOT use `ralph auto --parallel` on server.py-touching issues — 21 and 23 both touch `server.py`/`billing.py`/`audit.py`; if run in the same Ralph pass, run them serially, not in parallel (merge-conflict risk, same lesson as 03/04 previously).
|
||||||
|
|
||||||
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
|
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
|
||||||
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].
|
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].
|
||||||
|
|||||||
29
.claude/memory/dgr-rocm-setup.md
Normal file
29
.claude/memory/dgr-rocm-setup.md
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
# DGR ROCm and llama.cpp setup
|
||||||
|
|
||||||
|
As of 2026-07-13:
|
||||||
|
|
||||||
|
- Project ROCm runtime: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm`
|
||||||
|
- ROCm/TheRock build: `7.13.0a20260513`, target `gfx1151`
|
||||||
|
- `rocm-sdk-devel` is installed. Its expanded SDK lives under the venv at
|
||||||
|
`site-packages/_rocm_sdk_devel`.
|
||||||
|
- The wheel's redundant packaged payload was relocated to
|
||||||
|
`/home/popov/.local/share/rocm-sdk/7.13.0a20260513/rocm_sdk_devel` and symlinked
|
||||||
|
back into the venv because installing both packaged and expanded forms filled
|
||||||
|
the mounted drive. Do not reinstall it blindly; the wheel expands beyond
|
||||||
|
20 GB.
|
||||||
|
- HIP llama.cpp source: `/run/media/popov/d/DEV/llamacpp/llama.cpp`, commit
|
||||||
|
`e920c523e3b8a0163fe498af5bf90df35ff51d25` (version 9991).
|
||||||
|
- HIP build: `/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip`
|
||||||
|
- HIP `llama-server` SHA-256:
|
||||||
|
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`
|
||||||
|
- Verified device: `ROCm0: Radeon 8060S Graphics`, `gfx1151`.
|
||||||
|
- Model artifacts remain under `/run/media/popov/DATA/llm`; none were put under
|
||||||
|
`/home`.
|
||||||
|
|
||||||
|
DGR-001's immutable contract remains CPU-only. GPU evidence uses the distinct
|
||||||
|
signed `gpu-diagnostic` profile because llama-server process VRAM is not yet
|
||||||
|
measurable by the benchmark driver. The profile must capture measured
|
||||||
|
llama-server startup evidence for `ROCm0` and the actual offloaded/total layer
|
||||||
|
count; configured `device` and `n_gpu_layers` values alone are not evidence.
|
||||||
|
The accepted signer fingerprint is anchored in
|
||||||
|
`.scratch/distributed-gguf-runtime/trusted-evidence-signers.json`.
|
||||||
@@ -6,13 +6,7 @@ metadata:
|
|||||||
type: project
|
type: project
|
||||||
---
|
---
|
||||||
|
|
||||||
# Project Status (2026-07-13)
|
# Project Status (2026-07-02)
|
||||||
|
|
||||||
## Distributed inference performance (2026-07-14)
|
|
||||||
|
|
||||||
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
|
|
||||||
|
|
||||||
> Doc reconciliation 2026-07-13: `docs/prd.json` tracks US-001…US-050 (048 memory budget, 049 mainnet pilot, 050 Qwen demand placement). ADRs 0025–0026 added (TAI phase B/C, assignment ownership).
|
|
||||||
|
|
||||||
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:
|
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:
|
||||||
|
|
||||||
|
|||||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -20,7 +20,6 @@ dist/
|
|||||||
!.env.testnet
|
!.env.testnet
|
||||||
.rocm-local/*
|
.rocm-local/*
|
||||||
.pytest-tmp/*
|
.pytest-tmp/*
|
||||||
.cache/
|
|
||||||
|
|
||||||
# Local tracker/node sqlite databases (never commit runtime state)
|
# Local tracker/node sqlite databases (never commit runtime state)
|
||||||
*.sqlite
|
*.sqlite
|
||||||
|
|||||||
@@ -4,8 +4,7 @@
|
|||||||
|
|
||||||
configVersion = "2.1"
|
configVersion = "2.1"
|
||||||
tracker = "json"
|
tracker = "json"
|
||||||
agent = "codex"
|
agent = "opencode"
|
||||||
model = "gpt-5.6-terra"
|
|
||||||
maxIterations = 0
|
maxIterations = 0
|
||||||
autoCommit = true
|
autoCommit = true
|
||||||
|
|
||||||
|
|||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
|
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
|
||||||
|
|
||||||
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers; no upfront stake, probation only). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet SOL/USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers, stake-free). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
||||||
|
|
||||||
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch. Runbook: [04-toploc-calibration-run](./runbooks/04-toploc-calibration-run.md).
|
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch.
|
||||||
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
||||||
|
|
||||||
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
||||||
@@ -77,7 +77,6 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
|
|||||||
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
||||||
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
|
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
|
||||||
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
|
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
|
||||||
| [04 TOPLOC calibration run](./runbooks/04-toploc-calibration-run.md) (issue 21 ops) |
|
|
||||||
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
|
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
|
||||||
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@
|
|||||||
|
|
||||||
## 1. Mission / where we are
|
## 1. Mission / where we are
|
||||||
|
|
||||||
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer. Pre-release audits found the money/trust path was not alpha-ready; **Bucket 1 alpha blockers are implemented** (see `.scratch/alpha-hardening/README.md`). Remaining launch gates: issue **21** (human calibration run), post-alpha Bucket 2 (12–15), and active scratch tracks (NCA, perf, distributed GGUF).
|
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 0016–0019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -42,7 +42,7 @@ Point to artifacts — do not re-derive from this handoff.
|
|||||||
| Path | What it contains |
|
| Path | What it contains |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
|
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
|
||||||
| `.scratch/alpha-hardening/issues/` | 25 work items (Buckets 1–3 + perf follow-ups) |
|
| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 1–3) |
|
||||||
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
|
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
|
||||||
| `docs/adr/0016–0019` | Alpha scope, auth, fraud, multi-tracker design |
|
| `docs/adr/0016–0019` | Alpha scope, auth, fraud, multi-tracker design |
|
||||||
| `docs/agents/issue-tracker.md` | Issue file conventions |
|
| `docs/agents/issue-tracker.md` | Issue file conventions |
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
Status: ready-for-human
|
Status: ready-for-human
|
||||||
|
|
||||||
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money mainnet USDT traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money (mainnet SOL/USDT) traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
||||||
|
|
||||||
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
|
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
|
||||||
|
|
||||||
@@ -14,9 +14,9 @@ Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and
|
|||||||
|
|
||||||
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
||||||
|
|
||||||
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay on probation (no upfront stake) until it's done, then move to paid USDT serving once thresholds derive from their own hardware.
|
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware.
|
||||||
|
|
||||||
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
|
**Current gap (confirmed 2026-07-06 by code read):** none of the three pieces below exist yet.
|
||||||
|
|
||||||
1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (06–10) currently runs on a guessed threshold baked into that bool, not a calibrated one.
|
1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (06–10) currently runs on a guessed threshold baked into that bool, not a calibrated one.
|
||||||
2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 1–3-node *routes* and measures latency, not TOPLOC divergence across *every* registered node.
|
2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 1–3-node *routes* and measures latency, not TOPLOC divergence across *every* registered node.
|
||||||
@@ -36,7 +36,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
|
|||||||
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
|
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
|
||||||
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
|
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
|
||||||
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
|
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
|
||||||
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section; [runbook 04](../runbooks/04-toploc-calibration-run.md).
|
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section.
|
||||||
|
|
||||||
## ADR links
|
## ADR links
|
||||||
|
|
||||||
|
|||||||
@@ -440,12 +440,12 @@
|
|||||||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||||||
],
|
],
|
||||||
"priority": 21,
|
"priority": 21,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money mainnet USDT). Operator runbook: .scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md",
|
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"AH-006"
|
"AH-006"
|
||||||
],
|
],
|
||||||
"completionNotes": "Engineering complete and unit-tested. Remaining: human runs POST /v1/calibration/toploc/run on live hired-VPS fleet, records envelope/FPR, wires thresholds — see runbook 04 and packages/validator/README.md."
|
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "AH-022",
|
"id": "AH-022",
|
||||||
|
|||||||
@@ -1,70 +0,0 @@
|
|||||||
# Runbook 04 — Honest-noise TOPLOC calibration (issue 21)
|
|
||||||
|
|
||||||
**Status:** engineering complete; **operator action required** before production audit thresholds.
|
|
||||||
|
|
||||||
**Blocks:** enabling calibrated TOPLOC thresholds on a mainnet / friends-test fleet (issue 21, ADR-0018).
|
|
||||||
|
|
||||||
## When to run
|
|
||||||
|
|
||||||
- Before first real-money traffic with audit enforcement enabled.
|
|
||||||
- Again whenever the fleet’s **hardware mix** changes materially (new GPU generation, CPU-only nodes added, precision/recipe change per model).
|
|
||||||
|
|
||||||
Alpha exception: with a **small hired-VPS-only** fleet, `gate_status.ready` may mean “covers every node we operate today” (`--toploc-calibration-gate-min-hardware-profiles 1`).
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Tracker running with billing + registry + `--toploc-calibration-db PATH` (or default under tracker cwd).
|
|
||||||
- At least one **solo-capable** node per hardware profile you want in the corpus (full model coverage — partial shards are skipped).
|
|
||||||
- Admin or validator credentials (`Authorization` header or validator service token per ADR-0017).
|
|
||||||
- Reference validator can replay the fixed calibration prompt (same model/seed as dispatch uses).
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
1. **Register the fleet** — all nodes you intend to pay on mainnet should be up, admitted (NCA when enabled), and solo-serving the calibration model.
|
|
||||||
|
|
||||||
2. **Dispatch the job** (admin/validator only):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST "https://<tracker>/v1/calibration/toploc/run" \
|
|
||||||
-H "Authorization: Bearer <admin-or-validator-token>" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{}'
|
|
||||||
```
|
|
||||||
|
|
||||||
Partial-shard nodes appear under `skipped_partial_shard_node_ids`. Per-node failures appear under `skipped` with reasons.
|
|
||||||
|
|
||||||
3. **Wait for completion** — watch tracker logs and node consoles until every solo-capable node has a row in the corpus.
|
|
||||||
|
|
||||||
4. **Fetch results**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl "https://<tracker>/v1/calibration/toploc/results" \
|
|
||||||
-H "Authorization: Bearer <admin-or-validator-token>"
|
|
||||||
```
|
|
||||||
|
|
||||||
Record:
|
|
||||||
- `envelope` — p99 metrics + 20% safety margin (recommended tolerances).
|
|
||||||
- `gate_status.ready` and `gate_status.hardware_profiles`.
|
|
||||||
- `estimated_false_positive_rate` (in-sample sanity check only).
|
|
||||||
|
|
||||||
5. **Write up thresholds** — paste envelope values into operator notes / issue 21 comment. Do **not** wire into production `ToplocAuditConfig` until you have reviewed FPR on this fleet.
|
|
||||||
|
|
||||||
6. **Mark issue 21 done** — when corpus covers the launch fleet and thresholds are documented.
|
|
||||||
|
|
||||||
## Two-wallet / minimal pilot variant
|
|
||||||
|
|
||||||
If your “fleet” is one node machine + one client:
|
|
||||||
|
|
||||||
- Run calibration against the **node** profile only (one hardware row is enough for `gate_status` with min profiles = 1).
|
|
||||||
- Client wallet is irrelevant to calibration — it never serves inference.
|
|
||||||
|
|
||||||
## Do not
|
|
||||||
|
|
||||||
- Enable stricter production audit thresholds before this completes.
|
|
||||||
- Reuse a corpus collected on devnet/mock hardware for a different mainnet GPU mix without re-running.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue: `.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`
|
|
||||||
- Code: `packages/tracker/meshnet_tracker/calibration.py`, `POST /v1/calibration/toploc/run`
|
|
||||||
- Validator: `packages/validator/README.md` — TOPLOC audit contract
|
|
||||||
@@ -12,10 +12,4 @@ Provide an opt-in, admin-only tracker Dashboard Testing tab that dynamically dis
|
|||||||
- One active run.
|
- One active run.
|
||||||
- Real inference stays separately environment-gated and excluded from default suites.
|
- Real inference stays separately environment-gated and excluded from default suites.
|
||||||
|
|
||||||
## Operator workflow
|
|
||||||
|
|
||||||
See [`docs/dev/dashboard-test-runner.md`](../../docs/dev/dashboard-test-runner.md)
|
|
||||||
for launch configuration, default safe suites vs the gated real-inference suite,
|
|
||||||
and required environment variables.
|
|
||||||
|
|
||||||
See `prd.json` for executable Ralph user stories and acceptance criteria.
|
See `prd.json` for executable Ralph user stories and acceptance criteria.
|
||||||
|
|||||||
@@ -51,16 +51,15 @@
|
|||||||
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
|
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
|
||||||
],
|
],
|
||||||
"priority": 3,
|
"priority": 3,
|
||||||
"passes": true,
|
"passes": false,
|
||||||
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
|
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"US-001",
|
"US-001",
|
||||||
"US-002"
|
"US-002"
|
||||||
],
|
]
|
||||||
"completionNotes": "Completed by agent"
|
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"updatedAt": "2026-07-12T01:58:06.286Z"
|
"updatedAt": "2026-07-11T17:02:30.520Z"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,12 +1,8 @@
|
|||||||
# ADR-0024: Lean Native Distributed GGUF Runtime
|
# ADR-0020: Lean Native Distributed GGUF Runtime
|
||||||
|
|
||||||
Status: Accepted
|
Status: Accepted
|
||||||
Date: 2026-07-13
|
Date: 2026-07-13
|
||||||
|
|
||||||
> **Numbering note:** ADR-0020 is reserved for dashboard chat streaming and mixed-topology routing (`docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md`). This record was originally drafted as ADR-0020 in `.scratch/distributed-gguf-runtime/` and renumbered to avoid the collision.
|
|
||||||
>
|
|
||||||
> **Relation to ADR-0001:** PyTorch/safetensors remains the correctness reference and current production backend. This ADR defines a benchmark-gated native GGUF path; it does not revoke ADR-0001 until release gates pass.
|
|
||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The project currently uses Transformers/safetensors as its real model execution backend. This provides broad architecture coverage and a correctness reference, but reported and observed consumer CPU/GPU inference performance motivates evaluating llama.cpp/GGML and quantized GGUF.
|
The project currently uses Transformers/safetensors as its real model execution backend. This provides broad architecture coverage and a correctness reference, but reported and observed consumer CPU/GPU inference performance motivates evaluating llama.cpp/GGML and quantized GGUF.
|
||||||
@@ -123,11 +119,3 @@ Rejected. gRPC/HTTP2 already provides mature streaming, flow control, deadlines,
|
|||||||
4. Real two-machine execution using both Shards.
|
4. Real two-machine execution using both Shards.
|
||||||
5. End-to-end performance/fit advantage over the current distributed route.
|
5. End-to-end performance/fit advantage over the current distributed route.
|
||||||
6. Separate Qwen3-family architecture certification.
|
6. Separate Qwen3-family architecture certification.
|
||||||
|
|
||||||
## Relationship to US-042 (whole-model GGUF shortcut)
|
|
||||||
|
|
||||||
[US-042](../issues/42-gguf-llamacpp-node-backend.md) **phase C** ships first: a node with enough RAM serves a **full** GGUF via llama.cpp on a single-hop Inference Route using the existing HTTP activation seam and PyTorch-era tracker integration. That is intentionally small and does not require this ADR's gRPC worker or llama.cpp patch stack.
|
|
||||||
|
|
||||||
This ADR's track starts only after **DGR-001** (controlled safetensors-vs-GGUF benchmark) shows a meaningful speed or fit benefit. Then implement the native worker (DGR-002+) — which subsumes US-042 direction A (layer-range GGUF + boundary tensors) if the benchmark warrants it.
|
|
||||||
|
|
||||||
Do not run US-042 phase C and DGR-008+ in parallel on the same node backend without an explicit integration plan; phase C uses llama-cpp-python (or equivalent) whole-model path; ADR-0024 uses the standalone C++ worker.
|
|
||||||
384
.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md
Normal file
384
.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md
Normal file
@@ -0,0 +1,384 @@
|
|||||||
|
# GLM-5.2 Max distributed alpha roadmap
|
||||||
|
|
||||||
|
Status: proposed executable epic target
|
||||||
|
Last updated: 2026-07-13
|
||||||
|
|
||||||
|
## Executive decision
|
||||||
|
|
||||||
|
The alpha-release target is the exact open-weight model `zai-org/GLM-5.2` served with `reasoning_effort=max` from the smallest published Unsloth GGUF recipe, `UD-IQ1_S`, across multiple consumer machines.
|
||||||
|
|
||||||
|
“Max” is a reasoning mode selected by the chat template/API request. It is not a separate model checkpoint.
|
||||||
|
|
||||||
|
Alpha is earned only when the real target model:
|
||||||
|
|
||||||
|
1. cannot fit within any one participating node's admitted memory;
|
||||||
|
2. loads as contiguous layer Shards on at least two physical consumer machines;
|
||||||
|
3. performs real GLM-5.2 MoE + DSA + IndexShare computation on every selected node;
|
||||||
|
4. responds through the existing OpenAI-compatible Meshnet API with `reasoning_effort=max`;
|
||||||
|
5. passes locked parity, usefulness, performance, telemetry, cancellation, and cleanup checks; and
|
||||||
|
6. stores all model artifacts on configured mounted-drive storage, never under `/home`.
|
||||||
|
|
||||||
|
The shortest safe path is not “support every GGUF architecture.” Dense Llama remains a small structural fixture. GLM-5.2 moves onto the critical path immediately after exact recipe identity and the pinned llama.cpp boundary. Qwen expansion, 1M context certification, MTP/speculative decoding, broad concurrency, and automatic route repair are post-alpha.
|
||||||
|
|
||||||
|
## 1. Exact target contract
|
||||||
|
|
||||||
|
### 1.1 Source model
|
||||||
|
|
||||||
|
| Field | Locked target |
|
||||||
|
|---|---|
|
||||||
|
| Official repository | `zai-org/GLM-5.2` |
|
||||||
|
| Official revision observed 2026-07-13 | `b4734de4facf877f85769a911abafc5283eab3d9` |
|
||||||
|
| Model-weight license | MIT |
|
||||||
|
| Official code/documentation license | Apache-2.0 |
|
||||||
|
| Architecture | `glm_moe_dsa` / `GlmMoeDsaForCausalLM` |
|
||||||
|
| Official architecture label | 744B total / approximately 40B active per token |
|
||||||
|
| Exact stored checkpoint tensors | 753,329,940,480 parameters |
|
||||||
|
| Transformer layers | 78 backbone layers plus one shared NextN/MTP layer in the artifact |
|
||||||
|
| Layer types | first 3 dense; remaining 75 sparse MoE |
|
||||||
|
| Routed experts | 256 |
|
||||||
|
| Experts selected | 8 routed experts plus shared expert path |
|
||||||
|
| Hidden width | 6,144 |
|
||||||
|
| Attention | MLA under DSA, lightning indexer top-k 2,048 |
|
||||||
|
| IndexShare | indexer roles are encoded by `indexer_types`; consumers reuse prior Full-layer indices |
|
||||||
|
| Architectural maximum context | 1,048,576 tokens |
|
||||||
|
| Alpha reasoning mode | `reasoning_effort=max` |
|
||||||
|
|
||||||
|
The runtime must derive these values from the pinned artifact and fail closed on contradictory metadata. Marketing names are not compatibility identity.
|
||||||
|
|
||||||
|
### 1.2 Alpha GGUF artifact
|
||||||
|
|
||||||
|
| Field | Locked target |
|
||||||
|
|---|---|
|
||||||
|
| GGUF repository | `unsloth/GLM-5.2-GGUF` |
|
||||||
|
| GGUF revision observed 2026-07-13 | `abc55e72527792c6e77069c99b4cb7de16fa9f23` |
|
||||||
|
| Quantization | `UD-IQ1_S` |
|
||||||
|
| Files | six GGUF shards |
|
||||||
|
| Exact published bytes | 216,715,360,960 bytes |
|
||||||
|
| Binary GiB | 201.832 GiB |
|
||||||
|
| Published quality indication | about 76.2% top-1 agreement with the high-precision reference on Unsloth's quantization analysis |
|
||||||
|
| Mounted storage rule | configured mounted drive only; never `/home` |
|
||||||
|
|
||||||
|
Before downloading 216.7 GB, DGR-017 must generate a checked-in target manifest containing repository revisions, expected filenames, byte sizes, and resolved LFS SHA-256 values. Download is resumable and verified before route admission.
|
||||||
|
|
||||||
|
`UD-IQ1_M` (228,492,966,624 bytes / 212.801 GiB) is the first diagnostic fallback if `UD-IQ1_S` exposes a runtime or quality defect. It does not satisfy the explicit “lowest quantization” alpha target unless the target contract is changed by human review.
|
||||||
|
|
||||||
|
### 1.3 Runtime semantics required for alpha
|
||||||
|
|
||||||
|
Required:
|
||||||
|
|
||||||
|
- GGUF parsing and quantized kernels from one exact llama.cpp pin.
|
||||||
|
- Correct GLM-5.2 MoE routing, selected experts, and shared expert.
|
||||||
|
- Correct compressed MLA KV cache for locally owned layers.
|
||||||
|
- Native DSA lightning indexer and sparse attention.
|
||||||
|
- Correct IndexShare Full/Shared role execution from artifact metadata.
|
||||||
|
- Range-owned contiguous transformer layers; each owned layer keeps all of its experts local.
|
||||||
|
- Head-only embeddings and tail-only final norm/output head/sampling.
|
||||||
|
- Architecture-defined activation boundary and optional DSA index sideband.
|
||||||
|
- `reasoning_effort=max` chat-template behavior through the public API.
|
||||||
|
- F32 seam correctness lane and a separately certified production activation dtype.
|
||||||
|
|
||||||
|
Not required for alpha:
|
||||||
|
|
||||||
|
- MTP/speculative decoding. The trailing NextN tensors may be loaded or explicitly excluded according to a certified recipe, but cannot be silently misinterpreted.
|
||||||
|
- Full 1,048,576-token context.
|
||||||
|
- Continuous batching beyond one target session.
|
||||||
|
- Public-WAN tensor or expert parallel collectives.
|
||||||
|
- Automatic mid-generation repair or KV migration.
|
||||||
|
- Every CPU/GPU backend combination.
|
||||||
|
|
||||||
|
## 2. Minimum resource envelope
|
||||||
|
|
||||||
|
### 2.1 Weight and runtime memory
|
||||||
|
|
||||||
|
The smallest artifact occupies 201.832 GiB before KV, DSA indexer state, scratch buffers, backend workspaces, process memory, and the operating system. **224 GiB aggregate runtime-accessible memory is only the experimental hard-fit floor**, consistent with Unsloth's approximate 223 GB one-bit requirement. It is not a conservative operational envelope.
|
||||||
|
|
||||||
|
For admission, each node reserves:
|
||||||
|
|
||||||
|
```text
|
||||||
|
max(20% of physically usable memory, 8 GiB)
|
||||||
|
```
|
||||||
|
|
||||||
|
The remainder is the combined weight-plus-KV placement budget. Actual peak scratch is measured by backend/context and can force one extra node. Unified memory is counted once: integrated-GPU “VRAM” must not be added again to the same physical system RAM.
|
||||||
|
|
||||||
|
| Physical usable tier | Minimum reserve | Weight + KV placement budget | IQ1_S 16K arithmetic minimum | Operational position |
|
||||||
|
|---:|---:|---:|---:|---|
|
||||||
|
| 32 GiB | 8.0 GiB | 24.0 GiB | 9 nodes | use 10 if attempted; latency-heavy |
|
||||||
|
| 48 GiB | 9.6 GiB | 38.4 GiB | 6 nodes | possible; latency-heavy |
|
||||||
|
| 64 GiB | 12.8 GiB | 51.2 GiB | 4 nodes | hard minimum; **5 recommended** |
|
||||||
|
| 96 GiB | 19.2 GiB | 76.8 GiB | 3 nodes | recommended |
|
||||||
|
| 128 GiB unified/system | 25.6 GiB | 102.4 GiB | 2 nodes | arithmetic hard minimum; **3 recommended** |
|
||||||
|
|
||||||
|
The planner must use exact tensor byte ownership, not equal percentages. Embeddings, final head, dense versus MoE layers, shared experts, indexer tensors, quant block alignment, KV distribution, and backend workspace make equal layer counts unequal in memory.
|
||||||
|
|
||||||
|
Recommended first target route: **three 96/128-GiB-class physical machines** or **five 64-GiB-class machines**, on the same wired switch with mounted model storage. Four 64-GiB or two 128-GiB machines are fit probes only and qualify solely if exact placement and measured peak-memory evidence retain the required reserve with no swap/overcommit.
|
||||||
|
|
||||||
|
### 2.2 KV cache
|
||||||
|
|
||||||
|
GLM-5.2 MLA caches 576 latent/rope values per token per backbone layer. Correct DSA also caches 128-dimensional indexer keys: ideally only for the 21 Full indexer layers, while the current experimental implementation may allocate them across all 78 layers. Alpha locks **Q8_0 KV** for quality and budgets the conservative current-implementation layout.
|
||||||
|
|
||||||
|
| Context × concurrency | MLA-only Q8 | Optimized DSA Q8 | Conservative current-DSA Q8 | Conservative current-DSA F16 |
|
||||||
|
|---:|---:|---:|---:|---:|
|
||||||
|
| 16,384 × 1 | 0.73 GiB | 0.77 GiB | **0.89 GiB** | 1.68 GiB |
|
||||||
|
| 131,072 × 1 | 5.83 GiB | 6.18 GiB | **7.12 GiB** | 13.41 GiB |
|
||||||
|
| 1,048,576 × 1 | 46.62 GiB | 49.41 GiB | **56.98 GiB** | 107.25 GiB |
|
||||||
|
|
||||||
|
These are planning estimates, not admission truth. The runtime must report measured allocated/resident MLA and indexer cache by Shard. Alpha configures a 16,384-token window, Q8_0 KV, and one session. Longer contexts and lower-bit KV are separate quality/resource certification gates.
|
||||||
|
|
||||||
|
### 2.3 Activation seams and network
|
||||||
|
|
||||||
|
A BF16 hidden-state boundary is 6,144 elements = 12,288 bytes/token before framing.
|
||||||
|
|
||||||
|
- A 16,384-token prefill sends about 192 MiB per seam.
|
||||||
|
- One decode token sends about 12 KiB per seam.
|
||||||
|
- A 512-token decode sends about 6 MiB per seam.
|
||||||
|
- Four nodes imply three serial seams.
|
||||||
|
|
||||||
|
If a Shard boundary splits an IndexShare producer/consumer group, a 2,048-entry int32 top-k sideband can add up to 8 KiB/query before framing. The route planner should prefer boundaries that preserve complete IndexShare ownership groups. The protocol must still support and validate the named sideband because memory fit may force an internal group split.
|
||||||
|
|
||||||
|
Decode bandwidth is small, but every generated token crosses all seams serially, so node count and per-hop latency dominate. Alpha requires a same-switch wired route: **2.5 GbE minimum and 10 GbE recommended**, with measured one-way/RTT, serialization, and queue latency. A 1 GbE route may be retained as fit-only evidence but is not the recommended alpha topology. Alpha records per-seam bytes, p50/p95 transfer latency, retries, and checksum failures; no speed claim is inferred from link rate alone.
|
||||||
|
|
||||||
|
### 2.4 Storage
|
||||||
|
|
||||||
|
The shortest alpha path allows every node to hold the complete six-file source GGUF while mapping/allocating only owned tensors. This minimizes artifact-transformation risk but costs 216.7 GB disk per node.
|
||||||
|
|
||||||
|
Deterministic source-bound layer packages are a follow-up optimization. If needed before target fit, every package must retain:
|
||||||
|
|
||||||
|
- source repository/revision and source file hashes;
|
||||||
|
- exact owned tensor names, layer range, and endpoint role;
|
||||||
|
- tokenizer/config identity;
|
||||||
|
- deterministic package hash; and
|
||||||
|
- proof that composing all packages matches the source tensor inventory.
|
||||||
|
|
||||||
|
## 3. Current state and critical gaps
|
||||||
|
|
||||||
|
### Completed foundation
|
||||||
|
|
||||||
|
- DGR-001: immutable CPU contract plus separate signed ROCm diagnostic. CPU v1 remains `stop`; the GPU diagnostic establishes a viable fit/performance investigation lane but does not rewrite CPU evidence.
|
||||||
|
- DGR-002: versioned backend-neutral gRPC/Protobuf Shard protocol with bounded fragments and compatibility checks.
|
||||||
|
- Existing Meshnet: Tracker, contiguous Shards, Route Sessions/epochs, relay/direct transport, local Hot KV semantics in the reference backend, cancellation, telemetry, billing, and model-agnostic admission.
|
||||||
|
|
||||||
|
### Missing before target alpha
|
||||||
|
|
||||||
|
1. Exact GLM target/artifact manifest and memory-fit planner.
|
||||||
|
2. A current llama.cpp pin proven to load and generate with the exact `UD-IQ1_S` artifact.
|
||||||
|
3. A narrow decision on native GLM-5.2 DSA/IndexShare support. As observed 2026-07-13, merged llama.cpp PR #24770 loads GLM-5.2 through a dense-MLA compatibility path, while full IndexShare/DSA PR #25407 remains open and its generic sparse path can be slower than dense fallback. Generic CPU lightning-indexer support is merged; backend coverage remains uneven.
|
||||||
|
4. A decode protocol amendment. `ActivationChunk` carries `TensorBundle`, but the current `DecodeStep` fast path carries only one `NamedTensor`; it cannot transport a hidden state plus GLM top-k sideband. Tail token/logit and sampling behavior also needs an explicit typed result contract.
|
||||||
|
5. Correct range-owned GGUF loading and memory proof.
|
||||||
|
6. GLM-specific boundary/KV/IndexShare semantics.
|
||||||
|
7. Standalone native worker and Meshnet integration.
|
||||||
|
8. Real target hardware route with no node individually able to admit the whole model.
|
||||||
|
9. Locked target parity, usefulness, speed, failure, and cleanup evidence.
|
||||||
|
|
||||||
|
### Donor policy
|
||||||
|
|
||||||
|
`Mesh-LLM/mesh-llm` is a high-value test and patch donor. Its live GLM branch was observed with 261 llama.cpp patches, 167 named for GLM/DSA/MTP-related work. That is evidence of the problem's depth, not an acceptable maintained fork boundary.
|
||||||
|
|
||||||
|
Audit and selectively reproduce the smallest independently understood pieces for:
|
||||||
|
|
||||||
|
- GLM DSA graph semantics;
|
||||||
|
- lightning indexer and sparse-attention tests;
|
||||||
|
- IndexShare metadata/Full/Shared validation;
|
||||||
|
- top-k sideband shape and lifecycle;
|
||||||
|
- stage-local KV filtering; and
|
||||||
|
- target parity/performance fixtures.
|
||||||
|
|
||||||
|
Do not import Mesh-LLM routing, discovery, scheduler, public mesh, package manager, or full patch stack. Keep Meshnet as the sole control plane and collaborate narrowly upstream with llama.cpp/Mesh-LLM maintainers where practical.
|
||||||
|
|
||||||
|
## 4. Revised roadmap
|
||||||
|
|
||||||
|
## Phase 0 — lock the target before implementation
|
||||||
|
|
||||||
|
### DGR-017: lock GLM-5.2 Max target and alpha contract
|
||||||
|
|
||||||
|
Deliver:
|
||||||
|
|
||||||
|
- machine-readable target manifest for official and GGUF revisions;
|
||||||
|
- exact `UD-IQ1_S` file/size/hash inventory;
|
||||||
|
- architecture/config/chat-template snapshot;
|
||||||
|
- memory/KV/network planner with unified-memory de-duplication;
|
||||||
|
- immutable alpha acceptance thresholds from section 5; and
|
||||||
|
- current upstream/donor status report.
|
||||||
|
|
||||||
|
Exit: target identity and alpha requirements are reviewable without downloading the model.
|
||||||
|
|
||||||
|
## Phase 1 — establish a correct whole-model oracle
|
||||||
|
|
||||||
|
### DGR-003: exact runtime recipe identity
|
||||||
|
|
||||||
|
Extend the existing generic identity with GLM fields: DSA/IndexShare metadata, adapter version, reasoning template revision, activation bundle schema, KV dtype/layout, llama.cpp pin/patch hash, and target artifact manifest hash.
|
||||||
|
|
||||||
|
### DGR-004: reproducible llama.cpp pin and narrow patch boundary
|
||||||
|
|
||||||
|
Select a current exact upstream commit only after testing its stock GLM behavior. Add clean fetch/apply/build checks. Record every donor patch and whether it is adopted, rewritten, rejected, or waiting upstream.
|
||||||
|
|
||||||
|
### DGR-018: certify whole-model GLM-5.2 runtime semantics
|
||||||
|
|
||||||
|
On a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation, or a measured equivalent:
|
||||||
|
|
||||||
|
1. verify all six `UD-IQ1_S` shards;
|
||||||
|
2. load with a stock pinned runtime and capture tensor/metadata warnings;
|
||||||
|
3. prove whether DSA, IndexShare, shared expert, and Max template are actually active;
|
||||||
|
4. add the minimum correctness patches/tests required;
|
||||||
|
5. run deterministic prefill/decode and fixed Max-mode sentinel prompts; and
|
||||||
|
6. sign the oracle recipe, output, telemetry, and limitations.
|
||||||
|
|
||||||
|
Exit: one whole-model oracle exists for the same artifact/runtime semantics the distributed path will implement. “It emits text” is insufficient.
|
||||||
|
|
||||||
|
## Phase 2 — build the generic local seam using small fixtures
|
||||||
|
|
||||||
|
### DGR-005: range-owned GGUF tensors
|
||||||
|
|
||||||
|
Keep dense Llama as a cheap structural fixture. Implement authoritative owned-tensor registration/loading, head/tail ownership, and measured resident-memory scaling. Design tensor classification so GLM adds explicit rules rather than unchecked name substitution.
|
||||||
|
|
||||||
|
### DGR-006: architecture-defined boundary
|
||||||
|
|
||||||
|
Implement named boundary bundles, F32 correctness lane, bounded fragmentation, and optional sidebands. Amend the decode fast path so it carries a versioned `TensorBundle` rather than one `NamedTensor`, while preserving a small one-tensor encoding. Define an explicit typed tail result for logits/token output and bind sampling/chat-template parameters to the recipe/request. Regenerate Python/C++ schema code and compatibility goldens. Dense fixture parity proves the seam mechanism, not GLM certification.
|
||||||
|
|
||||||
|
Exit: two local processes can execute a small dense model with correct range ownership and boundary parity.
|
||||||
|
|
||||||
|
## Phase 3 — add GLM-5.2 as the product adapter
|
||||||
|
|
||||||
|
### DGR-019: implement and certify GLM-5.2 range/DSA/IndexShare semantics
|
||||||
|
|
||||||
|
Deliver explicit support for:
|
||||||
|
|
||||||
|
- 78 main layers and endpoint tensor ownership;
|
||||||
|
- 256-expert MoE routing/top-8 and shared expert;
|
||||||
|
- compressed MLA KV by owned layers;
|
||||||
|
- DSA lightning indexer and sparse attention;
|
||||||
|
- IndexShare metadata, Full producer, Shared consumer, and sideband behavior;
|
||||||
|
- NextN/MTP tensor policy with MTP disabled or enabled explicitly;
|
||||||
|
- shard-boundary planner aware of IndexShare ownership groups; and
|
||||||
|
- whole-model versus two-stage parity against DGR-018.
|
||||||
|
|
||||||
|
Exit: a same-host two-stage target run matches the locked oracle tolerance with real GLM computation in both stages. If the full target cannot fit on one host for this check, use a layer-reduced GLM architecture fixture for graph parity and defer full-artifact output parity to DGR-020; label the distinction explicitly.
|
||||||
|
|
||||||
|
## Phase 4 — worker, KV, and Meshnet route
|
||||||
|
|
||||||
|
Execute existing stories with GLM requirements included:
|
||||||
|
|
||||||
|
1. DGR-007 — isolated local Hot KV keyed by `(Route Session, epoch)`, including DSA/IndexShare state.
|
||||||
|
2. DGR-008 — standalone C++ gRPC worker.
|
||||||
|
3. DGR-009 — Meshnet backend, capability, relay/direct, cancellation, and telemetry integration.
|
||||||
|
4. DGR-010 — small-model local two-process acceptance.
|
||||||
|
5. DGR-011 — real two-physical-machine route and heterogeneous fail-closed behavior.
|
||||||
|
6. DGR-013 subset required by alpha — node loss, cancellation, stale epoch, restart, and memory/KV cleanup.
|
||||||
|
|
||||||
|
Continuous batching (DGR-012) is deliberately not an alpha dependency. The first target release supports one admitted GLM route session; concurrency follows after target correctness and fit.
|
||||||
|
|
||||||
|
## Phase 5 — target alpha gate
|
||||||
|
|
||||||
|
### DGR-020: pass real distributed GLM-5.2 Max alpha acceptance
|
||||||
|
|
||||||
|
Use at least two physical machines and enough aggregate usable memory to meet the locked target planner. No participating node may individually admit the complete target. All stages must report real compute and exact tensor ownership.
|
||||||
|
|
||||||
|
Run the complete acceptance matrix in section 5, preserve raw logs/metrics/output, sign the evidence, and publish an explicit `alpha` or `stop` verdict. Thresholds cannot be weakened after results are known.
|
||||||
|
|
||||||
|
## Phase 6 — post-alpha hardening
|
||||||
|
|
||||||
|
After DGR-020 passes:
|
||||||
|
|
||||||
|
1. DGR-012 — 1/2/4-session continuous batching and bounded admission.
|
||||||
|
2. DGR-014 — final distributed GGUF versus reference-route performance decision.
|
||||||
|
3. 32K, 128K, 200K, then 1M context certification with quantized KV.
|
||||||
|
4. MTP/speculative decoding.
|
||||||
|
5. Deterministic range packages to remove full-artifact replication.
|
||||||
|
6. Additional backend compatibility classes and route topologies.
|
||||||
|
7. DGR-016 — narrow upstream collaboration package, split by independently reviewable llama.cpp changes.
|
||||||
|
8. DGR-015 — Qwen3/Qwen3-MoE only as later architecture expansion, not as the GLM alpha target.
|
||||||
|
|
||||||
|
## 5. Locked alpha acceptance matrix
|
||||||
|
|
||||||
|
These thresholds are set before target execution.
|
||||||
|
|
||||||
|
### 5.1 Identity and fit
|
||||||
|
|
||||||
|
- Exact official and GGUF repository revisions match the target manifest.
|
||||||
|
- All six source GGUF sizes and LFS SHA-256 values verify.
|
||||||
|
- Every route node reports owned tensor names/bytes, layer range, endpoint role, backend, KV recipe, and patch fingerprint.
|
||||||
|
- Union of owned tensors equals the certified runtime-required tensor inventory; unintended overlap is zero.
|
||||||
|
- No node's weight-plus-KV placement budget can hold the complete recipe.
|
||||||
|
- Every node reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-KV placement; measured peak scratch must remain inside that reserve.
|
||||||
|
- Aggregate peak RSS/VRAM stays within physical budgets with no swap, overcommit, mmap-only, or double-counted unified-memory success claim.
|
||||||
|
- Arithmetic-minimum topologies require exact contiguous tensor placement evidence; recommended alpha topology is 5×64 GiB or 3×96/128 GiB.
|
||||||
|
- Unified RAM/VRAM is not double-counted.
|
||||||
|
|
||||||
|
### 5.2 Semantic correctness
|
||||||
|
|
||||||
|
- Logs and graph tests prove GLM MoE/shared-expert, DSA lightning indexer, sparse attention, and IndexShare Full/Shared paths are active; dense-attention compatibility fallback cannot satisfy alpha.
|
||||||
|
- `reasoning_effort=max` is observable in the rendered template/API recipe.
|
||||||
|
- F32 same-backend seam fixture: 32 greedy decode tokens exactly match the whole-model oracle and activation tolerance is locked by DGR-006.
|
||||||
|
- Production seam on the fixed prompt corpus: greedy token agreement is at least 0.90 and mean compared-state/logit cosine similarity is at least 0.999 versus DGR-018, with no malformed or non-finite tensors.
|
||||||
|
- Incompatible artifact, tokenizer, adapter, DSA metadata, boundary, activation, KV, backend class, or runtime patch fingerprints fail closed.
|
||||||
|
|
||||||
|
### 5.3 End-to-end target run
|
||||||
|
|
||||||
|
- Context configured to 16,384 tokens with Q8_0 MLA/indexer KV.
|
||||||
|
- Fixed 4,096-token prompt lane completes prefill.
|
||||||
|
- Route uses a same-switch wired network; 2.5 GbE is the alpha minimum and 10 GbE is recommended.
|
||||||
|
- One Max-mode request generates at least 512 output tokens or reaches a valid natural EOS after at least 128 tokens.
|
||||||
|
- Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable, relevant outputs; raw prompts and outputs are retained for review.
|
||||||
|
- OpenAI-compatible response includes stable model ID, finish reason, and token usage.
|
||||||
|
|
||||||
|
### 5.4 Minimum useful performance
|
||||||
|
|
||||||
|
On the declared minimum alpha topology after one warm-up:
|
||||||
|
|
||||||
|
- median decode throughput is at least 0.5 generated token/s for the fixed Max-mode lane;
|
||||||
|
- 4,096-token-prompt TTFT is at most 10 minutes;
|
||||||
|
- no unexplained stall exceeds 60 seconds without progress telemetry;
|
||||||
|
- per-stage compute, queue, KV, seam bytes/latency, RSS/VRAM, and backend timing are present; and
|
||||||
|
- results are labeled by hardware/topology and are not generalized to other consumer systems.
|
||||||
|
|
||||||
|
If output quality passes but the speed floor fails, verdict is `stop` for alpha and the evidence selects the next optimization target. It is not relabeled as success merely because the model loaded.
|
||||||
|
|
||||||
|
### 5.5 Reliability and security
|
||||||
|
|
||||||
|
- Two consecutive cold starts load, generate, release, and exit cleanly.
|
||||||
|
- Cancellation during prefill and decode releases every stage's queued buffers and KV lease.
|
||||||
|
- One worker loss aborts the route; alpha retries only from token zero on a new compatible route.
|
||||||
|
- Stale epochs and duplicate step IDs are rejected.
|
||||||
|
- Artifact paths stay outside `/home`; logs contain no secrets or unrestricted prompt payloads.
|
||||||
|
- Synthetic workers and layer-reduced fixtures are labeled unit/integration coverage and cannot satisfy target alpha.
|
||||||
|
|
||||||
|
## 6. First execution order
|
||||||
|
|
||||||
|
The next unattended work should run in this order:
|
||||||
|
|
||||||
|
1. DGR-017 — target contract, manifest, planner, and upstream status.
|
||||||
|
2. DGR-003 — exact recipe identity.
|
||||||
|
3. DGR-004 — current llama.cpp pin and minimal patch harness.
|
||||||
|
4. Run in parallel:
|
||||||
|
- DGR-018 — whole-model oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory.
|
||||||
|
- DGR-005 and DGR-006 — generic range/boundary seam on local small fixtures.
|
||||||
|
5. DGR-019 — GLM semantics and parity after both parallel lanes pass.
|
||||||
|
6. DGR-007 through DGR-011 — native worker and real transport route.
|
||||||
|
7. Required DGR-013 failure subset.
|
||||||
|
8. DGR-020 — real target alpha verdict.
|
||||||
|
|
||||||
|
The first external hardware blocker is DGR-018, but DGR-005/DGR-006 proceed locally while that host is sourced. Do not download the full model until DGR-017's exact manifest and storage preflight pass.
|
||||||
|
|
||||||
|
## 7. Sources checked on 2026-07-13
|
||||||
|
|
||||||
|
Authoritative or primary:
|
||||||
|
|
||||||
|
- Official model card and config: <https://huggingface.co/zai-org/GLM-5.2>
|
||||||
|
- Official release/architecture blog: <https://z.ai/blog/glm-5.2>
|
||||||
|
- Official code/documentation repository: <https://github.com/zai-org/GLM-5>
|
||||||
|
- Official source revision API: <https://huggingface.co/api/models/zai-org/GLM-5.2>
|
||||||
|
- Official GLM-5 technical report: <https://arxiv.org/abs/2602.15763>
|
||||||
|
- Unsloth GGUF repository: <https://huggingface.co/unsloth/GLM-5.2-GGUF>
|
||||||
|
- Unsloth local-run/quantization guide: <https://unsloth.ai/docs/models/glm-5.2>
|
||||||
|
- llama.cpp GLM-5.2 support issue: <https://github.com/ggml-org/llama.cpp/issues/24730>
|
||||||
|
- llama.cpp merged dense-MLA compatibility loader: <https://github.com/ggml-org/llama.cpp/pull/24770>
|
||||||
|
- llama.cpp open GLM-5.2 DSA/IndexShare implementation: <https://github.com/ggml-org/llama.cpp/pull/25407>
|
||||||
|
- llama.cpp merged generic CPU lightning indexer: <https://github.com/ggml-org/llama.cpp/pull/24231>
|
||||||
|
- llama.cpp 1M-context discussion: <https://github.com/ggml-org/llama.cpp/discussions/24622>
|
||||||
|
- IndexCache/IndexShare paper: <https://arxiv.org/abs/2603.12201>
|
||||||
|
|
||||||
|
Donor/current implementation evidence:
|
||||||
|
|
||||||
|
- Mesh-LLM repository: <https://github.com/Mesh-LLM/mesh-llm>
|
||||||
|
- Mesh-LLM GLM branch noted by llama.cpp collaborator in issue #24730: `feat/jianyang-glm-52`
|
||||||
|
|
||||||
|
Web/repository observations are pinned by date and must be refreshed in DGR-017 before implementation because upstream support is moving quickly.
|
||||||
@@ -6,6 +6,8 @@ Build one lean native GGUF execution path that lets an Inference Route combine c
|
|||||||
|
|
||||||
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
|
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
|
||||||
|
|
||||||
|
The alpha target is now exact: `zai-org/GLM-5.2` at a pinned revision, the lowest published Unsloth `UD-IQ1_S` GGUF, and `reasoning_effort=max`, distributed across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a cheap structural fixture; Qwen is post-alpha expansion.
|
||||||
|
|
||||||
## Goals
|
## Goals
|
||||||
|
|
||||||
- Execute one GGUF model across independently addressable contiguous Shards.
|
- Execute one GGUF model across independently addressable contiguous Shards.
|
||||||
@@ -15,6 +17,7 @@ The program is benchmark-gated. GGUF is not assumed faster merely because it is
|
|||||||
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
|
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
|
||||||
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
|
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
|
||||||
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
|
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
|
||||||
|
- Pass an immutable GLM-5.2 Max `alpha`/`stop` gate with native MoE, DSA, IndexShare, parity, usefulness, speed, failure, and cleanup evidence.
|
||||||
|
|
||||||
## Quality Gates
|
## Quality Gates
|
||||||
|
|
||||||
@@ -41,6 +44,102 @@ Real-model/hardware stories must:
|
|||||||
|
|
||||||
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
||||||
|
|
||||||
|
## Dependency Graph and Status
|
||||||
|
|
||||||
|
Status as of 2026-07-14 (MAINT-003). Authoritative per-story status is
|
||||||
|
`passes` in [prd.json](prd.json); closed issues live in
|
||||||
|
`docs/issues/distributed-gguf-runtime/`, open and blocked issues in
|
||||||
|
[issues/](issues/).
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
classDef done fill:#c8e6c9,stroke:#2e7d32;
|
||||||
|
classDef blocked fill:#ffcdd2,stroke:#c62828;
|
||||||
|
|
||||||
|
DGR001[DGR-001 perf contract]:::done
|
||||||
|
DGR002[DGR-002 gRPC Shard protocol]:::done
|
||||||
|
DGR003[DGR-003 artifact/recipe identity]:::done
|
||||||
|
DGR004[DGR-004 pinned llama.cpp patch stack]:::done
|
||||||
|
DGR005[DGR-005 dense-Llama range ownership]:::done
|
||||||
|
DGR006[DGR-006 boundary input/output]:::done
|
||||||
|
DGR017[DGR-017 GLM-5.2 target/alpha contract]:::done
|
||||||
|
DGR018[DGR-018 whole-model GLM oracle]:::blocked
|
||||||
|
DGR019[DGR-019 GLM range/DSA/IndexShare]:::blocked
|
||||||
|
DGR020[DGR-020 distributed GLM alpha]:::blocked
|
||||||
|
DGR007[DGR-007 Hot KV State]
|
||||||
|
DGR008[DGR-008 C++ gRPC worker]
|
||||||
|
DGR009[DGR-009 Meshnet integration]
|
||||||
|
DGR010[DGR-010 local two-process acceptance]
|
||||||
|
DGR011[DGR-011 two-machine route]
|
||||||
|
DGR012[DGR-012 continuous batching]
|
||||||
|
DGR013[DGR-013 failure/cancel/restart]
|
||||||
|
DGR014[DGR-014 release gate]
|
||||||
|
DGR015[DGR-015 Qwen3 adapter]
|
||||||
|
DGR016[DGR-016 upstream package]
|
||||||
|
|
||||||
|
DGR002 --> DGR003
|
||||||
|
DGR017 --> DGR003
|
||||||
|
DGR001 --> DGR004
|
||||||
|
DGR017 --> DGR004
|
||||||
|
DGR003 --> DGR005
|
||||||
|
DGR004 --> DGR005
|
||||||
|
DGR002 --> DGR006
|
||||||
|
DGR005 --> DGR006
|
||||||
|
DGR001 --> DGR017
|
||||||
|
DGR002 --> DGR017
|
||||||
|
DGR003 --> DGR018
|
||||||
|
DGR004 --> DGR018
|
||||||
|
DGR017 --> DGR018
|
||||||
|
DGR005 --> DGR019
|
||||||
|
DGR006 --> DGR019
|
||||||
|
DGR018 --> DGR019
|
||||||
|
DGR006 --> DGR007
|
||||||
|
DGR019 --> DGR007
|
||||||
|
DGR002 --> DGR008
|
||||||
|
DGR003 --> DGR008
|
||||||
|
DGR004 --> DGR008
|
||||||
|
DGR006 --> DGR008
|
||||||
|
DGR007 --> DGR008
|
||||||
|
DGR003 --> DGR009
|
||||||
|
DGR008 --> DGR009
|
||||||
|
DGR009 --> DGR010
|
||||||
|
DGR010 --> DGR011
|
||||||
|
DGR007 --> DGR012
|
||||||
|
DGR009 --> DGR012
|
||||||
|
DGR010 --> DGR012
|
||||||
|
DGR008 --> DGR013
|
||||||
|
DGR009 --> DGR013
|
||||||
|
DGR001 --> DGR014
|
||||||
|
DGR011 --> DGR014
|
||||||
|
DGR012 --> DGR014
|
||||||
|
DGR013 --> DGR014
|
||||||
|
DGR014 --> DGR015
|
||||||
|
DGR010 --> DGR016
|
||||||
|
DGR007 --> DGR020
|
||||||
|
DGR008 --> DGR020
|
||||||
|
DGR009 --> DGR020
|
||||||
|
DGR011 --> DGR020
|
||||||
|
DGR013 --> DGR020
|
||||||
|
DGR017 --> DGR020
|
||||||
|
DGR018 --> DGR020
|
||||||
|
DGR019 --> DGR020
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Done (`passes: true`):** DGR-001, DGR-002, DGR-003, DGR-004, DGR-005,
|
||||||
|
DGR-006, DGR-017.
|
||||||
|
- **Blocked on hardware:** DGR-018 requires a 256-GiB-class host with at least
|
||||||
|
224 GiB runtime-accessible memory and 250 GB free storage outside `/home`;
|
||||||
|
no such host is currently available (development host: 124.9 GiB MemTotal).
|
||||||
|
Exact preflight output: [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md).
|
||||||
|
DGR-019 (needs the DGR-018 oracle) and DGR-020 (needs DGR-018/DGR-019 plus
|
||||||
|
multiple physical consumer nodes) are blocked transitively.
|
||||||
|
- **Consequence of the graph as written:** DGR-007 depends on DGR-019, so every
|
||||||
|
remaining story (DGR-007 through DGR-016) is transitively blocked on the
|
||||||
|
256-GiB host. Unblocking the generic dense pipeline without that host would
|
||||||
|
require an explicit re-planning decision to relax the DGR-007 → DGR-019
|
||||||
|
dependency; that decision is out of scope for maintenance and has not been
|
||||||
|
made.
|
||||||
|
|
||||||
## User Stories
|
## User Stories
|
||||||
|
|
||||||
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
||||||
@@ -101,6 +200,8 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
|||||||
**Acceptance Criteria:**
|
**Acceptance Criteria:**
|
||||||
- [ ] Head accepts token IDs and owns token embedding.
|
- [ ] Head accepts token IDs and owns token embedding.
|
||||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||||
|
- [ ] Amend the decode fast path from one `NamedTensor` to a versioned `TensorBundle`, preserving compact one-tensor compatibility and regenerating Python/C++ protocol goldens.
|
||||||
|
- [ ] Define a typed tail logits/token result with sampling and chat-template/reasoning identity.
|
||||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||||
@@ -213,6 +314,35 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
|||||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||||
|
|
||||||
|
### DGR-017: Lock the GLM-5.2 Max target and alpha contract
|
||||||
|
**Description:** Pin exact official/GGUF revisions, `UD-IQ1_S` files and hashes, Max-mode semantics, resource accounting, and immutable target thresholds before implementation results exist.
|
||||||
|
|
||||||
|
**Acceptance Criteria:**
|
||||||
|
- [ ] Produce machine-readable target, resource, upstream-status, and immutable acceptance contracts without downloading full weights.
|
||||||
|
- [ ] Distinguish the 224-GiB hard-fit floor from the recommended 5×64 or 3×96/128 topology, using Q8 KV, 20%/8-GiB node reserves, and a wired 2.5-GbE minimum.
|
||||||
|
- [ ] Count unified RAM/VRAM once and test revision/shard/byte/threshold mutation failures.
|
||||||
|
|
||||||
|
### DGR-018: Certify whole-model GLM-5.2 runtime semantics
|
||||||
|
**Description:** Establish the exact IQ1_S oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory; lock Q8_0 MLA/indexer KV and native target semantics before distributed parity work.
|
||||||
|
|
||||||
|
**Acceptance Criteria:**
|
||||||
|
- [ ] Verify the complete artifact and prove native MoE/shared expert, DSA, IndexShare, KV, NextN policy, and Max-template behavior.
|
||||||
|
- [ ] Dense/replicated compatibility fallback cannot become the oracle merely because it emits text.
|
||||||
|
|
||||||
|
### DGR-019: Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||||
|
**Description:** Add explicit target-model tensor, graph, boundary, sideband, and local-KV ownership after the generic dense seam.
|
||||||
|
|
||||||
|
**Acceptance Criteria:**
|
||||||
|
- [ ] Preserve MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared semantics across contiguous Shards.
|
||||||
|
- [ ] Pass locked fixture/target parity and measured per-Shard memory ownership; never claim full-target parity from a reduced fixture.
|
||||||
|
|
||||||
|
### DGR-020: Pass real distributed GLM-5.2 Max alpha acceptance
|
||||||
|
**Description:** Run the exact lowest-quant target through Meshnet on enough physical consumer nodes that no node can admit the whole recipe.
|
||||||
|
|
||||||
|
**Acceptance Criteria:**
|
||||||
|
- [ ] Pass the immutable identity, semantic, parity, Max-mode usefulness, 0.5 token/s, TTFT, reliability, and mounted-storage gates.
|
||||||
|
- [ ] Preserve signed raw evidence and emit `alpha` only if every target criterion passes; otherwise emit `stop`.
|
||||||
|
|
||||||
## Functional Requirements
|
## Functional Requirements
|
||||||
|
|
||||||
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
|
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
|
||||||
@@ -234,7 +364,7 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
|||||||
- QUIC, WebRTC, or a custom socket protocol.
|
- QUIC, WebRTC, or a custom socket protocol.
|
||||||
- Automatic KV migration or mid-generation route repair in the first release.
|
- Automatic KV migration or mid-generation route repair in the first release.
|
||||||
- Speculative decoding or disaggregated prefill before the core release gate.
|
- Speculative decoding or disaggregated prefill before the core release gate.
|
||||||
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
|
- Supporting every GGUF architecture before the exact GLM-5.2 target; Qwen3-family certification is post-alpha.
|
||||||
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
|
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
|
||||||
|
|
||||||
## Success Metrics
|
## Success Metrics
|
||||||
@@ -247,6 +377,6 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
|
- DGR-001 remains immutable. DGR-017 locks the exact GLM-5.2/`UD-IQ1_S` target and alpha thresholds without rewriting DGR-001 evidence.
|
||||||
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
|
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
|
||||||
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
|
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
|
|||||||
|
|
||||||
1. Read this file completely.
|
1. Read this file completely.
|
||||||
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
|
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
|
||||||
3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
|
3. Read `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`, and the relevant part of `architecture.md`.
|
||||||
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
|
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
|
||||||
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
|
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
|
||||||
6. Read the evidence/handoff directories for every declared dependency.
|
6. Read the evidence/handoff directories for every declared dependency.
|
||||||
@@ -35,6 +35,8 @@ If interrupted after code changes, record every changed file, command result and
|
|||||||
|
|
||||||
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
|
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
|
||||||
|
|
||||||
|
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact served with `reasoning_effort=max` across physical consumer machines. Dense Llama is a structural fixture. Synthetic workers, dense-attention compatibility fallback, a smaller model, or a single host cannot satisfy target alpha. The immutable target contract and resource envelope are in `GLM-5.2-MAX-ALPHA-ROADMAP.md`.
|
||||||
|
|
||||||
A distributed demo is not success. The product must provide:
|
A distributed demo is not success. The product must provide:
|
||||||
|
|
||||||
- Useful measured prefill and decode speed.
|
- Useful measured prefill and decode speed.
|
||||||
@@ -84,6 +86,10 @@ Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp
|
|||||||
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
|
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
|
||||||
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
|
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
|
||||||
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
|
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
|
||||||
|
16. Unified system RAM and integrated-GPU memory are one physical pool and must never be double-counted for admission.
|
||||||
|
17. Alpha requires native GLM-5.2 MoE, DSA, and IndexShare semantics; MTP/speculative decoding and 1M-context certification are post-alpha.
|
||||||
|
18. DGR-006 amends the decode fast path to carry a versioned `TensorBundle` and defines a typed tail logits/token result; the current single-`NamedTensor` fast path is insufficient for GLM sidebands.
|
||||||
|
19. Alpha reserves at least `max(20% of physical usable memory, 8 GiB)` per node outside weight-plus-Q8-KV placement and uses a same-switch wired 2.5 GbE minimum route.
|
||||||
|
|
||||||
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
|
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
|
||||||
|
|
||||||
@@ -154,7 +160,7 @@ The project patch stack is limited to:
|
|||||||
|
|
||||||
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
|
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
|
||||||
|
|
||||||
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
|
Dense Llama is first only as a cheap range/boundary fixture. GLM-5.2 is the explicit product adapter and alpha target immediately afterward. Qwen3/Qwen3-MoE is post-alpha. Do not generalize through unchecked tensor-name substitutions.
|
||||||
|
|
||||||
## Existing code seams to inspect first
|
## Existing code seams to inspect first
|
||||||
|
|
||||||
@@ -211,7 +217,7 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
|
|||||||
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
|
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
|
||||||
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
|
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
|
||||||
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
|
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
|
||||||
| DGR-006 | architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
|
| DGR-006 | decode `TensorBundle` protocol amendment, typed tail-result contract, architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
|
||||||
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
|
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
|
||||||
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
|
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
|
||||||
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
|
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
|
||||||
@@ -222,6 +228,10 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
|
|||||||
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
|
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
|
||||||
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
|
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
|
||||||
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
|
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
|
||||||
|
| DGR-017 | exact GLM-5.2/GGUF target manifest, resource planner, immutable alpha contract and upstream status; `evidence/DGR-017/` |
|
||||||
|
| DGR-018 | verified whole-model `UD-IQ1_S` oracle with native GLM semantic evidence; `evidence/DGR-018/` |
|
||||||
|
| DGR-019 | explicit range-owned GLM MoE/MLA/DSA/IndexShare adapter, fixtures and parity; `evidence/DGR-019/` |
|
||||||
|
| DGR-020 | real multi-node GLM-5.2 Max target evidence and immutable `alpha`/`stop` verdict; `evidence/DGR-020/` |
|
||||||
|
|
||||||
## Dependency handoff rule
|
## Dependency handoff rule
|
||||||
|
|
||||||
@@ -296,7 +306,7 @@ Active decisions:
|
|||||||
- `.scratch/distributed-gguf-runtime/README.md`
|
- `.scratch/distributed-gguf-runtime/README.md`
|
||||||
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
|
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
|
||||||
- `.scratch/distributed-gguf-runtime/architecture.md`
|
- `.scratch/distributed-gguf-runtime/architecture.md`
|
||||||
- `docs/adr/0024-distributed-gguf-runtime.md`
|
- `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`
|
||||||
- `.scratch/distributed-gguf-runtime/PRD.md`
|
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||||
- `.scratch/distributed-gguf-runtime/prd.json`
|
- `.scratch/distributed-gguf-runtime/prd.json`
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ Status: active benchmark-gated implementation program.
|
|||||||
|
|
||||||
## Objective
|
## Objective
|
||||||
|
|
||||||
Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
|
Serve the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode across consumer machines with useful measured performance. Dense Llama is a structural fixture; the real multi-node GLM target is the alpha release gate.
|
||||||
|
|
||||||
|
See **[GLM-5.2 Max distributed alpha roadmap](GLM-5.2-MAX-ALPHA-ROADMAP.md)** for the target identity, minimum hardware, immutable acceptance matrix, and revised execution order. The 224-GiB figure is an experimental hard-fit floor; recommended topology is 5×64 GiB or 3×96/128 GiB after the required per-node reserve.
|
||||||
|
|
||||||
## Critical path
|
## Critical path
|
||||||
|
|
||||||
@@ -25,7 +27,7 @@ Transformers/safetensors remains the correctness baseline. vLLM remains an optio
|
|||||||
- [Current architecture](architecture.md)
|
- [Current architecture](architecture.md)
|
||||||
- [PRD](PRD.md)
|
- [PRD](PRD.md)
|
||||||
- [Ralph backlog](prd.json)
|
- [Ralph backlog](prd.json)
|
||||||
- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [ADR-0020](ADR-0020-distributed-gguf-runtime.md)
|
||||||
- [Milestones](milestones.md)
|
- [Milestones](milestones.md)
|
||||||
- [Issues](issues/)
|
- [Issues](issues/)
|
||||||
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
|
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ Last updated: 2026-07-13
|
|||||||
|
|
||||||
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
|
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
|
||||||
|
|
||||||
|
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode. Its target-specific architecture/resource/acceptance contract is [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a structural fixture, not the product target.
|
||||||
|
|
||||||
## Existing control plane
|
## Existing control plane
|
||||||
|
|
||||||
Meshnet remains the only public control plane:
|
Meshnet remains the only public control plane:
|
||||||
@@ -94,7 +96,9 @@ named tensor bundle
|
|||||||
compression/checksum
|
compression/checksum
|
||||||
```
|
```
|
||||||
|
|
||||||
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small.
|
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small. DGR-006 amends the current v1 decode fast path—which carries only one `NamedTensor`—to carry a versioned `TensorBundle`, while preserving compact one-tensor encoding and explicit compatibility behavior.
|
||||||
|
|
||||||
|
Tail completion is not inferred from an activation tensor name. The protocol exposes a typed logits and/or sampled-token result, and exact sampling parameters plus chat-template/reasoning mode are bound to request/runtime identity.
|
||||||
|
|
||||||
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
|
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
|
||||||
|
|
||||||
@@ -113,6 +117,8 @@ compression/checksum
|
|||||||
|
|
||||||
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
|
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
|
||||||
|
|
||||||
|
GLM-5.2 normally exchanges a 6,144-element hidden state. If a memory-balanced Shard boundary splits an IndexShare Full producer from Shared consumers, the bundle also carries the typed top-k index sideband. The planner prefers boundaries that keep an IndexShare ownership group local, but the protocol validates the sideband rather than assuming it never crosses a seam.
|
||||||
|
|
||||||
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
|
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
|
||||||
|
|
||||||
## Hot KV State and concurrency
|
## Hot KV State and concurrency
|
||||||
@@ -243,17 +249,16 @@ The GGUF path ships only if it is faster at acceptable quality or enables a larg
|
|||||||
|
|
||||||
## Implementation sequence
|
## Implementation sequence
|
||||||
|
|
||||||
1. Lock benchmark/performance contract.
|
1. Preserve completed DGR-001 performance and DGR-002 protocol contracts.
|
||||||
2. Define gRPC/protobuf and exact recipe identity.
|
2. DGR-017 locks exact GLM-5.2 Max artifact, resource, and alpha acceptance identity.
|
||||||
3. Pin llama.cpp and create the minimal patch stack.
|
3. Define exact recipe identity and pin one reproducible llama.cpp boundary.
|
||||||
4. Implement dense-Llama range loading and boundary parity.
|
4. Run two lanes in parallel: DGR-018 establishes the whole-model `UD-IQ1_S` oracle on 224+ GiB usable memory, while DGR-005/DGR-006 implement range loading and named boundary parity with a cheap dense fixture.
|
||||||
5. Implement concurrent local KV.
|
5. DGR-019 adds explicit GLM-5.2 MoE/MLA/DSA/IndexShare semantics after both lanes pass.
|
||||||
6. Build and integrate the standalone worker.
|
6. Implement local KV; build and integrate the standalone worker.
|
||||||
7. Pass local two-process real-model acceptance.
|
7. Pass local two-process and real two-physical-machine execution.
|
||||||
8. Pass real heterogeneous two-machine acceptance.
|
8. Harden cancellation, node loss, restart, and cleanup required by alpha.
|
||||||
9. Add continuous batching and failure hardening.
|
9. DGR-020 executes the exact multi-node target and emits immutable `alpha` or `stop`.
|
||||||
10. Enforce the GGUF-versus-safetensors release gate.
|
10. Post-alpha: continuous batching, final comparison, longer context, MTP, and package optimization.
|
||||||
11. Add Qwen3/Qwen3-MoE as a separately certified adapter.
|
11. Prepare narrow upstream patches/tests; add Qwen as later architecture expansion.
|
||||||
12. Prepare narrow upstream collaboration patches/tests.
|
|
||||||
|
|
||||||
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).
|
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Distributed GGUF Decision Framework
|
# Distributed GGUF Decision Framework
|
||||||
|
|
||||||
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
|
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0020](ADR-0020-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
|
||||||
|
|
||||||
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
|
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# DGR-001 downstream stop-condition handoff
|
||||||
|
|
||||||
|
Status: **DGR-001 is complete; native-track promotion is blocked by the immutable v1 verdict.**
|
||||||
|
|
||||||
|
This is no longer an execution-prerequisite blocker. The required real benchmark
|
||||||
|
ran successfully, every recipe completed at concurrency 1 and 4, artifacts were
|
||||||
|
verified, and deterministic/full test gates passed.
|
||||||
|
|
||||||
|
## Locked result
|
||||||
|
|
||||||
|
`contract-evaluation.json` records:
|
||||||
|
|
||||||
|
```text
|
||||||
|
verdict: stop
|
||||||
|
quality_lane_pass: false
|
||||||
|
speed_benefit: true
|
||||||
|
fit_benefit: true
|
||||||
|
stop_condition_met: true
|
||||||
|
```
|
||||||
|
|
||||||
|
The exact-revision BF16 GGUF quality lane compared every prompt but achieved
|
||||||
|
`0.3333` exact match and `0.9471` mean similarity against the Transformers BF16
|
||||||
|
reference. V1 requires `0.90` and `0.97`. Quantized Q4_K_M had substantial speed
|
||||||
|
and fit benefits, but the contract explicitly forbids speed from redeeming a
|
||||||
|
failed near-lossless quality lane.
|
||||||
|
|
||||||
|
## Scope of this stop
|
||||||
|
|
||||||
|
The measured baseline is Qwen2.5-0.5B on CPU using a CPU-only llama.cpp build.
|
||||||
|
It is not a Radeon, large-model, distributed, or native-shard result. Therefore:
|
||||||
|
|
||||||
|
1. Do not silently mark v1 promoted or weaken its thresholds after observing the
|
||||||
|
data.
|
||||||
|
2. Do not let DGR-004 or later runtime stories treat DGR-001 completion as a
|
||||||
|
positive promotion signal.
|
||||||
|
3. A human may choose one of these explicit paths:
|
||||||
|
- stop the native GGUF track as v1 directs;
|
||||||
|
- diagnose and fix the BF16 runtime divergence, then rerun the exact v1 plan;
|
||||||
|
- authorize a separately versioned GPU/large-model contract whose scope and
|
||||||
|
workload are locked before its measurements.
|
||||||
|
|
||||||
|
All raw evidence, configuration, artifacts, hashes, and reproduction commands
|
||||||
|
are in this directory and `README.md`.
|
||||||
@@ -1,84 +1,199 @@
|
|||||||
# DGR-001 — performance contract baseline
|
# DGR-001 — Safetensors versus GGUF performance contract
|
||||||
|
|
||||||
## Files changed
|
Status: **complete; immutable v1 verdict is `stop`.**
|
||||||
|
|
||||||
- `packages/node/meshnet_node/performance_contract.py`
|
DGR-001 successfully produced a controlled local-real CPU baseline. Completion
|
||||||
- `tests/test_performance_contract.py`
|
means the experiment and decision contract are durable and verified; it does
|
||||||
- `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md`
|
**not** mean the native GGUF track is approved to continue. The locked quality
|
||||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
|
gate failed, so dependent runtime work requires a human decision or a new,
|
||||||
|
explicitly versioned experiment/contract rather than silently weakening v1.
|
||||||
|
|
||||||
## What this slice does
|
## Controlled workload
|
||||||
|
|
||||||
- Locks the DGR-001 benchmark contract in code.
|
- Model: `Qwen/Qwen2.5-0.5B-Instruct`
|
||||||
- Pins the architecture-aligned baseline to **DeepSeek-V2-Lite-Chat** (`deepseek2`).
|
- Exact source revision: `7ae557604adf67be50417f59c2c2f167def9a775`
|
||||||
- Uses the same model on both sides of the comparison:
|
- Machine: `fedora`, Linux `7.0.14-101.fc43.x86_64`, 32 logical CPUs
|
||||||
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
|
- Device: CPU for every recipe; VRAM is therefore correctly reported as zero
|
||||||
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K**
|
- Runtime reference: Transformers `5.13.0`, PyTorch
|
||||||
- Exposes a machine-readable JSON contract with:
|
`2.10.0+rocm7.13.0a20260513`, BF16 safetensors
|
||||||
- benchmark lanes for `transformers` safetensors and `llama.cpp` GGUF on **CPU** and **GPU**
|
- GGUF runtime: llama.cpp version 9991, commit
|
||||||
- concurrency levels `1` and `4`
|
`e920c523e3b8a0163fe498af5bf90df35ff51d25`
|
||||||
- the required metrics list
|
- Workload: three fixed short/medium/long prompts, greedy sampling, 32 output
|
||||||
- an explicit stop condition for “no meaningful speed or fit benefit”
|
tokens, three repeats, two warmups, concurrency 1 and 4, 16 CPU threads
|
||||||
- Adds a deterministic stub benchmark report so the contract now has an executable report shape end to end.
|
- Evidence class: `local-real`
|
||||||
|
|
||||||
## Recent benchmark runner slice
|
All artifacts are beneath `/run/media/popov/DATA/llm/`; no model artifact was
|
||||||
|
created under `/home`.
|
||||||
|
|
||||||
The runner currently uses a deterministic stub backend to exercise the comparison matrix without downloading a model. It emits:
|
## Recipes and exact artifacts
|
||||||
|
|
||||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
|
| Recipe | Artifact | SHA-256 |
|
||||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/stub-benchmark-report.json`
|
|---|---|---|
|
||||||
|
| Transformers BF16 reference | complete mounted Hugging Face snapshot | `e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6` |
|
||||||
|
| llama.cpp BF16 quality lane | `Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf` | `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862` |
|
||||||
|
| llama.cpp Q4_K_M performance/fit lane | `Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf` | `a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5` |
|
||||||
|
|
||||||
The report includes per-device comparisons for:
|
The snapshot digest covers every sorted relative path, resolved size, and file
|
||||||
|
byte, so tokenizer/config drift is included. The BF16 GGUF was converted
|
||||||
|
directly from the exact snapshot while preserving BF16 weights. Q4_K_M was
|
||||||
|
quantized from an exact-revision F16 conversion with the pinned quantizer.
|
||||||
|
Runtime validation recomputes every declared digest before model loading.
|
||||||
|
|
||||||
- `transformers-safetensors-cpu` vs `llama-cpp-gguf-cpu`
|
## Real results
|
||||||
- `transformers-safetensors-gpu` vs `llama-cpp-gguf-gpu`
|
|
||||||
|
|
||||||
and records the memory metric (`rss_bytes` on CPU, `vram_bytes` on GPU), decode speedup, artifact ratio, and output drift.
|
All recipes completed every request with zero failures.
|
||||||
|
|
||||||
## Exact commands and real results
|
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|
||||||
|
|---|---:|---:|---:|
|
||||||
|
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
|
||||||
|
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
|
||||||
|
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
|
||||||
|
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
|
||||||
|
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
|
||||||
|
| Failures | 0 | 0 | 0 |
|
||||||
|
|
||||||
### Targeted tests
|
Against the reference, the eligible Q4_K_M lane measured:
|
||||||
|
|
||||||
```bash
|
- single-request decode speedup: **5.10×**;
|
||||||
pytest -q tests/test_performance_contract.py tests/test_route_session_benchmark.py
|
- concurrency-4 aggregate throughput speedup: **4.20×**;
|
||||||
|
- resident-memory ratio: **0.279×**;
|
||||||
|
- artifact-size ratio: **0.398×**.
|
||||||
|
|
||||||
|
The near-lossless BF16 quality lane compared all three prompts but measured:
|
||||||
|
|
||||||
|
- exact match: **0.3333** (v1 requires at least `0.90`);
|
||||||
|
- mean text similarity: **0.9471** (v1 requires at least `0.97`).
|
||||||
|
|
||||||
|
Tokenization and stopping were controlled: every runtime saw the same prompt
|
||||||
|
token counts and reported 31 post-TTFT decode tokens. The v1 mismatch is a
|
||||||
|
real greedy-output divergence on two prompts, not missing coverage or a
|
||||||
|
text-length artifact. Its root cause remains undetermined; no post-contract
|
||||||
|
logit-tie claim is acceptance evidence. Therefore `contract-evaluation.json`
|
||||||
|
records:
|
||||||
|
|
||||||
|
```text
|
||||||
|
verdict: stop
|
||||||
|
quality_lane_pass: false
|
||||||
|
speed_benefit: true
|
||||||
|
fit_benefit: true
|
||||||
|
stop_condition_met: true
|
||||||
```
|
```
|
||||||
|
|
||||||
Result: `9 passed in 0.14s`
|
Thresholds were not changed after observing these results.
|
||||||
|
|
||||||
### Contract artifact generation
|
## Post-contract parity and ROCm diagnostics
|
||||||
|
|
||||||
```bash
|
`summarize-quality-parity.py` verifies and separates two signed sources. The CPU
|
||||||
PYTHONPATH=packages/node python -m meshnet_node.performance_contract --json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json
|
v1 row uses CPU kernels and a Transformers BF16 oracle; it remains at `0.3333`
|
||||||
|
exact match with an unexplained divergence. The ROCm row uses a different plan,
|
||||||
|
GPU kernels, and a Transformers float32 oracle. In that narrower diagnostic,
|
||||||
|
the same BF16 GGUF artifact matches all three 32-token sequences exactly (`1.0`
|
||||||
|
exact match and `1.0` similarity). No conversion corruption was observed in
|
||||||
|
that three-sequence ROCm sample; this does not prove global conversion
|
||||||
|
correctness or explain the CPU result.
|
||||||
|
|
||||||
|
A separate HIP build at commit `e920c523` was compiled for `gfx1151` and
|
||||||
|
measured `ROCm0: Radeon 8060S Graphics`; its `llama-server` SHA-256 is
|
||||||
|
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`.
|
||||||
|
A signed `gpu-diagnostic` profile measured zero failures:
|
||||||
|
|
||||||
|
| GPU metric | Transformers BF16 ROCm | llama.cpp Q4 ROCm | Q4 ratio |
|
||||||
|
|---|---:|---:|---:|
|
||||||
|
| Decode tok/s, c=1 | 81.12 | 251.25 | **3.10×** |
|
||||||
|
| Aggregate decode tok/s, c=4 | 91.24 | 511.33 | **5.60×** |
|
||||||
|
| TTFT p50, c=1 | 13.77 ms | 11.80 ms | **0.857×** |
|
||||||
|
|
||||||
|
The GPU report is signed under the distinct
|
||||||
|
`run_configured_gpu_diagnostic/v1` producer. The v1 evaluator rejects that
|
||||||
|
producer even when its signature is valid. llama-server process VRAM remains
|
||||||
|
unmeasured, so this diagnostic cannot replace or satisfy the immutable v1
|
||||||
|
contract. Its signed backend detail records the measured `ROCm0: Radeon 8060S
|
||||||
|
Graphics` device and `25/25` offloaded layers.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- `recipe_benchmark.py` provides the runtime-neutral measurement core, true
|
||||||
|
concurrency, continuous in-flight peak-memory sampling, percentile/throughput
|
||||||
|
aggregation, failures, and output drift.
|
||||||
|
- `recipe_drivers.py` provides opt-in Transformers and llama-server drivers,
|
||||||
|
mounted-drive confinement, exact artifact/runtime verification, equal
|
||||||
|
device/thread budgets, greedy-only validation, measured host provenance, a
|
||||||
|
CPU-only v1 guard until process VRAM can be measured honestly, and a distinct
|
||||||
|
signed GPU diagnostic profile that the v1 evaluator cannot accept.
|
||||||
|
- Peak RSS is runtime-scoped: Transformers reports growth above its pre-runtime
|
||||||
|
Python baseline, while llama.cpp reports its isolated server process tree.
|
||||||
|
Both are sampled continuously during in-flight requests.
|
||||||
|
- TTFT uses each runtime's prompt/first-token compute boundary; end-to-end HTTP,
|
||||||
|
scheduling, and queue overhead remains in latency and `queue_wait_ms`.
|
||||||
|
- The exact canonical plan SHA-256 locks prompts, model/revision, sampling,
|
||||||
|
output length, repeats, warmups, and concurrency. The evaluator also requires
|
||||||
|
equal prompt/decode token counts across recipes.
|
||||||
|
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
|
||||||
|
after it; the driver subtracts that token so decode throughput matches the
|
||||||
|
Transformers inter-token convention.
|
||||||
|
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
|
||||||
|
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
|
||||||
|
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
|
||||||
|
failed references.
|
||||||
|
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
|
||||||
|
including raw outcomes and metrics. The contract pins the public key and exact
|
||||||
|
config SHA-256; the private key remains outside Git at mode `0600`.
|
||||||
|
- The signer fingerprint is independently anchored outside this evidence
|
||||||
|
directory in `../../trusted-evidence-signers.json` and checked by tests.
|
||||||
|
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
|
||||||
|
quality gate, and only performance-fit recipes can earn speed/fit benefits.
|
||||||
|
|
||||||
|
## Evidence files
|
||||||
|
|
||||||
|
- `performance-contract.json` — immutable v1 thresholds and stop condition
|
||||||
|
- `benchmark-config.json` — exact real-run plan, drivers, artifacts, and hashes
|
||||||
|
- `results.json` — raw machine-readable per-request and aggregate evidence
|
||||||
|
- `results.txt` — human-readable benchmark summary
|
||||||
|
- `baseline.json` — distilled measurements for later comparison
|
||||||
|
- `contract-evaluation.json` — fail-closed v1 verdict
|
||||||
|
- `quality-parity-diagnosis.json` / `.md` — run/device-scoped signed-evidence summary
|
||||||
|
- `summarize-quality-parity.py` — verifies both evidence chains and regenerates it
|
||||||
|
- `gpu-diagnostic-config.json` — exact ROCm diagnostic artifacts and runtimes
|
||||||
|
- `gpu-diagnostic-results.json` / `.txt` — signed GPU outcomes and summary
|
||||||
|
- `commands.txt` — reproducible conversion, benchmark, evaluation, and test commands
|
||||||
|
- `BLOCKED.md` — downstream stop-condition handoff
|
||||||
|
- `known-unrelated-failure.md` — clean-base reproduction of the tracker race
|
||||||
|
- `../../trusted-evidence-signers.json` — repository-reviewed signer fingerprint
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```text
|
||||||
|
Targeted: 28 passed (5/5 consecutive focused runs)
|
||||||
|
Latest full suite: 755 passed, 13 skipped
|
||||||
|
Earlier full suite: 751 passed, 13 skipped
|
||||||
|
Current cancellation retry matrix, DGR-001: 4/5 passed
|
||||||
|
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
|
||||||
|
compileall: passed
|
||||||
|
git diff --check: passed
|
||||||
|
Evidence JSON parse/integrity checks: passed
|
||||||
```
|
```
|
||||||
|
|
||||||
Result: wrote `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
|
The intermittent tracker cancellation race reproduced at the same rate on the
|
||||||
|
clean base and is retained in `known-unrelated-failure.md`; the final full suite
|
||||||
|
completed green. DGR-001 changes no tracker/proxy files.
|
||||||
|
|
||||||
### Python compile check
|
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
|
||||||
|
invalid: it used Hermes Agent's internal venv. Verification above used the
|
||||||
|
project `.venv`, which has the DGR-002-compatible runtime. Real inference used
|
||||||
|
`.venv-rocm` Python 3.12.
|
||||||
|
|
||||||
```bash
|
## Limitations and dependent-story handoff
|
||||||
python -m compileall packages/node/meshnet_node/performance_contract.py tests/test_performance_contract.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Result: passed
|
- The immutable contract result is a **0.5B CPU baseline**. The separate Radeon
|
||||||
|
diagnostic is real local GPU evidence, but neither result covers a large
|
||||||
## Limitations
|
model, distributed execution, network transport, or a native shard worker.
|
||||||
|
- A separate `GGML_HIP=ON` llama.cpp build exists and produced GPU timings, but
|
||||||
- This slice still uses a deterministic stub backend for the core comparison matrix.
|
llama-server process VRAM is not measurable by the current driver; GPU
|
||||||
- It now also includes a live endpoint runner that can fan out one OpenAI-compatible request per lane when the caller provides endpoints.
|
memory/fit claims therefore remain ineligible for v1.
|
||||||
- It does **not** download or run a real model from within the repo.
|
- Absolute timings are developer-machine measurements; locked ratios and raw
|
||||||
- Real safetensors vs GGUF execution, TTFT/prefill/decode measurements, RSS/VRAM capture, and output-drift comparison are still to be implemented against the contract.
|
artifacts are provided for reproducibility.
|
||||||
|
- DGR-014 may consume v1 only with the exact plan/evidence requirements enforced
|
||||||
## Compatibility notes
|
by `performance_contract.py`.
|
||||||
|
- DGR-004 and later native-runtime work must not treat DGR-001 completion as a
|
||||||
- The contract stays on the DeepSeek2 family to remain close to the DeepSeek-V4-Flash end goal.
|
promotion. V1 says `stop`; proceeding requires a human decision backed by a
|
||||||
- A smaller non-DeepSeek model can still be used later for loader-plumbing smoke tests, but it does not replace this baseline.
|
separately versioned GPU/large-model contract or a diagnosed quality fix.
|
||||||
- Model artifacts must stay on the mounted drive and not under `/home`.
|
|
||||||
|
|
||||||
## Dependent-story handoff
|
|
||||||
|
|
||||||
Next implementation work should attach to this contract and add the live benchmark runner that actually compares:
|
|
||||||
|
|
||||||
1. current Transformers/safetensors recipe
|
|
||||||
2. whole-model llama.cpp GGUF recipe
|
|
||||||
|
|
||||||
using the same model architecture/revision and the same prompt/context/concurrency settings.
|
|
||||||
|
|||||||
169
.scratch/distributed-gguf-runtime/evidence/DGR-001/baseline.json
Normal file
169
.scratch/distributed-gguf-runtime/evidence/DGR-001/baseline.json
Normal file
@@ -0,0 +1,169 @@
|
|||||||
|
{
|
||||||
|
"artifact_sha256": {
|
||||||
|
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
|
||||||
|
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6"
|
||||||
|
},
|
||||||
|
"backend_detail": {
|
||||||
|
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||||
|
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||||
|
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16"
|
||||||
|
},
|
||||||
|
"evidence_class": "local-real",
|
||||||
|
"host": {
|
||||||
|
"accelerator_name": "Radeon 8060S Graphics",
|
||||||
|
"accelerator_runtime": "7.13.26183",
|
||||||
|
"benchmark_lane": "cpu-controlled-baseline",
|
||||||
|
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
|
||||||
|
"cpu_count": 32,
|
||||||
|
"cuda_available": true,
|
||||||
|
"hostname": "fedora",
|
||||||
|
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"llama_cpp_version": "9991",
|
||||||
|
"llama_server_identities": {
|
||||||
|
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
|
||||||
|
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
|
||||||
|
"python": "3.12.13",
|
||||||
|
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
|
||||||
|
"torch_version": "2.10.0+rocm7.13.0a20260513",
|
||||||
|
"transformers_version": "5.13.0"
|
||||||
|
},
|
||||||
|
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
|
||||||
|
"provenance": {
|
||||||
|
"completed_at": "2026-07-13T16:27:19.647692Z",
|
||||||
|
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||||
|
"producer": "meshnet_node.recipe_drivers.run_configured_benchmark/v1",
|
||||||
|
"run_id": "e4eedadf-22f6-4907-8990-985456961099",
|
||||||
|
"schema_version": 1,
|
||||||
|
"signature": "owev+/ToswP20C923G6E+srOCUBV5vrjmndVatr9CbTXakiFGqlHrTiEo+aymA4BcSwmG6KJTxlxO6WpLnpcAg==",
|
||||||
|
"signature_algorithm": "ed25519",
|
||||||
|
"signer_public_key_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
|
||||||
|
"started_at": "2026-07-13T16:26:22.361501Z"
|
||||||
|
},
|
||||||
|
"recipe_runtime": {
|
||||||
|
"llama-cpp-near-lossless-quality": {
|
||||||
|
"device": "cpu",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "bfloat16"
|
||||||
|
},
|
||||||
|
"llama-cpp-quantized-performance-fit": {
|
||||||
|
"device": "cpu",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "Q4_K_M"
|
||||||
|
},
|
||||||
|
"transformers-safetensors-reference": {
|
||||||
|
"device": "cpu",
|
||||||
|
"runtime": "transformers-5.13.0",
|
||||||
|
"weight_format": "safetensors",
|
||||||
|
"weight_quantization": "bfloat16"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipes": {
|
||||||
|
"llama-cpp-near-lossless-quality": {
|
||||||
|
"artifact_bytes": 994156448,
|
||||||
|
"available": true,
|
||||||
|
"concurrency": {
|
||||||
|
"1": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 86.7339,
|
||||||
|
"decode_tokens_per_sec": 98.5178,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 333.023,
|
||||||
|
"latency_p95_ms": 383.0597,
|
||||||
|
"peak_rss_bytes": 1110728704,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 1717.9451,
|
||||||
|
"ttft_p50_ms": 15.069,
|
||||||
|
"ttft_p95_ms": 63.766
|
||||||
|
},
|
||||||
|
"4": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 222.788,
|
||||||
|
"decode_tokens_per_sec": 76.6297,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 490.8738,
|
||||||
|
"latency_p95_ms": 646.26,
|
||||||
|
"peak_rss_bytes": 1139466240,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 859.8985,
|
||||||
|
"ttft_p50_ms": 32.445,
|
||||||
|
"ttft_p95_ms": 218.387
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"device": "cpu",
|
||||||
|
"lane": "quality"
|
||||||
|
},
|
||||||
|
"llama-cpp-quantized-performance-fit": {
|
||||||
|
"artifact_bytes": 397807520,
|
||||||
|
"available": true,
|
||||||
|
"concurrency": {
|
||||||
|
"1": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 139.2693,
|
||||||
|
"decode_tokens_per_sec": 207.712,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 168.3307,
|
||||||
|
"latency_p95_ms": 305.1338,
|
||||||
|
"peak_rss_bytes": 542081024,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 967.0195,
|
||||||
|
"ttft_p50_ms": 21.582,
|
||||||
|
"ttft_p95_ms": 147.859
|
||||||
|
},
|
||||||
|
"4": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 195.6789,
|
||||||
|
"decode_tokens_per_sec": 76.9497,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 437.9196,
|
||||||
|
"latency_p95_ms": 885.5355,
|
||||||
|
"peak_rss_bytes": 573259776,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 572.4424,
|
||||||
|
"ttft_p50_ms": 48.127,
|
||||||
|
"ttft_p95_ms": 416.531
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"device": "cpu",
|
||||||
|
"lane": "performance-fit"
|
||||||
|
},
|
||||||
|
"transformers-safetensors-reference": {
|
||||||
|
"artifact_bytes": 999586347,
|
||||||
|
"available": true,
|
||||||
|
"concurrency": {
|
||||||
|
"1": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 35.4722,
|
||||||
|
"decode_tokens_per_sec": 40.7545,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 818.3864,
|
||||||
|
"latency_p95_ms": 1258.0673,
|
||||||
|
"peak_rss_bytes": 1941458944,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 625.6467,
|
||||||
|
"ttft_p50_ms": 40.0018,
|
||||||
|
"ttft_p95_ms": 195.2551
|
||||||
|
},
|
||||||
|
"4": {
|
||||||
|
"aggregate_decode_tokens_per_sec": 46.5375,
|
||||||
|
"decode_tokens_per_sec": 12.9506,
|
||||||
|
"failures": 0,
|
||||||
|
"latency_p50_ms": 2481.8662,
|
||||||
|
"latency_p95_ms": 3365.8395,
|
||||||
|
"peak_rss_bytes": 2104832000,
|
||||||
|
"peak_vram_bytes": 0,
|
||||||
|
"prefill_tokens_per_sec": 264.0101,
|
||||||
|
"ttft_p50_ms": 97.0403,
|
||||||
|
"ttft_p95_ms": 429.0665
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"device": "cpu",
|
||||||
|
"lane": "quality"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"reference_recipe_id": "transformers-safetensors-reference"
|
||||||
|
}
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
{
|
||||||
|
"artifact_storage_root": "/run/media/popov/DATA/llm",
|
||||||
|
"evidence_class": "local-real",
|
||||||
|
"host": {
|
||||||
|
"benchmark_lane": "cpu-controlled-baseline",
|
||||||
|
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"llama_cpp_version": "9991",
|
||||||
|
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
|
||||||
|
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
|
||||||
|
"transformers_version": "5.13.0"
|
||||||
|
},
|
||||||
|
"plan": {
|
||||||
|
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
|
||||||
|
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"prompts": [
|
||||||
|
{
|
||||||
|
"id": "short-fact",
|
||||||
|
"text": "The capital of France is",
|
||||||
|
"context_class": "short"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "medium-code",
|
||||||
|
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
|
||||||
|
"context_class": "medium"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "long-summary",
|
||||||
|
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
|
||||||
|
"context_class": "long"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"sampling": {
|
||||||
|
"temperature": 0.0,
|
||||||
|
"top_p": 1.0,
|
||||||
|
"top_k": 1,
|
||||||
|
"seed": 1234,
|
||||||
|
"max_output_tokens": 32
|
||||||
|
},
|
||||||
|
"concurrency_levels": [1, 4],
|
||||||
|
"repeats": 3,
|
||||||
|
"warmup_requests": 2
|
||||||
|
},
|
||||||
|
"recipes": [
|
||||||
|
{
|
||||||
|
"id": "transformers-safetensors-reference",
|
||||||
|
"runtime": "transformers-5.13.0",
|
||||||
|
"weight_format": "safetensors",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"lane": "quality",
|
||||||
|
"device": "cpu",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": true,
|
||||||
|
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
|
||||||
|
"driver": {
|
||||||
|
"type": "transformers",
|
||||||
|
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"device": "cpu",
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"threads": 16
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "llama-cpp-near-lossless-quality",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"lane": "quality",
|
||||||
|
"device": "cpu",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||||
|
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": false,
|
||||||
|
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
|
||||||
|
"driver": {
|
||||||
|
"type": "llama-cpp-server",
|
||||||
|
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
|
||||||
|
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||||
|
"device": "cpu",
|
||||||
|
"threads": 16,
|
||||||
|
"n_parallel": 4,
|
||||||
|
"context_per_slot": 512,
|
||||||
|
"n_gpu_layers": 0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "llama-cpp-quantized-performance-fit",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "Q4_K_M",
|
||||||
|
"lane": "performance-fit",
|
||||||
|
"device": "cpu",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||||
|
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": false,
|
||||||
|
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
|
||||||
|
"driver": {
|
||||||
|
"type": "llama-cpp-server",
|
||||||
|
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
|
||||||
|
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||||
|
"device": "cpu",
|
||||||
|
"threads": 16,
|
||||||
|
"n_parallel": 4,
|
||||||
|
"context_per_slot": 512,
|
||||||
|
"n_gpu_layers": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# Exact source snapshot (already present on mounted storage)
|
||||||
|
SOURCE=/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775
|
||||||
|
LLAMA=/run/media/popov/d/DEV/llamacpp/llama.cpp
|
||||||
|
ROCM_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm/bin/python
|
||||||
|
PROJECT_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
|
||||||
|
OUT=/run/media/popov/DATA/llm/dgr-001
|
||||||
|
SIGNING_KEY=/home/popov/.config/neuron-tai/keys/dgr-001-evidence-ed25519.pem
|
||||||
|
|
||||||
|
# Private signing key is outside Git and must remain owner-only
|
||||||
|
stat -c '%a %n' "$SIGNING_KEY" # expected: 600
|
||||||
|
|
||||||
|
# Converter support check (no writes)
|
||||||
|
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" --dry-run
|
||||||
|
|
||||||
|
# Exact-revision near-lossless and performance-fit artifacts
|
||||||
|
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf"
|
||||||
|
$LLAMA/build/bin/llama-quantize "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf" Q4_K_M
|
||||||
|
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype bf16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf"
|
||||||
|
|
||||||
|
# Runtime and artifact identity
|
||||||
|
git -C "$LLAMA" rev-parse HEAD
|
||||||
|
$LLAMA/build/bin/llama-server --version
|
||||||
|
sha256sum "$LLAMA/build/bin/llama-server" "$LLAMA/convert_hf_to_gguf.py" "$LLAMA/build/bin/llama-quantize"
|
||||||
|
sha256sum "$SOURCE/model.safetensors" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf"
|
||||||
|
|
||||||
|
# Deterministic complete-snapshot digest used by benchmark-config.json
|
||||||
|
PYTHONPATH=packages/node $ROCM_PY - <<'PY'
|
||||||
|
from pathlib import Path
|
||||||
|
from meshnet_node.recipe_drivers import _artifact_sha256
|
||||||
|
print(_artifact_sha256(Path('/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775')))
|
||||||
|
PY
|
||||||
|
|
||||||
|
# Canonical opt-in local-real benchmark
|
||||||
|
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
|
||||||
|
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
|
||||||
|
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json \
|
||||||
|
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.json \
|
||||||
|
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt
|
||||||
|
|
||||||
|
# Distil the baseline and evaluate immutable v1
|
||||||
|
PYTHONPATH=packages/node $PROJECT_PY - <<'PY'
|
||||||
|
from pathlib import Path
|
||||||
|
import json
|
||||||
|
from meshnet_node.performance_contract import baseline_from_report, evaluate_contract, load_contract
|
||||||
|
root = Path('.scratch/distributed-gguf-runtime/evidence/DGR-001')
|
||||||
|
report = json.loads((root / 'results.json').read_text())
|
||||||
|
contract = load_contract(root / 'performance-contract.json')
|
||||||
|
(root / 'baseline.json').write_text(json.dumps(baseline_from_report(report), indent=2, sort_keys=True) + '\n')
|
||||||
|
(root / 'contract-evaluation.json').write_text(json.dumps(evaluate_contract(contract, report).to_dict(), indent=2, sort_keys=True) + '\n')
|
||||||
|
PY
|
||||||
|
|
||||||
|
# Optional ROCm GPU diagnostic (not eligible for immutable v1)
|
||||||
|
# The version-matched rocm[devel] wheel expands beyond 20 GB; ensure sufficient
|
||||||
|
# space or relocate its packaged payload before installation.
|
||||||
|
uv pip install --python "$ROCM_PY" --prerelease=allow \
|
||||||
|
--index-url https://rocm.nightlies.amd.com/v2/gfx1151/ \
|
||||||
|
'rocm[devel]==7.13.0a20260513'
|
||||||
|
ROCM_VENV=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm
|
||||||
|
ROCM_SDK="$ROCM_VENV/bin/rocm-sdk"
|
||||||
|
ROCM_ROOT="$($ROCM_SDK path --root)"
|
||||||
|
ROCM_BIN="$($ROCM_SDK path --bin)"
|
||||||
|
export PATH="$ROCM_VENV/bin:$ROCM_BIN:$PATH"
|
||||||
|
export ROCM_PATH="$ROCM_ROOT" HIP_PATH="$ROCM_ROOT"
|
||||||
|
export CMAKE_PREFIX_PATH="$($ROCM_SDK path --cmake):$ROCM_ROOT"
|
||||||
|
export LD_LIBRARY_PATH="$ROCM_ROOT/lib:$ROCM_ROOT/lib64:${LD_LIBRARY_PATH:-}"
|
||||||
|
$ROCM_VENV/bin/cmake -S /run/media/popov/d/DEV/llamacpp/llama.cpp \
|
||||||
|
-B /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip -G Ninja \
|
||||||
|
-DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
|
||||||
|
-DCMAKE_HIP_COMPILER="$ROCM_VENV/bin/amdclang++" \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=OFF \
|
||||||
|
-DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_BUILD_SERVER=ON
|
||||||
|
$ROCM_VENV/bin/cmake --build /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip \
|
||||||
|
--target llama-server llama-cli llama-bench -j 16
|
||||||
|
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
|
||||||
|
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
|
||||||
|
--profile gpu-diagnostic \
|
||||||
|
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-config.json \
|
||||||
|
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.json \
|
||||||
|
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.txt
|
||||||
|
PYTHONPATH=packages/node $PROJECT_PY \
|
||||||
|
.scratch/distributed-gguf-runtime/evidence/DGR-001/summarize-quality-parity.py
|
||||||
|
|
||||||
|
# Deterministic verification
|
||||||
|
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q tests/test_recipe_benchmark.py
|
||||||
|
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q
|
||||||
|
PYTHONPATH=packages/node $PROJECT_PY -m compileall -q packages tests
|
||||||
|
git diff --check
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
{
|
||||||
|
"contract_version": 1,
|
||||||
|
"fit_benefit": true,
|
||||||
|
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
|
||||||
|
"quality_lane_pass": false,
|
||||||
|
"rationale": [
|
||||||
|
"the near-lossless quality lane failed: the GGUF runtime disagrees with the safetensors reference beyond what near-lossless weights can explain",
|
||||||
|
"a meaningful speed benefit was measured",
|
||||||
|
"a meaningful fit benefit was measured"
|
||||||
|
],
|
||||||
|
"recipes": [
|
||||||
|
{
|
||||||
|
"comparable": true,
|
||||||
|
"failures": 0,
|
||||||
|
"fit_benefit": false,
|
||||||
|
"incomparable_reason": "",
|
||||||
|
"lane": "quality",
|
||||||
|
"measurements": {
|
||||||
|
"aggregate_concurrency": 4,
|
||||||
|
"aggregate_throughput_speedup": 4.7873,
|
||||||
|
"artifact_size_ratio": 0.9946,
|
||||||
|
"artifact_size_win": false,
|
||||||
|
"compared_prompts": 3,
|
||||||
|
"decode_speedup": 2.4173,
|
||||||
|
"exact_match_rate": 0.3333,
|
||||||
|
"expected_prompts": 3,
|
||||||
|
"failure_rate": 0.0,
|
||||||
|
"mean_similarity": 0.9471,
|
||||||
|
"resident_memory_ratio": 0.5721,
|
||||||
|
"ttft_ratio": 0.3767
|
||||||
|
},
|
||||||
|
"quality_pass": false,
|
||||||
|
"reasons": [
|
||||||
|
"single-request decode 2.42x reference (>= 1.25x) at TTFT ratio 0.38",
|
||||||
|
"aggregate throughput at concurrency 4 is 4.79x reference (>= 1.25x)",
|
||||||
|
"peak resident memory is 0.57x reference (<= 0.75x)",
|
||||||
|
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
|
||||||
|
],
|
||||||
|
"recipe_id": "llama-cpp-near-lossless-quality",
|
||||||
|
"speed_benefit": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"comparable": true,
|
||||||
|
"failures": 0,
|
||||||
|
"fit_benefit": true,
|
||||||
|
"incomparable_reason": "",
|
||||||
|
"lane": "performance-fit",
|
||||||
|
"measurements": {
|
||||||
|
"aggregate_concurrency": 4,
|
||||||
|
"aggregate_throughput_speedup": 4.2048,
|
||||||
|
"artifact_size_ratio": 0.398,
|
||||||
|
"artifact_size_win": true,
|
||||||
|
"decode_speedup": 5.0967,
|
||||||
|
"failure_rate": 0.0,
|
||||||
|
"resident_memory_ratio": 0.2792,
|
||||||
|
"ttft_ratio": 0.5395
|
||||||
|
},
|
||||||
|
"quality_pass": null,
|
||||||
|
"reasons": [
|
||||||
|
"single-request decode 5.10x reference (>= 1.25x) at TTFT ratio 0.54",
|
||||||
|
"aggregate throughput at concurrency 4 is 4.20x reference (>= 1.25x)",
|
||||||
|
"peak resident memory is 0.28x reference (<= 0.75x)"
|
||||||
|
],
|
||||||
|
"recipe_id": "llama-cpp-quantized-performance-fit",
|
||||||
|
"speed_benefit": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"speed_benefit": true,
|
||||||
|
"stop_condition_met": true,
|
||||||
|
"verdict": "stop"
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
{
|
||||||
|
"artifact_storage_root": "/run/media/popov/DATA/llm",
|
||||||
|
"evidence_class": "local-real",
|
||||||
|
"host": {
|
||||||
|
"benchmark_lane": "rocm-gpu-diagnostic",
|
||||||
|
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"llama_cpp_version": "9991",
|
||||||
|
"llama_server_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||||
|
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
|
||||||
|
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
|
||||||
|
"transformers_version": "5.13.0",
|
||||||
|
"rocm_target": "gfx1151"
|
||||||
|
},
|
||||||
|
"plan": {
|
||||||
|
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
|
||||||
|
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"prompts": [
|
||||||
|
{
|
||||||
|
"id": "short-fact",
|
||||||
|
"text": "The capital of France is",
|
||||||
|
"context_class": "short"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "medium-code",
|
||||||
|
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
|
||||||
|
"context_class": "medium"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "long-summary",
|
||||||
|
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
|
||||||
|
"context_class": "long"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"sampling": {
|
||||||
|
"temperature": 0.0,
|
||||||
|
"top_p": 1.0,
|
||||||
|
"top_k": 1,
|
||||||
|
"seed": 1234,
|
||||||
|
"max_output_tokens": 32
|
||||||
|
},
|
||||||
|
"concurrency_levels": [
|
||||||
|
1,
|
||||||
|
4
|
||||||
|
],
|
||||||
|
"repeats": 3,
|
||||||
|
"warmup_requests": 2
|
||||||
|
},
|
||||||
|
"recipes": [
|
||||||
|
{
|
||||||
|
"id": "transformers-fp32-rocm-quality-oracle",
|
||||||
|
"runtime": "transformers-5.13.0-rocm-float32",
|
||||||
|
"weight_format": "safetensors",
|
||||||
|
"weight_quantization": "bfloat16-weights-float32-accumulation",
|
||||||
|
"lane": "quality",
|
||||||
|
"device": "cuda",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": true,
|
||||||
|
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
|
||||||
|
"driver": {
|
||||||
|
"type": "transformers",
|
||||||
|
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"device": "cuda",
|
||||||
|
"dtype": "float32",
|
||||||
|
"threads": 16
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "llama-cpp-bf16-rocm-quality",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"lane": "quality",
|
||||||
|
"device": "cuda",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||||
|
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": false,
|
||||||
|
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
|
||||||
|
"driver": {
|
||||||
|
"type": "llama-cpp-server",
|
||||||
|
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
|
||||||
|
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||||
|
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||||
|
"device": "cuda",
|
||||||
|
"threads": 16,
|
||||||
|
"n_parallel": 4,
|
||||||
|
"context_per_slot": 512,
|
||||||
|
"n_gpu_layers": 99
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "transformers-bf16-rocm-throughput",
|
||||||
|
"runtime": "transformers-5.13.0-rocm-bfloat16",
|
||||||
|
"weight_format": "safetensors",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"lane": "performance-fit",
|
||||||
|
"device": "cuda",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": false,
|
||||||
|
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
|
||||||
|
"driver": {
|
||||||
|
"type": "transformers",
|
||||||
|
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"device": "cuda",
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"threads": 16
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "llama-cpp-q4-rocm-throughput",
|
||||||
|
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "Q4_K_M",
|
||||||
|
"lane": "performance-fit",
|
||||||
|
"device": "cuda",
|
||||||
|
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||||
|
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
|
||||||
|
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"is_reference": false,
|
||||||
|
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
|
||||||
|
"driver": {
|
||||||
|
"type": "llama-cpp-server",
|
||||||
|
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
|
||||||
|
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||||
|
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||||
|
"device": "cuda",
|
||||||
|
"threads": 16,
|
||||||
|
"n_parallel": 4,
|
||||||
|
"context_per_slot": 512,
|
||||||
|
"n_gpu_layers": 99
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
|||||||
|
Recipe benchmark dgr-001-rocm-gpu-diagnostic-v1 (local-real)
|
||||||
|
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
|
||||||
|
transformers-fp32-rocm-quality-oracle [quality ] c= 1 ttft p50/p95 11.0/ 35.5 ms; prefill 5746.7 tok/s; decode 35.7 tok/s; aggregate 29.6 tok/s; rss 1.39 GB; vram 2.26 GB; artifact 1.00 GB; failures 0
|
||||||
|
transformers-fp32-rocm-quality-oracle [quality ] c= 4 ttft p50/p95 27.5/ 80.4 ms; prefill 1985.4 tok/s; decode 9.4 tok/s; aggregate 35.4 tok/s; rss 1.39 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||||
|
llama-cpp-bf16-rocm-quality [quality ] c= 1 ttft p50/p95 13.2/ 83.4 ms; prefill 4154.4 tok/s; decode 148.0 tok/s; aggregate 127.4 tok/s; rss 0.84 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||||
|
llama-cpp-bf16-rocm-quality [quality ] c= 4 ttft p50/p95 25.1/ 52.1 ms; prefill 2205.4 tok/s; decode 115.1 tok/s; aggregate 337.1 tok/s; rss 0.86 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||||
|
transformers-bf16-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 13.8/ 22.2 ms; prefill 4787.3 tok/s; decode 81.1 tok/s; aggregate 73.5 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||||
|
transformers-bf16-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 29.7/ 58.5 ms; prefill 2666.5 tok/s; decode 24.4 tok/s; aggregate 91.2 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||||
|
llama-cpp-q4-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 11.8/ 37.1 ms; prefill 4219.3 tok/s; decode 251.2 tok/s; aggregate 200.1 tok/s; rss 0.69 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||||
|
llama-cpp-q4-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 21.4/ 101.0 ms; prefill 2126.9 tok/s; decode 189.7 tok/s; aggregate 511.3 tok/s; rss 0.72 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||||
|
drift llama-cpp-bf16-rocm-quality vs transformers-fp32-rocm-quality-oracle exact 1.00; similarity 1.000 (gated)
|
||||||
|
drift transformers-bf16-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.33; similarity 0.946 (advisory)
|
||||||
|
drift llama-cpp-q4-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.00; similarity 0.628 (advisory)
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Observed pre-existing intermittent tracker race
|
||||||
|
|
||||||
|
This file records an unrelated timing observation and its repeated reproduction;
|
||||||
|
it is **not** a DGR-001 benchmark/contract failure.
|
||||||
|
|
||||||
|
Test:
|
||||||
|
|
||||||
|
```text
|
||||||
|
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
```
|
||||||
|
|
||||||
|
One earlier full-suite run produced:
|
||||||
|
|
||||||
|
```text
|
||||||
|
1 failed, 745 passed, 13 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
A five-run isolated retry matrix reproduced the same rate repeatedly:
|
||||||
|
|
||||||
|
```text
|
||||||
|
current DGR-001 branch: 4/5 passed, 1/5 failed
|
||||||
|
clean d904c40: 4/5 passed, 1/5 failed
|
||||||
|
```
|
||||||
|
|
||||||
|
An earlier full-suite run on the signed-provenance DGR-001 state completed
|
||||||
|
green:
|
||||||
|
|
||||||
|
```text
|
||||||
|
751 passed, 13 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
Two full-suite runs after adding the isolated GPU diagnostic profile each hit
|
||||||
|
the same race and otherwise passed:
|
||||||
|
|
||||||
|
```text
|
||||||
|
1 failed, 750 passed, 13 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
The latest expanded hardening suite hit the same race and otherwise passed:
|
||||||
|
|
||||||
|
```text
|
||||||
|
1 failed, 754 passed, 13 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
The final hardened state subsequently completed a full green run:
|
||||||
|
|
||||||
|
```text
|
||||||
|
755 passed, 13 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
In each failure, the mock upstream's three-second release timeout completed the
|
||||||
|
stream before the cancel POST, so the request was already absent and the cancel
|
||||||
|
endpoint returned 404. No tracker/proxy file changed in DGR-001. The race is
|
||||||
|
therefore timing-sensitive, pre-existing, and unrelated to the benchmark,
|
||||||
|
provenance, or GPU-diagnostic code.
|
||||||
@@ -1,75 +1,87 @@
|
|||||||
{
|
{
|
||||||
"benchmark_lanes": [
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "cpu",
|
|
||||||
"id": "transformers-safetensors-cpu",
|
|
||||||
"recipe": "current safetensors recipe",
|
|
||||||
"runtime": "transformers"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "cpu",
|
|
||||||
"id": "llama-cpp-gguf-cpu",
|
|
||||||
"recipe": "whole-model GGUF recipe",
|
|
||||||
"runtime": "llama.cpp"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "gpu",
|
|
||||||
"id": "transformers-safetensors-gpu",
|
|
||||||
"recipe": "current safetensors recipe",
|
|
||||||
"runtime": "transformers"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "gpu",
|
|
||||||
"id": "llama-cpp-gguf-gpu",
|
|
||||||
"recipe": "whole-model GGUF recipe",
|
|
||||||
"runtime": "llama.cpp"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"metrics": [
|
|
||||||
"ttft_ms",
|
|
||||||
"prefill_tok_per_sec",
|
|
||||||
"decode_tok_per_sec",
|
|
||||||
"p50_latency_ms",
|
|
||||||
"p95_latency_ms",
|
|
||||||
"aggregate_throughput_tok_per_sec",
|
|
||||||
"rss_bytes",
|
|
||||||
"vram_bytes",
|
|
||||||
"artifact_bytes",
|
|
||||||
"failure_count",
|
|
||||||
"output_drift"
|
|
||||||
],
|
|
||||||
"model_target": {
|
|
||||||
"architecture": "deepseek2",
|
|
||||||
"comparison_policy": "same model/revision, closest practical low-footprint precision pair: BF16 safetensors versus Q2_K GGUF",
|
|
||||||
"gguf_quant": "Q2_K",
|
|
||||||
"gguf_repo": "second-state/DeepSeek-V2-Lite-Chat-GGUF",
|
|
||||||
"gguf_size_gb": 6.43,
|
|
||||||
"name": "DeepSeek-V2-Lite-Chat",
|
|
||||||
"rationale": "Smallest DeepSeek-family benchmark anchor that still points toward DeepSeek-V4-Flash; keeps the runtime on the DeepSeek2 path instead of falling back to a tiny but architecture-mismatched smoke model.",
|
|
||||||
"safetensors_precision": "bfloat16",
|
|
||||||
"safetensors_repo": "deepseek-ai/DeepSeek-V2-Lite-Chat"
|
|
||||||
},
|
|
||||||
"notes": [
|
|
||||||
"Real model execution stays opt-in and must keep model artifacts on the mounted drive.",
|
|
||||||
"Use the tiny fallback only for loader plumbing smoke tests; it does not replace the architecture-aligned baseline."
|
|
||||||
],
|
|
||||||
"schema_version": 1,
|
"schema_version": 1,
|
||||||
"stop_condition": "Stop if GGUF does not provide a meaningful speed or fit benefit over the safetensors baseline for the chosen DeepSeek-family model target.",
|
"contract_version": 1,
|
||||||
"story_id": "DGR-001"
|
"locked_at": "2026-07-13T00:00:00Z",
|
||||||
|
"locked_by": "DGR-001",
|
||||||
|
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
|
||||||
|
"thresholds": {
|
||||||
|
"min_decode_speedup": 1.25,
|
||||||
|
"max_ttft_ratio": 1.25,
|
||||||
|
"min_aggregate_throughput_speedup": 1.25,
|
||||||
|
"max_resident_memory_ratio": 0.75,
|
||||||
|
"max_artifact_size_ratio": 0.6,
|
||||||
|
"min_quality_exact_match_rate": 0.9,
|
||||||
|
"min_quality_mean_similarity": 0.97,
|
||||||
|
"max_failure_rate": 0.0
|
||||||
|
},
|
||||||
|
"baseline": {
|
||||||
|
"status": "pending-real-evidence",
|
||||||
|
"required_evidence_class": "local-real",
|
||||||
|
"required_recipes": [
|
||||||
|
"transformers-safetensors-reference",
|
||||||
|
"llama-cpp-near-lossless-quality",
|
||||||
|
"llama-cpp-quantized-performance-fit"
|
||||||
|
],
|
||||||
|
"required_concurrency_levels": [
|
||||||
|
1,
|
||||||
|
4
|
||||||
|
],
|
||||||
|
"required_controlled_variables": [
|
||||||
|
"model architecture",
|
||||||
|
"model revision",
|
||||||
|
"machine and device",
|
||||||
|
"formatted prompts and context lengths",
|
||||||
|
"output length and greedy sampling policy"
|
||||||
|
],
|
||||||
|
"required_plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
|
||||||
|
"minimum_prompt_count": 3,
|
||||||
|
"minimum_repeats": 3,
|
||||||
|
"minimum_output_tokens": 32,
|
||||||
|
"required_device": "cpu",
|
||||||
|
"required_config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||||
|
"required_signer_public_key": "zQ/qRMwF/ydazzaxEI24Xvnrl5bZxzw16JYpP0bfRuI=",
|
||||||
|
"required_artifact_sha256": {
|
||||||
|
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||||
|
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5"
|
||||||
|
},
|
||||||
|
"required_recipe_runtime": {
|
||||||
|
"transformers-safetensors-reference": {
|
||||||
|
"runtime": "transformers-5.13.0",
|
||||||
|
"weight_format": "safetensors",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"device": "cpu"
|
||||||
|
},
|
||||||
|
"llama-cpp-near-lossless-quality": {
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "bfloat16",
|
||||||
|
"device": "cpu"
|
||||||
|
},
|
||||||
|
"llama-cpp-quantized-performance-fit": {
|
||||||
|
"runtime": "llama.cpp-9991-e920c523",
|
||||||
|
"weight_format": "gguf",
|
||||||
|
"weight_quantization": "Q4_K_M",
|
||||||
|
"device": "cpu"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required_backend_detail": {
|
||||||
|
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16",
|
||||||
|
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||||
|
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0"
|
||||||
|
},
|
||||||
|
"required_host_identity": {
|
||||||
|
"python": "3.12.13",
|
||||||
|
"torch_version": "2.10.0+rocm7.13.0a20260513",
|
||||||
|
"transformers_version": "5.13.0",
|
||||||
|
"llama_server_identities": {
|
||||||
|
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
|
||||||
|
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||||
|
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
|
||||||
|
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract. Non-synthetic evidence must be Ed25519-signed by the pinned key and match the exact locked config, artifacts, runtimes, backends, and host runtime identity."
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
{
|
||||||
|
"conclusion": {
|
||||||
|
"conversion_corruption_observed_in_rocm_sample": false,
|
||||||
|
"cpu_bf16_divergence_explained": false,
|
||||||
|
"recommended_v2_design": "Predeclare a float32 quality oracle separately from the BF16 performance reference, with a larger prompt corpus and immutable thresholds.",
|
||||||
|
"scope": "The ROCm diagnostic establishes only that the same BF16 GGUF artifact matched the float32 oracle for three GPU sequences; it does not explain the CPU BF16 divergence or prove global conversion correctness.",
|
||||||
|
"v1_verdict_changed": false
|
||||||
|
},
|
||||||
|
"cpu_v1": {
|
||||||
|
"candidate": "llama.cpp BF16 GGUF",
|
||||||
|
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||||
|
"contract_verdict": "stop",
|
||||||
|
"device": "cpu",
|
||||||
|
"exact_match_rate": 0.3333,
|
||||||
|
"mean_similarity": 0.9471,
|
||||||
|
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
|
||||||
|
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
|
||||||
|
"quality_oracle": "Transformers BF16 safetensors",
|
||||||
|
"report": "results.json",
|
||||||
|
"report_sha256": "5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5",
|
||||||
|
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
|
||||||
|
"run_id": "e4eedadf-22f6-4907-8990-985456961099"
|
||||||
|
},
|
||||||
|
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||||
|
"rocm_diagnostic": {
|
||||||
|
"candidate": "llama.cpp BF16 GGUF",
|
||||||
|
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||||
|
"config_sha256": "b0f0c846c818f1307d034cee1f81daa311efc20985c32a4cdbbbd8ffe4153892",
|
||||||
|
"device": "cuda (ROCm)",
|
||||||
|
"exact_match_rate": 1.0,
|
||||||
|
"failures": 0,
|
||||||
|
"mean_similarity": 1.0,
|
||||||
|
"measured_backend_detail": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139; threads 16; parallel slots 4; ctx/slot 512; requested gpu layers 99; measured accelerator ROCm0: Radeon 8060S Graphics; measured offload 25/25 layers",
|
||||||
|
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
|
||||||
|
"plan_sha256": "dae8e40963588f71f5d201fd163d39bd762e392544b5603d483e90d21abee2e8",
|
||||||
|
"producer": "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1",
|
||||||
|
"quality_oracle": "Transformers float32 safetensors",
|
||||||
|
"report": "gpu-diagnostic-results.json",
|
||||||
|
"report_sha256": "527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d",
|
||||||
|
"run_id": "31bf44e7-ccd4-4277-84ac-c775dee65411",
|
||||||
|
"signer_fingerprint": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
|
||||||
|
"v1_eligible": false
|
||||||
|
},
|
||||||
|
"schema_version": 2
|
||||||
|
}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
# DGR-001 quality-parity evidence summary
|
||||||
|
|
||||||
|
This summary is generated by `summarize-quality-parity.py` from signed reports.
|
||||||
|
It contains no independent logit measurements or self-asserted verification flag.
|
||||||
|
|
||||||
|
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|
||||||
|
|---|---|---|---|---:|---:|---|
|
||||||
|
| CPU v1 (`e4eedadf-22f6-4907-8990-985456961099`) | CPU | Transformers BF16 | llama.cpp BF16 | 0.3333 | 0.9471 | immutable `stop` |
|
||||||
|
| ROCm diagnostic (`31bf44e7-ccd4-4277-84ac-c775dee65411`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | 1.0000 | 1.0000 | diagnostic only |
|
||||||
|
|
||||||
|
## Interpretation
|
||||||
|
|
||||||
|
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
|
||||||
|
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
|
||||||
|
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
|
||||||
|
matched the float32 oracle for all three GPU sequences with zero failures.
|
||||||
|
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
|
||||||
|
`25/25` layer offload.
|
||||||
|
|
||||||
|
No conversion corruption was observed in that three-sequence ROCm sample. This
|
||||||
|
does not prove global conversion correctness and does not retroactively change
|
||||||
|
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
|
||||||
|
separately from its BF16 performance reference and use a larger corpus.
|
||||||
|
|
||||||
|
## Reproduction and bindings
|
||||||
|
|
||||||
|
- CPU report SHA-256: `5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5`
|
||||||
|
- GPU report SHA-256: `527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d`
|
||||||
|
- BF16 GGUF SHA-256: `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862`
|
||||||
|
- Signer fingerprint: `8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de`
|
||||||
|
- Exact verification command: see `commands.txt`.
|
||||||
2491
.scratch/distributed-gguf-runtime/evidence/DGR-001/results.json
Normal file
2491
.scratch/distributed-gguf-runtime/evidence/DGR-001/results.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,10 @@
|
|||||||
|
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
|
||||||
|
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
|
||||||
|
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 40.0/ 195.3 ms; prefill 625.6 tok/s; decode 40.8 tok/s; aggregate 35.5 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||||
|
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 97.0/ 429.1 ms; prefill 264.0 tok/s; decode 13.0 tok/s; aggregate 46.5 tok/s; rss 2.10 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||||
|
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 15.1/ 63.8 ms; prefill 1717.9 tok/s; decode 98.5 tok/s; aggregate 86.7 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||||
|
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 32.4/ 218.4 ms; prefill 859.9 tok/s; decode 76.6 tok/s; aggregate 222.8 tok/s; rss 1.14 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||||
|
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 21.6/ 147.9 ms; prefill 967.0 tok/s; decode 207.7 tok/s; aggregate 139.3 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||||
|
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 48.1/ 416.5 ms; prefill 572.4 tok/s; decode 76.9 tok/s; aggregate 195.7 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||||
|
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
|
||||||
|
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)
|
||||||
@@ -1,247 +0,0 @@
|
|||||||
{
|
|
||||||
"comparisons": {
|
|
||||||
"cpu": {
|
|
||||||
"artifact_bytes_ratio": 0.2048,
|
|
||||||
"decode_speedup": 2.3333,
|
|
||||||
"gguf_benefit": true,
|
|
||||||
"gguf_lane": "llama-cpp-gguf-cpu",
|
|
||||||
"memory_bytes_ratio": 0.2152,
|
|
||||||
"memory_metric": "rss_bytes",
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"safetensors_lane": "transformers-safetensors-cpu",
|
|
||||||
"ttft_speedup": 1.8947
|
|
||||||
},
|
|
||||||
"gpu": {
|
|
||||||
"artifact_bytes_ratio": 0.2048,
|
|
||||||
"decode_speedup": 1.5294,
|
|
||||||
"gguf_benefit": true,
|
|
||||||
"gguf_lane": "llama-cpp-gguf-gpu",
|
|
||||||
"memory_bytes_ratio": 0.2273,
|
|
||||||
"memory_metric": "vram_bytes",
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"safetensors_lane": "transformers-safetensors-gpu",
|
|
||||||
"ttft_speedup": 1.6154
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"lanes": [
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "cpu",
|
|
||||||
"id": "transformers-safetensors-cpu",
|
|
||||||
"output_tokens": [
|
|
||||||
"mesh",
|
|
||||||
"activation",
|
|
||||||
"seam",
|
|
||||||
"baseline"
|
|
||||||
],
|
|
||||||
"recipe": "current safetensors recipe",
|
|
||||||
"results": [
|
|
||||||
{
|
|
||||||
"concurrency": 1,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 6.0,
|
|
||||||
"artifact_bytes": 33715493273,
|
|
||||||
"decode_tok_per_sec": 6.0,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 166.6667,
|
|
||||||
"p95_latency_ms": 208.3334,
|
|
||||||
"prefill_tok_per_sec": 45.0,
|
|
||||||
"rss_bytes": 35433480192,
|
|
||||||
"ttft_ms": 1800.0,
|
|
||||||
"vram_bytes": 0
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency": 4,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 20.4,
|
|
||||||
"artifact_bytes": 33715493273,
|
|
||||||
"decode_tok_per_sec": 5.1,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 196.0784,
|
|
||||||
"p95_latency_ms": 245.098,
|
|
||||||
"prefill_tok_per_sec": 38.25,
|
|
||||||
"rss_bytes": 35433480192,
|
|
||||||
"ttft_ms": 2340.0,
|
|
||||||
"vram_bytes": 0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"runtime": "transformers"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "cpu",
|
|
||||||
"id": "llama-cpp-gguf-cpu",
|
|
||||||
"output_tokens": [
|
|
||||||
"mesh",
|
|
||||||
"activation",
|
|
||||||
"seam",
|
|
||||||
"baseline"
|
|
||||||
],
|
|
||||||
"recipe": "whole-model GGUF recipe",
|
|
||||||
"results": [
|
|
||||||
{
|
|
||||||
"concurrency": 1,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 14.0,
|
|
||||||
"artifact_bytes": 6904159928,
|
|
||||||
"decode_tok_per_sec": 14.0,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 71.4286,
|
|
||||||
"p95_latency_ms": 89.2858,
|
|
||||||
"prefill_tok_per_sec": 90.0,
|
|
||||||
"rss_bytes": 7623566950,
|
|
||||||
"ttft_ms": 950.0,
|
|
||||||
"vram_bytes": 0
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency": 4,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 47.6,
|
|
||||||
"artifact_bytes": 6904159928,
|
|
||||||
"decode_tok_per_sec": 11.9,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 84.0336,
|
|
||||||
"p95_latency_ms": 105.042,
|
|
||||||
"prefill_tok_per_sec": 76.5,
|
|
||||||
"rss_bytes": 7623566950,
|
|
||||||
"ttft_ms": 1235.0,
|
|
||||||
"vram_bytes": 0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"runtime": "llama.cpp"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "gpu",
|
|
||||||
"id": "transformers-safetensors-gpu",
|
|
||||||
"output_tokens": [
|
|
||||||
"mesh",
|
|
||||||
"activation",
|
|
||||||
"seam",
|
|
||||||
"baseline"
|
|
||||||
],
|
|
||||||
"recipe": "current safetensors recipe",
|
|
||||||
"results": [
|
|
||||||
{
|
|
||||||
"concurrency": 1,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 34.0,
|
|
||||||
"artifact_bytes": 33715493273,
|
|
||||||
"decode_tok_per_sec": 34.0,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 29.4118,
|
|
||||||
"p95_latency_ms": 36.7647,
|
|
||||||
"prefill_tok_per_sec": 850.0,
|
|
||||||
"rss_bytes": 4294967296,
|
|
||||||
"ttft_ms": 420.0,
|
|
||||||
"vram_bytes": 35433480192
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency": 4,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 115.6,
|
|
||||||
"artifact_bytes": 33715493273,
|
|
||||||
"decode_tok_per_sec": 28.9,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 34.6021,
|
|
||||||
"p95_latency_ms": 43.2526,
|
|
||||||
"prefill_tok_per_sec": 722.5,
|
|
||||||
"rss_bytes": 4294967296,
|
|
||||||
"ttft_ms": 546.0,
|
|
||||||
"vram_bytes": 35433480192
|
|
||||||
}
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"runtime": "transformers"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency_levels": [
|
|
||||||
1,
|
|
||||||
4
|
|
||||||
],
|
|
||||||
"device": "gpu",
|
|
||||||
"id": "llama-cpp-gguf-gpu",
|
|
||||||
"output_tokens": [
|
|
||||||
"mesh",
|
|
||||||
"activation",
|
|
||||||
"seam",
|
|
||||||
"baseline"
|
|
||||||
],
|
|
||||||
"recipe": "whole-model GGUF recipe",
|
|
||||||
"results": [
|
|
||||||
{
|
|
||||||
"concurrency": 1,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 52.0,
|
|
||||||
"artifact_bytes": 6904159928,
|
|
||||||
"decode_tok_per_sec": 52.0,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 19.2308,
|
|
||||||
"p95_latency_ms": 24.0385,
|
|
||||||
"prefill_tok_per_sec": 640.0,
|
|
||||||
"rss_bytes": 1610612736,
|
|
||||||
"ttft_ms": 260.0,
|
|
||||||
"vram_bytes": 8053063680
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"concurrency": 4,
|
|
||||||
"metrics": {
|
|
||||||
"aggregate_throughput_tok_per_sec": 176.8,
|
|
||||||
"artifact_bytes": 6904159928,
|
|
||||||
"decode_tok_per_sec": 44.2,
|
|
||||||
"failure_count": 0,
|
|
||||||
"output_drift": 0.0,
|
|
||||||
"p50_latency_ms": 22.6244,
|
|
||||||
"p95_latency_ms": 28.2805,
|
|
||||||
"prefill_tok_per_sec": 544.0,
|
|
||||||
"rss_bytes": 1610612736,
|
|
||||||
"ttft_ms": 338.0,
|
|
||||||
"vram_bytes": 8053063680
|
|
||||||
}
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"runtime": "llama.cpp"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"model_target": {
|
|
||||||
"architecture": "deepseek2",
|
|
||||||
"comparison_policy": "same model/revision, closest practical low-footprint precision pair: BF16 safetensors versus Q2_K GGUF",
|
|
||||||
"gguf_quant": "Q2_K",
|
|
||||||
"gguf_repo": "second-state/DeepSeek-V2-Lite-Chat-GGUF",
|
|
||||||
"gguf_size_gb": 6.43,
|
|
||||||
"name": "DeepSeek-V2-Lite-Chat",
|
|
||||||
"rationale": "Smallest DeepSeek-family benchmark anchor that still points toward DeepSeek-V4-Flash; keeps the runtime on the DeepSeek2 path instead of falling back to a tiny but architecture-mismatched smoke model.",
|
|
||||||
"safetensors_precision": "bfloat16",
|
|
||||||
"safetensors_repo": "deepseek-ai/DeepSeek-V2-Lite-Chat"
|
|
||||||
},
|
|
||||||
"schema_version": 1,
|
|
||||||
"source": "stub-backend",
|
|
||||||
"stop_condition": {
|
|
||||||
"gguf_benefit": true,
|
|
||||||
"text": "Stop if GGUF does not provide a meaningful speed or fit benefit over the safetensors baseline for the chosen DeepSeek-family model target.",
|
|
||||||
"triggered": false
|
|
||||||
},
|
|
||||||
"story_id": "DGR-001"
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,261 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Build the DGR-001 parity summary from cryptographically verified reports."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
|
||||||
|
|
||||||
|
from meshnet_node.performance_contract import (
|
||||||
|
_canonical_sha256,
|
||||||
|
evaluate_contract,
|
||||||
|
load_contract,
|
||||||
|
report_signing_payload,
|
||||||
|
)
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent
|
||||||
|
|
||||||
|
|
||||||
|
def _read(name: str) -> dict:
|
||||||
|
return json.loads((ROOT / name).read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _file_sha256(name: str) -> str:
|
||||||
|
return hashlib.sha256((ROOT / name).read_bytes()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _drift(report: dict, recipe_id: str) -> dict:
|
||||||
|
return next(item for item in report["drift"] if item["recipe_id"] == recipe_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _recipe(report: dict, recipe_id: str) -> dict:
|
||||||
|
return next(item for item in report["recipes"] if item["recipe"]["id"] == recipe_id)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
contract = load_contract(ROOT / "performance-contract.json")
|
||||||
|
cpu_report = _read("results.json")
|
||||||
|
gpu_config = _read("gpu-diagnostic-config.json")
|
||||||
|
gpu_report = _read("gpu-diagnostic-results.json")
|
||||||
|
|
||||||
|
cpu_evaluation = evaluate_contract(contract, cpu_report)
|
||||||
|
if cpu_evaluation.verdict != "stop":
|
||||||
|
raise RuntimeError("immutable CPU v1 evidence no longer evaluates to stop")
|
||||||
|
|
||||||
|
public_key_bytes = base64.b64decode(contract.baseline["required_signer_public_key"])
|
||||||
|
public_key = Ed25519PublicKey.from_public_bytes(public_key_bytes)
|
||||||
|
public_key.verify(
|
||||||
|
base64.b64decode(gpu_report["provenance"]["signature"]),
|
||||||
|
report_signing_payload(gpu_report),
|
||||||
|
)
|
||||||
|
signer_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
|
||||||
|
if gpu_report["provenance"]["signer_public_key_sha256"] != signer_fingerprint:
|
||||||
|
raise RuntimeError("GPU report signer fingerprint does not match the contract trust key")
|
||||||
|
if gpu_report["provenance"]["config_sha256"] != _canonical_sha256(gpu_config):
|
||||||
|
raise RuntimeError("GPU report is not bound to gpu-diagnostic-config.json")
|
||||||
|
|
||||||
|
if gpu_report.get("schema_version") != 1 or gpu_report.get("evidence_class") != "local-real":
|
||||||
|
raise RuntimeError("GPU report must be schema-v1 local-real evidence")
|
||||||
|
expected_producer = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
|
||||||
|
if gpu_report["provenance"].get("producer") != expected_producer:
|
||||||
|
raise RuntimeError("GPU report was not emitted by the canonical diagnostic producer")
|
||||||
|
if gpu_report.get("reference_recipe_id") != "transformers-fp32-rocm-quality-oracle":
|
||||||
|
raise RuntimeError("GPU report uses the wrong quality reference")
|
||||||
|
if gpu_report.get("host", {}).get("benchmark_lane") != "rocm-gpu-diagnostic":
|
||||||
|
raise RuntimeError("GPU report lacks the diagnostic host marker")
|
||||||
|
|
||||||
|
trusted = json.loads(
|
||||||
|
(ROOT.parents[1] / "trusted-evidence-signers.json").read_text(encoding="utf-8")
|
||||||
|
)
|
||||||
|
if not any(
|
||||||
|
signer.get("algorithm") == "ed25519"
|
||||||
|
and signer.get("fingerprint_sha256") == signer_fingerprint
|
||||||
|
and signer.get("status") == "active"
|
||||||
|
for signer in trusted.get("signers", ())
|
||||||
|
):
|
||||||
|
raise RuntimeError("GPU signer is not active in the trusted-signers registry")
|
||||||
|
|
||||||
|
for field in ("model_id", "model_revision"):
|
||||||
|
if gpu_report["plan"].get(field) != cpu_report["plan"].get(field):
|
||||||
|
raise RuntimeError(f"CPU and GPU reports do not share {field}")
|
||||||
|
if gpu_config["plan"].get(field) != gpu_report["plan"].get(field):
|
||||||
|
raise RuntimeError(f"GPU config and report do not share {field}")
|
||||||
|
|
||||||
|
expected_recipes = {
|
||||||
|
"transformers-fp32-rocm-quality-oracle": ("quality", "cuda"),
|
||||||
|
"llama-cpp-bf16-rocm-quality": ("quality", "cuda"),
|
||||||
|
"transformers-bf16-rocm-throughput": ("performance-fit", "cuda"),
|
||||||
|
"llama-cpp-q4-rocm-throughput": ("performance-fit", "cuda"),
|
||||||
|
}
|
||||||
|
actual_recipes = {
|
||||||
|
entry["recipe"]["id"]: (entry["recipe"]["lane"], entry["recipe"]["device"])
|
||||||
|
for entry in gpu_report["recipes"]
|
||||||
|
}
|
||||||
|
if actual_recipes != expected_recipes:
|
||||||
|
raise RuntimeError("GPU report recipe identities, lanes, or devices changed")
|
||||||
|
|
||||||
|
gpu_prompt_ids = {prompt["id"] for prompt in gpu_report["plan"]["prompts"]}
|
||||||
|
levels = {int(level) for level in gpu_report["plan"]["concurrency_levels"]}
|
||||||
|
repeats = int(gpu_report["plan"]["repeats"])
|
||||||
|
expected_outcomes = len(gpu_prompt_ids) * repeats * sum(levels)
|
||||||
|
for entry in gpu_report["recipes"]:
|
||||||
|
recipe_id = entry["recipe"]["id"]
|
||||||
|
if not entry.get("available") or len(entry.get("outcomes", ())) != expected_outcomes:
|
||||||
|
raise RuntimeError(f"GPU recipe {recipe_id!r} lacks complete outcomes")
|
||||||
|
if any(
|
||||||
|
not outcome.get("ok")
|
||||||
|
or outcome.get("recipe_id") != recipe_id
|
||||||
|
or outcome.get("prompt_id") not in gpu_prompt_ids
|
||||||
|
or int(outcome.get("concurrency", 0)) not in levels
|
||||||
|
or not 0 <= int(outcome.get("repeat", -1)) < repeats
|
||||||
|
for outcome in entry["outcomes"]
|
||||||
|
):
|
||||||
|
raise RuntimeError(f"GPU recipe {recipe_id!r} contains failed or invalid outcomes")
|
||||||
|
if {int(level) for level in entry["concurrency"]} != levels:
|
||||||
|
raise RuntimeError(f"GPU recipe {recipe_id!r} has wrong concurrency cells")
|
||||||
|
for prompt_id in gpu_prompt_ids:
|
||||||
|
for level in levels:
|
||||||
|
for repeat in range(repeats):
|
||||||
|
count = sum(
|
||||||
|
outcome["prompt_id"] == prompt_id
|
||||||
|
and int(outcome["concurrency"]) == level
|
||||||
|
and int(outcome["repeat"]) == repeat
|
||||||
|
for outcome in entry["outcomes"]
|
||||||
|
)
|
||||||
|
if count != level:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"GPU recipe {recipe_id!r} lacks complete request coverage"
|
||||||
|
)
|
||||||
|
if any(
|
||||||
|
int(cell.get("failures", -1)) != 0
|
||||||
|
or int(cell.get("requests", -1))
|
||||||
|
!= len(
|
||||||
|
[
|
||||||
|
outcome
|
||||||
|
for outcome in entry["outcomes"]
|
||||||
|
if int(outcome["concurrency"]) == int(level)
|
||||||
|
]
|
||||||
|
)
|
||||||
|
for level, cell in entry["concurrency"].items()
|
||||||
|
):
|
||||||
|
raise RuntimeError(f"GPU recipe {recipe_id!r} aggregates do not match outcomes")
|
||||||
|
|
||||||
|
cpu_quality = _drift(cpu_report, "llama-cpp-near-lossless-quality")
|
||||||
|
gpu_quality = _drift(gpu_report, "llama-cpp-bf16-rocm-quality")
|
||||||
|
cpu_recipe = _recipe(cpu_report, "llama-cpp-near-lossless-quality")
|
||||||
|
gpu_recipe = _recipe(gpu_report, "llama-cpp-bf16-rocm-quality")
|
||||||
|
gpu_backend = gpu_recipe["load"]["backend_detail"]
|
||||||
|
if "measured accelerator ROCm0: Radeon 8060S Graphics" not in gpu_backend:
|
||||||
|
raise RuntimeError("GPU report lacks measured ROCm device evidence")
|
||||||
|
if "measured offload 25/25 layers" not in gpu_backend:
|
||||||
|
raise RuntimeError("GPU report lacks measured layer-offload evidence")
|
||||||
|
if cpu_recipe["recipe"]["artifact_sha256"] != gpu_recipe["recipe"]["artifact_sha256"]:
|
||||||
|
raise RuntimeError("CPU and GPU diagnostics use different BF16 GGUF artifacts")
|
||||||
|
if gpu_quality.get("compared_prompts") != len(gpu_prompt_ids):
|
||||||
|
raise RuntimeError("GPU quality drift lacks complete prompt coverage")
|
||||||
|
if {item["prompt_id"] for item in gpu_quality.get("per_prompt", ())} != gpu_prompt_ids:
|
||||||
|
raise RuntimeError("GPU quality drift prompt identities do not match the plan")
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"schema_version": 2,
|
||||||
|
"model_id": cpu_report["plan"]["model_id"],
|
||||||
|
"model_revision": cpu_report["plan"]["model_revision"],
|
||||||
|
"cpu_v1": {
|
||||||
|
"report": "results.json",
|
||||||
|
"report_sha256": _file_sha256("results.json"),
|
||||||
|
"run_id": cpu_report["provenance"]["run_id"],
|
||||||
|
"plan_id": cpu_report["plan"]["plan_id"],
|
||||||
|
"plan_sha256": _canonical_sha256(cpu_report["plan"]),
|
||||||
|
"config_sha256": cpu_report["provenance"]["config_sha256"],
|
||||||
|
"device": "cpu",
|
||||||
|
"quality_oracle": "Transformers BF16 safetensors",
|
||||||
|
"candidate": "llama.cpp BF16 GGUF",
|
||||||
|
"candidate_artifact_sha256": cpu_recipe["recipe"]["artifact_sha256"],
|
||||||
|
"exact_match_rate": cpu_quality["exact_match_rate"],
|
||||||
|
"mean_similarity": cpu_quality["mean_similarity"],
|
||||||
|
"contract_verdict": cpu_evaluation.verdict,
|
||||||
|
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
|
||||||
|
},
|
||||||
|
"rocm_diagnostic": {
|
||||||
|
"report": "gpu-diagnostic-results.json",
|
||||||
|
"report_sha256": _file_sha256("gpu-diagnostic-results.json"),
|
||||||
|
"run_id": gpu_report["provenance"]["run_id"],
|
||||||
|
"producer": gpu_report["provenance"]["producer"],
|
||||||
|
"signer_fingerprint": signer_fingerprint,
|
||||||
|
"plan_id": gpu_report["plan"]["plan_id"],
|
||||||
|
"plan_sha256": _canonical_sha256(gpu_report["plan"]),
|
||||||
|
"config_sha256": gpu_report["provenance"]["config_sha256"],
|
||||||
|
"device": "cuda (ROCm)",
|
||||||
|
"quality_oracle": "Transformers float32 safetensors",
|
||||||
|
"candidate": "llama.cpp BF16 GGUF",
|
||||||
|
"candidate_artifact_sha256": gpu_recipe["recipe"]["artifact_sha256"],
|
||||||
|
"measured_backend_detail": gpu_backend,
|
||||||
|
"exact_match_rate": gpu_quality["exact_match_rate"],
|
||||||
|
"mean_similarity": gpu_quality["mean_similarity"],
|
||||||
|
"failures": sum(
|
||||||
|
metrics["failures"]
|
||||||
|
for entry in gpu_report["recipes"]
|
||||||
|
for metrics in entry["concurrency"].values()
|
||||||
|
),
|
||||||
|
"v1_eligible": False,
|
||||||
|
},
|
||||||
|
"conclusion": {
|
||||||
|
"v1_verdict_changed": False,
|
||||||
|
"cpu_bf16_divergence_explained": False,
|
||||||
|
"conversion_corruption_observed_in_rocm_sample": False,
|
||||||
|
"scope": (
|
||||||
|
"The ROCm diagnostic establishes only that the same BF16 GGUF artifact "
|
||||||
|
"matched the float32 oracle for three GPU sequences; it does not explain "
|
||||||
|
"the CPU BF16 divergence or prove global conversion correctness."
|
||||||
|
),
|
||||||
|
"recommended_v2_design": (
|
||||||
|
"Predeclare a float32 quality oracle separately from the BF16 performance "
|
||||||
|
"reference, with a larger prompt corpus and immutable thresholds."
|
||||||
|
),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
(ROOT / "quality-parity-diagnosis.json").write_text(
|
||||||
|
json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
md = f"""# DGR-001 quality-parity evidence summary
|
||||||
|
|
||||||
|
This summary is generated by `summarize-quality-parity.py` from signed reports.
|
||||||
|
It contains no independent logit measurements or self-asserted verification flag.
|
||||||
|
|
||||||
|
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|
||||||
|
|---|---|---|---|---:|---:|---|
|
||||||
|
| CPU v1 (`{summary['cpu_v1']['run_id']}`) | CPU | Transformers BF16 | llama.cpp BF16 | {summary['cpu_v1']['exact_match_rate']:.4f} | {summary['cpu_v1']['mean_similarity']:.4f} | immutable `stop` |
|
||||||
|
| ROCm diagnostic (`{summary['rocm_diagnostic']['run_id']}`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | {summary['rocm_diagnostic']['exact_match_rate']:.4f} | {summary['rocm_diagnostic']['mean_similarity']:.4f} | diagnostic only |
|
||||||
|
|
||||||
|
## Interpretation
|
||||||
|
|
||||||
|
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
|
||||||
|
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
|
||||||
|
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
|
||||||
|
matched the float32 oracle for all three GPU sequences with zero failures.
|
||||||
|
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
|
||||||
|
`25/25` layer offload.
|
||||||
|
|
||||||
|
No conversion corruption was observed in that three-sequence ROCm sample. This
|
||||||
|
does not prove global conversion correctness and does not retroactively change
|
||||||
|
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
|
||||||
|
separately from its BF16 performance reference and use a larger corpus.
|
||||||
|
|
||||||
|
## Reproduction and bindings
|
||||||
|
|
||||||
|
- CPU report SHA-256: `{summary['cpu_v1']['report_sha256']}`
|
||||||
|
- GPU report SHA-256: `{summary['rocm_diagnostic']['report_sha256']}`
|
||||||
|
- BF16 GGUF SHA-256: `{summary['rocm_diagnostic']['candidate_artifact_sha256']}`
|
||||||
|
- Signer fingerprint: `{signer_fingerprint}`
|
||||||
|
- Exact verification command: see `commands.txt`.
|
||||||
|
"""
|
||||||
|
(ROOT / "quality-parity-diagnosis.md").write_text(md, encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
242
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
242
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
@@ -0,0 +1,242 @@
|
|||||||
|
# DGR-002 — Adopt the versioned gRPC Shard protocol
|
||||||
|
|
||||||
|
Status: **done**. Every acceptance criterion is met with real command output.
|
||||||
|
Evidence class: **synthetic/unit** — this story defines a schema and proves both
|
||||||
|
languages agree on it. No model, GPU, network peer or benchmark is involved, and
|
||||||
|
none is claimed.
|
||||||
|
|
||||||
|
## 1. Summary
|
||||||
|
|
||||||
|
`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for
|
||||||
|
the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python
|
||||||
|
and C++ both generate from it, and a shared committed conformance vector proves
|
||||||
|
they encode it identically — byte for byte.
|
||||||
|
|
||||||
|
Design decisions worth carrying forward:
|
||||||
|
|
||||||
|
- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation,
|
||||||
|
identity and flow control are carried as fields, not left to HTTP/2 metadata,
|
||||||
|
because the existing relay carries these frames as **opaque binary**. A relayed
|
||||||
|
frame has no HTTP/2 context to inherit a deadline or a channel identity from.
|
||||||
|
If it is not in the schema, it does not survive the relay.
|
||||||
|
- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the
|
||||||
|
stream; `Cancel` is also a unary RPC. A cancel that can only travel down a
|
||||||
|
stream that flow control has wedged is not a cancel.
|
||||||
|
- **Checksums cover the uncompressed payload.** Compression is a per-hop
|
||||||
|
transport decision (reusing the existing `activation_compression` policies), so
|
||||||
|
a checksum over the compressed frame would be invalidated by a hop that merely
|
||||||
|
chose differently.
|
||||||
|
- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2
|
||||||
|
bounds *bytes in flight*; it does not bound how much *work* a worker has queued,
|
||||||
|
and a relayed frame gets no window at all. Credits bound queue occupancy and KV
|
||||||
|
pressure, and negotiation takes the strictest bound of either peer so a sender
|
||||||
|
cannot talk a worker into unbounded queues.
|
||||||
|
|
||||||
|
## 2. Files changed
|
||||||
|
|
||||||
|
New:
|
||||||
|
|
||||||
|
| Path | What |
|
||||||
|
|---|---|
|
||||||
|
| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `9e211660…`, see `protocol.json`) |
|
||||||
|
| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest |
|
||||||
|
| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
|
||||||
|
| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors |
|
||||||
|
| `packages/node/native/README.md` | How to regenerate and build |
|
||||||
|
| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface |
|
||||||
|
| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation |
|
||||||
|
| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages |
|
||||||
|
| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) |
|
||||||
|
| `scripts/generate_native_protocol.py` | Python generation, with `--check` |
|
||||||
|
| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` |
|
||||||
|
| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source |
|
||||||
|
| `tests/test_native_shard_protocol.py` | 45 Python tests |
|
||||||
|
|
||||||
|
Modified:
|
||||||
|
|
||||||
|
- `packages/node/pyproject.toml` — added runtime floors `grpcio>=1.82.1` and
|
||||||
|
`protobuf>=7.35.0`, matching the committed generated-code requirements; new
|
||||||
|
`proto` extra pinning `grpcio-tools==1.82.1`.
|
||||||
|
- `packages/node/meshnet_node/activation_compression.py` — optional bounded zstd
|
||||||
|
output for untrusted protocol frames; existing callers remain compatible.
|
||||||
|
- `packages/node/meshnet_node/native_protocol/__init__.py` — exports negotiated
|
||||||
|
bound constants and whole-session-message validation.
|
||||||
|
|
||||||
|
The canonical PRD marks only DGR-002 passed. `git status` before this story was clean.
|
||||||
|
|
||||||
|
## 3. Commands and real results
|
||||||
|
|
||||||
|
See `commands.txt` for the exact ordered list. Results:
|
||||||
|
|
||||||
|
```
|
||||||
|
python scripts/generate_native_protocol.py --check -> generated stubs are up to date
|
||||||
|
python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date
|
||||||
|
|
||||||
|
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||||
|
-- gRPC C++ not found: building message types only (sufficient for the conformance test)
|
||||||
|
cmake --build build/native -j -> Built target shard_protocol_conformance
|
||||||
|
ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed
|
||||||
|
100% tests passed out of 1
|
||||||
|
|
||||||
|
cmp build/native/cpp_roundtrip.binpb \
|
||||||
|
packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0)
|
||||||
|
|
||||||
|
pytest -q tests/test_native_shard_protocol.py -> 45 passed
|
||||||
|
pytest -q tests/test_native_shard_protocol.py \
|
||||||
|
tests/test_activation_compression.py -> 51 passed
|
||||||
|
pytest -q (final full suite) -> 728 passed, 12 skipped
|
||||||
|
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
(after an earlier flaky full-suite failure) -> 1 passed, 1 passed, 1 passed
|
||||||
|
clean minimum-runtime import + codec smoke test -> passed
|
||||||
|
grpcio==1.82.1, protobuf==7.35.0
|
||||||
|
compileall -q packages tests -> OK (exit 0)
|
||||||
|
git diff --check -> clean (exit 0)
|
||||||
|
```
|
||||||
|
|
||||||
|
The C++ lane was rebuilt from scratch by Ralph (`rm -rf build/native`) using only
|
||||||
|
the documented commands, and reproduced the same result. During controller
|
||||||
|
review the user explicitly chose not to repeat the destructive build-directory
|
||||||
|
cleanup, so the independent controller relied on the recorded CMake/CTest run
|
||||||
|
while reproducing every Python/generation/full-suite gate.
|
||||||
|
|
||||||
|
### Controller review corrections
|
||||||
|
|
||||||
|
Independent controller review found and fixed two classes of issue before
|
||||||
|
integration:
|
||||||
|
|
||||||
|
1. Generated stubs required gRPC 1.82.1 and Protobuf 7.35.0, while the initial
|
||||||
|
package metadata allowed much older runtimes that could fail at import time.
|
||||||
|
2. Flow-control bounds were described but not enforced by the reference decoder.
|
||||||
|
Tensor declarations, shape rank/dimensions, fragment/tensor counts, fragments,
|
||||||
|
wire bodies, whole bundles, complete session messages (including envelope
|
||||||
|
overhead), and zstd window/output expansion are now fail-closed against the
|
||||||
|
negotiated/default bounds. Unspecified bundle versions, compression and
|
||||||
|
checksums are rejected rather than interpreted as valid data.
|
||||||
|
3. Negotiated initial credits could exceed `max_inflight_chunks`; credits are now
|
||||||
|
capped by the settled in-flight limit.
|
||||||
|
|
||||||
|
Controller results: protocol tests `45 passed`; protocol plus shared compression
|
||||||
|
tests `51 passed`; final full suite `728 passed, 12 skipped`. A clean environment
|
||||||
|
at the declared minimum gRPC/Protobuf runtime versions imported both generated
|
||||||
|
stub modules and round-tripped the codec. Generation checks, `compileall`, static
|
||||||
|
secret scan, and `git diff --check` all passed.
|
||||||
|
|
||||||
|
### Full-suite note — a pre-existing flaky test
|
||||||
|
|
||||||
|
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy`
|
||||||
|
is **flaky on a clean tree, independent of this story**. Reproduction, run
|
||||||
|
*before any DGR-002 file existed* (working tree clean, `git status` empty):
|
||||||
|
|
||||||
|
```
|
||||||
|
pytest -q -> 1 failed, 682 passed, 12 skipped
|
||||||
|
FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
|
||||||
|
# same test, three consecutive isolated runs on the same clean tree:
|
||||||
|
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
-> 1 passed in 1.76s
|
||||||
|
-> 1 failed in 4.39s
|
||||||
|
-> 1 passed in 1.10s
|
||||||
|
```
|
||||||
|
|
||||||
|
It is a timing race in proxy cancellation (a 3-second in-flight generation raced
|
||||||
|
against the cancel assertion), not a deterministic failure, and it touches no code
|
||||||
|
this story changes. One controller full-suite run reported exactly that one failure
|
||||||
|
(`1 failed, 719 passed, 12 skipped`); three immediate isolated retries all passed
|
||||||
|
in 1.11 seconds, and the final exact-code full suite was green (`728 passed,
|
||||||
|
12 skipped`). It is flagged for whoever owns the tracker cancel path and is **not**
|
||||||
|
fixed here, since silently touching another story's code is out of scope.
|
||||||
|
|
||||||
|
## 4. Acceptance criteria
|
||||||
|
|
||||||
|
| Criterion | Where it is proven |
|
||||||
|
|---|---|
|
||||||
|
| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` |
|
||||||
|
| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` |
|
||||||
|
| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` |
|
||||||
|
| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test |
|
||||||
|
| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` |
|
||||||
|
| Round-trip + compatibility tests in Python and C++ | 45 Python tests; C++ `ctest` 1/1; cross-language byte equality |
|
||||||
|
| Targeted pytest passes | 45 passed |
|
||||||
|
| `compileall packages tests` | exit 0 |
|
||||||
|
| `git diff --check` | exit 0 |
|
||||||
|
| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU |
|
||||||
|
| Full deterministic pytest passes, or pre-existing failure recorded | Final exact-code run: 728 passed, 12 skipped; earlier sole flaky failure documented with clean-tree reproduction and 3/3 passing retries |
|
||||||
|
|
||||||
|
## 5. How the cross-language claim is actually earned
|
||||||
|
|
||||||
|
Two codecs that each round-trip their own output prove only that each is
|
||||||
|
self-consistent. Instead:
|
||||||
|
|
||||||
|
1. Python builds the canonical `SessionRequest` and commits its bytes.
|
||||||
|
2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C
|
||||||
|
**from the polynomial in independent C++ code**, reassembles the multi-fragment
|
||||||
|
tensor, and re-serializes to `cpp_roundtrip.binpb`.
|
||||||
|
3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden.
|
||||||
|
|
||||||
|
Compatibility is tested in both languages: an unknown field from a newer peer
|
||||||
|
survives a parse/serialize hop (a Shard forwards activations — silently stripping
|
||||||
|
fields would corrupt a route it is merely a waypoint on), and a sparse message
|
||||||
|
from an older peer parses to proto3 defaults.
|
||||||
|
|
||||||
|
## 6. Limitations and deferred work
|
||||||
|
|
||||||
|
- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message
|
||||||
|
types), not a running gRPC C++ server, because this machine has no gRPC C++ stack
|
||||||
|
and building it is a large dependency the conformance test does not need.
|
||||||
|
`CMakeLists.txt` already generates and exports `shard_runtime_grpc` when
|
||||||
|
`find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend
|
||||||
|
`scripts/bootstrap_native_toolchain.sh`.**
|
||||||
|
- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no
|
||||||
|
deadline actually fires, no credit is actually consumed. This story defines and
|
||||||
|
proves the contract; DGR-008/DGR-009 implement it.
|
||||||
|
- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral).
|
||||||
|
`scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such
|
||||||
|
as `build/native-toolchain`.
|
||||||
|
- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when
|
||||||
|
present. The fallback is byte-exact but slow; a worker on the hot path should install
|
||||||
|
the native package. Not a correctness limitation.
|
||||||
|
- Compression on the wire is zstd-or-none only, matching the existing seam.
|
||||||
|
|
||||||
|
## 7. Compatibility and migration notes
|
||||||
|
|
||||||
|
- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays
|
||||||
|
at `2` and the legacy `/forward` path is untouched. The native protocol is a
|
||||||
|
*separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this
|
||||||
|
story is on any live request path — it is additive.
|
||||||
|
- Semantics are deliberately preserved from the existing ADRs so the two transports
|
||||||
|
mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len`
|
||||||
|
and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022),
|
||||||
|
bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe
|
||||||
|
identity mirroring the capability report (ADR-0023).
|
||||||
|
- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed
|
||||||
|
because `NamedTensor.total_bytes` is the single source of truth. Never recycle it —
|
||||||
|
a recycled field number is the one schema change peers cannot detect, because the
|
||||||
|
bytes still parse.
|
||||||
|
- Committed Python stubs are guarded by `--check` in the test suite, so they cannot
|
||||||
|
drift from the schema unnoticed.
|
||||||
|
|
||||||
|
## 8. Handoff to dependent stories
|
||||||
|
|
||||||
|
- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint`
|
||||||
|
(`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`,
|
||||||
|
`catalogue_version`). The mismatch outcome is already specified:
|
||||||
|
`ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct.
|
||||||
|
- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a
|
||||||
|
**named bundle**, not a bare tensor — a boundary needing more than one tensor is
|
||||||
|
already representable. Execute `[effective_start_layer, end_layer)`, never from
|
||||||
|
`start_layer`.
|
||||||
|
- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`.
|
||||||
|
`CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a
|
||||||
|
decode step whose `expected_past_len` does not match **must** miss, never fall back
|
||||||
|
to a silent stateless forward. `idempotency_step` means a retried step is
|
||||||
|
acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache
|
||||||
|
twice and desynchronises the route.
|
||||||
|
- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must
|
||||||
|
first install gRPC C++ (see limitations). Honour `FlowControl` credits and the
|
||||||
|
`max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py`
|
||||||
|
as the reference for fragment reassembly and checksum validation.
|
||||||
|
- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as
|
||||||
|
opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add
|
||||||
|
a second control plane.
|
||||||
|
- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately
|
||||||
|
changes, regenerate it and say so, because the C++ test asserts those exact bytes.
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# DGR-002 — exact commands, in order. Run from the repository root.
|
||||||
|
# Interpreter: <repo>/.venv/bin/python (CPython 3.14.6). Deterministic, GPU-free,
|
||||||
|
# no model download, no API credits.
|
||||||
|
|
||||||
|
# --- toolchain (this machine had no protoc, no cmake, no protobuf C++ headers)
|
||||||
|
.venv/bin/python -m pip install grpcio-tools==1.82.1 grpcio==1.82.1 cmake==4.4.0
|
||||||
|
scripts/bootstrap_native_toolchain.sh /tmp/pbsrc/install # protobuf C++ 33.1 + abseil 20250814.1
|
||||||
|
|
||||||
|
# --- schema generation (Python stubs; committed)
|
||||||
|
.venv/bin/python scripts/generate_native_protocol.py
|
||||||
|
.venv/bin/python scripts/generate_native_protocol.py --check # -> "generated stubs are up to date"
|
||||||
|
|
||||||
|
# --- cross-language conformance vectors (committed)
|
||||||
|
.venv/bin/python scripts/generate_protocol_goldens.py
|
||||||
|
.venv/bin/python scripts/generate_protocol_goldens.py --check # -> "conformance vectors are up to date"
|
||||||
|
|
||||||
|
# --- C++ generation, build and conformance test
|
||||||
|
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||||
|
cmake --build build/native -j"$(nproc)"
|
||||||
|
ctest --test-dir build/native --output-on-failure # -> 1/1 Passed
|
||||||
|
cmp build/native/cpp_roundtrip.binpb packages/node/native/testdata/session_request_golden.binpb
|
||||||
|
|
||||||
|
# --- Python tests
|
||||||
|
.venv/bin/python -m pytest -q tests/test_native_shard_protocol.py # -> 29 passed
|
||||||
|
.venv/bin/python -m pytest -q # full suite
|
||||||
|
|
||||||
|
# --- repository gates
|
||||||
|
.venv/bin/python -m compileall -q packages tests
|
||||||
|
git diff --check
|
||||||
|
|
||||||
|
# --- independent controller review after Ralph
|
||||||
|
PYTHONPATH=packages/node .venv/bin/python -m pytest -q tests/test_native_shard_protocol.py
|
||||||
|
# -> 45 passed
|
||||||
|
PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
|
||||||
|
tests/test_native_shard_protocol.py tests/test_activation_compression.py
|
||||||
|
# -> 51 passed
|
||||||
|
PYTHONPATH=packages/node .venv/bin/python -m pytest -q
|
||||||
|
# -> final exact-code run: 728 passed, 12 skipped
|
||||||
|
for i in 1 2 3; do PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
|
||||||
|
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy; done
|
||||||
|
# -> 1 passed, 1 passed, 1 passed
|
||||||
|
# clean minimum-runtime venv: protobuf==7.35.0 grpcio==1.82.1
|
||||||
|
# generated pb2 + pb2_grpc imports and one-byte codec round trip -> passed
|
||||||
|
# The user chose to rely on Ralph's recorded successful C++ CMake/CTest run
|
||||||
|
# rather than repeat deletion of an isolated generated build directory.
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
{
|
||||||
|
"schema_version": "SCHEMA_VERSION_1",
|
||||||
|
"bundle_version": 1,
|
||||||
|
"proto_path": "packages/node/native/proto/shard_runtime.proto",
|
||||||
|
"proto_sha256": "9e211660b3fcefc88bcdf3851c3571088c00349aacb5adc5ef45083c83d0cce2",
|
||||||
|
"protoc": "grpc_tools 1.82.1 (python) / protobuf 33.1 (C++)",
|
||||||
|
"service": {
|
||||||
|
"GetCapability": {
|
||||||
|
"client_streaming": false,
|
||||||
|
"server_streaming": false
|
||||||
|
},
|
||||||
|
"Health": {
|
||||||
|
"client_streaming": false,
|
||||||
|
"server_streaming": false
|
||||||
|
},
|
||||||
|
"Session": {
|
||||||
|
"client_streaming": true,
|
||||||
|
"server_streaming": true
|
||||||
|
},
|
||||||
|
"Release": {
|
||||||
|
"client_streaming": false,
|
||||||
|
"server_streaming": false
|
||||||
|
},
|
||||||
|
"Cancel": {
|
||||||
|
"client_streaming": false,
|
||||||
|
"server_streaming": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"envelope_fields": [
|
||||||
|
"cache_expectation",
|
||||||
|
"chunk",
|
||||||
|
"deadline_unix_nanos",
|
||||||
|
"fingerprint",
|
||||||
|
"idempotency_step",
|
||||||
|
"phase",
|
||||||
|
"position",
|
||||||
|
"route_epoch",
|
||||||
|
"route_session_id",
|
||||||
|
"schema_version",
|
||||||
|
"shard_range",
|
||||||
|
"work_id"
|
||||||
|
],
|
||||||
|
"named_tensor_fields": [
|
||||||
|
"byte_order",
|
||||||
|
"checksum",
|
||||||
|
"compression",
|
||||||
|
"dtype",
|
||||||
|
"fragments",
|
||||||
|
"name",
|
||||||
|
"shape",
|
||||||
|
"total_bytes"
|
||||||
|
],
|
||||||
|
"phases": [
|
||||||
|
"PHASE_UNSPECIFIED",
|
||||||
|
"PHASE_PREFILL",
|
||||||
|
"PHASE_DECODE",
|
||||||
|
"PHASE_RELEASE",
|
||||||
|
"PHASE_CANCEL"
|
||||||
|
],
|
||||||
|
"error_codes": [
|
||||||
|
"ERROR_CODE_UNSPECIFIED",
|
||||||
|
"ERROR_CODE_SCHEMA_UNSUPPORTED",
|
||||||
|
"ERROR_CODE_FINGERPRINT_MISMATCH",
|
||||||
|
"ERROR_CODE_EPOCH_STALE",
|
||||||
|
"ERROR_CODE_SHARD_RANGE_MISMATCH",
|
||||||
|
"ERROR_CODE_CACHE_MISS",
|
||||||
|
"ERROR_CODE_RESOURCE_EXHAUSTED",
|
||||||
|
"ERROR_CODE_PAYLOAD_CORRUPT",
|
||||||
|
"ERROR_CODE_CANCELLED",
|
||||||
|
"ERROR_CODE_DEADLINE_EXCEEDED",
|
||||||
|
"ERROR_CODE_FLOW_CONTROL_VIOLATION",
|
||||||
|
"ERROR_CODE_INTERNAL"
|
||||||
|
],
|
||||||
|
"bounds": {
|
||||||
|
"max_prefill_chunk_tokens": 128,
|
||||||
|
"max_chunk_bytes": 4194304,
|
||||||
|
"max_fragment_bytes": 1048576,
|
||||||
|
"max_inflight_chunks": 8,
|
||||||
|
"max_fragments_per_tensor": 64,
|
||||||
|
"max_tensors_per_bundle": 64,
|
||||||
|
"max_tensor_rank": 8,
|
||||||
|
"max_tensor_dimension": 2147483647,
|
||||||
|
"whole_session_message_enforced": true
|
||||||
|
},
|
||||||
|
"golden_vectors": {
|
||||||
|
"session_request_golden.binpb": "c2c3df8a717ddeae7bd99624d2c7f34c09a518988de990237fe313b75cff0817",
|
||||||
|
"capability_report_golden.binpb": "71ac5f150775f398515b43a63596a5cbe8d2ad607e7e4de56bd44fbe7987080c"
|
||||||
|
},
|
||||||
|
"verification": {
|
||||||
|
"python_protocol_tests": "45 passed",
|
||||||
|
"python_protocol_and_compression_tests": "51 passed",
|
||||||
|
"full_suite": "728 passed, 12 skipped",
|
||||||
|
"minimum_runtime": "grpcio 1.82.1 / protobuf 7.35.0 passed import and codec smoke"
|
||||||
|
}
|
||||||
|
}
|
||||||
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
@@ -0,0 +1,186 @@
|
|||||||
|
# DGR-003 — exact Artifact and runtime recipe identity
|
||||||
|
|
||||||
|
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
|
||||||
|
network node, or API credit is required or claimed.
|
||||||
|
|
||||||
|
## Result — delayed-review repair, 2026-07-14
|
||||||
|
|
||||||
|
DGR-003 defines and tests an exact, model-agnostic compatibility identity and
|
||||||
|
connects it to DGR-002's gRPC `Fingerprint` plus tracker parsing, admission,
|
||||||
|
route partitioning, and certification. It is **not complete**: the existing
|
||||||
|
production doctor/backend path still emits the legacy capability report without
|
||||||
|
constructing a `ShardIdentity` from authoritative loaded artifact/runtime state.
|
||||||
|
No exact recipe is therefore claimed live or routable from that path; supplied
|
||||||
|
exact identities remain dark until tracker-owned certification.
|
||||||
|
|
||||||
|
A matching digest proves canonical consistency, **not node authenticity or real
|
||||||
|
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
|
||||||
|
complete, multi-node distributed forward is the execution trust boundary.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
|
||||||
|
architecture/config digest, layer count, and optional derivative binding.
|
||||||
|
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
|
||||||
|
and its end-exclusive layer range. A Shard cannot advertise outside that range.
|
||||||
|
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
|
||||||
|
a backend label:
|
||||||
|
- weight quantization;
|
||||||
|
- activation and compute dtypes;
|
||||||
|
- KV dtype and layout;
|
||||||
|
- tokenizer revision;
|
||||||
|
- architecture adapter;
|
||||||
|
- backend and runtime version;
|
||||||
|
- boundary and protocol schema versions;
|
||||||
|
- recipe ID/version and catalogue version.
|
||||||
|
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
|
||||||
|
`Fingerprint`; `check_session_open()` fails closed on schema, fingerprint,
|
||||||
|
advertised/effective range, non-empty route session, positive route epoch,
|
||||||
|
and (when supplied) exact tracker route-session/epoch assignment.
|
||||||
|
- Node and tracker implementations independently canonicalize the declaration.
|
||||||
|
This is intentional: the tracker must not trust a digest copied from a node,
|
||||||
|
and future native/C++ workers also need an independent implementation. Their
|
||||||
|
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
|
||||||
|
- Tracker admission cross-checks the exact identity against the capability
|
||||||
|
proof's model, range, recipe labels, backend, and weight quantization. Any
|
||||||
|
disagreement fails closed.
|
||||||
|
- `TrackerServer` owns the sole live certification ledger and passes it through
|
||||||
|
direct and replicated registration paths. A known exact recipe is
|
||||||
|
`uncertified` and dark for user traffic until the same exact fingerprint is
|
||||||
|
certified. Restart fails closed; durable/cluster-wide certification events
|
||||||
|
require the later real-forward control path and are not claimed here.
|
||||||
|
- Certification evidence is bound to the promoted fingerprint, requires at
|
||||||
|
least two distinct nodes, complete layer coverage, generated tokens, and
|
||||||
|
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/meshnet_node/runtime_recipe.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/recipe.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/capability.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py`
|
||||||
|
- `tests/data/recipe_fingerprint_vectors.json`
|
||||||
|
- `tests/test_runtime_recipe_identity.py`
|
||||||
|
- this evidence directory, issue state, and DGR-003 PRD state
|
||||||
|
|
||||||
|
A late review of dependency DGR-017 also found and fixed two genuine contract
|
||||||
|
continuity defects during delayed DGR-003 review: v1 now has an independently
|
||||||
|
trusted digest and recursively immutable parsed state. Those changes and tests
|
||||||
|
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Exact commands and outcomes are in `commands.txt`.
|
||||||
|
|
||||||
|
Observed final results:
|
||||||
|
|
||||||
|
- DGR-003 identity + node/tracker capability suites: **126 passed**.
|
||||||
|
- DGR-017 focused dependency repair suite: **99 passed**.
|
||||||
|
- Tracker routing suite: **93 passed**.
|
||||||
|
- First delayed-review integrated run: **898 passed, 13 skipped, 1 failed** on
|
||||||
|
the pre-existing tracker-cancellation race.
|
||||||
|
- Final delayed-review integrated rerun: **899 passed, 13 skipped** in
|
||||||
|
**253.64s**; Hermes controller acceptance rerun: **899 passed, 13 skipped**
|
||||||
|
in **252.66s**.
|
||||||
|
- `python -m compileall -q packages tests`: pass.
|
||||||
|
- `git diff --check`: pass.
|
||||||
|
- Ruff on the changed identity, capability, contract, and test modules: pass.
|
||||||
|
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
|
||||||
|
current tree; DGR-003 added no finding.
|
||||||
|
|
||||||
|
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
|
||||||
|
on the known unrelated
|
||||||
|
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
|
||||||
|
completed after three seconds just before cancellation, so the cancel endpoint
|
||||||
|
returned 404. In this delayed repair it again produced a 404 after the stream
|
||||||
|
finished (first integrated run: **898 passed, 13 skipped, 1 failed**); three
|
||||||
|
immediate isolated repeats passed before a fourth reproduced the same race.
|
||||||
|
No cancellation-test code was changed. The final complete integrated rerun
|
||||||
|
passed **899/899** tests.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Certification state is process-local in this story. The same running tracker
|
||||||
|
reuses it across registrations, but durable/cluster-wide certification-event
|
||||||
|
persistence belongs with the later real distributed-forward control path.
|
||||||
|
Restart or failover therefore returns exact recipes to the safe dark state;
|
||||||
|
it never makes an unsupported recipe routable.
|
||||||
|
- The node module has no certification ledger or admission policy; it holds only
|
||||||
|
identity construction and handshake validation. The Tracker is the sole
|
||||||
|
promotion authority.
|
||||||
|
- **Completion blocker:** `doctor._validate_recipe()` calls
|
||||||
|
`build_capability_report()` without `identity=`, because the legacy
|
||||||
|
Transformers backend does not expose an immutable artifact-content pin and
|
||||||
|
full runtime recipe axes authoritative enough to build one. Adding a guessed
|
||||||
|
identity would weaken this contract. Production emission must be added with
|
||||||
|
the authoritative native worker/backend loading seam; until then the issue and
|
||||||
|
PRD deliberately remain incomplete.
|
||||||
|
- This story proves identity and admission behavior with deterministic fixtures.
|
||||||
|
It does not claim a real GLM forward or hardware certification.
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
- Capability report identity is additive. Legacy reports without the new block
|
||||||
|
retain ADR-0023's explicit compatibility-policy behavior.
|
||||||
|
- Reports that opt into exact identity are held to it and fail closed on malformed,
|
||||||
|
inconsistent, unknown, dark, or mismatched declarations.
|
||||||
|
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
|
||||||
|
representation.
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
|
||||||
|
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
|
||||||
|
schema versions, and owned range. At `SessionOpen`, compare its
|
||||||
|
`CompatibilityFingerprint` and return DGR-002's
|
||||||
|
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
|
||||||
|
|
||||||
|
A digest match is not certification. Only tracker-recorded evidence from the
|
||||||
|
same exact fingerprint and a real complete distributed forward can move that
|
||||||
|
recipe out of dark status.
|
||||||
|
|
||||||
|
## Native emission closure — 2026-07-14
|
||||||
|
|
||||||
|
Status: **done**. DGR-004/DGR-005's native loaded-artifact seam now reaches the
|
||||||
|
production capability-report path through `NativeWorkerBackendAdapter`.
|
||||||
|
|
||||||
|
### Files changed
|
||||||
|
|
||||||
|
- `packages/node/meshnet_node/native_backend.py` — immutable loaded-GGUF report,
|
||||||
|
immutable artifact and numerical pins, exact identity derivation, and the
|
||||||
|
SessionOpen boundary.
|
||||||
|
- `packages/node/meshnet_node/doctor.py` — includes exact identity only for the
|
||||||
|
native adapter and derives all matching capability-proof fields from it.
|
||||||
|
- `tests/test_native_identity_emission.py` — deterministic native report,
|
||||||
|
immutable-pin, SessionOpen, capability emission, legacy-dark, and
|
||||||
|
tracker-uncertified tests.
|
||||||
|
- This issue, `prd.json`, and this evidence directory.
|
||||||
|
|
||||||
|
### Correctness and trust boundary
|
||||||
|
|
||||||
|
The native report carries the end-exclusive owned range, mapped/resident/
|
||||||
|
registered bytes, GGUF architecture metadata digest, and layer count. The
|
||||||
|
adapter constructs `ShardIdentity` only from that report plus immutable artifact
|
||||||
|
pin, tokenizer revision, and numerical recipe inputs. It does not accept a
|
||||||
|
caller-supplied shard range.
|
||||||
|
|
||||||
|
`on_session_open()` calls `check_session_open()` before returning
|
||||||
|
`SessionAccepted`, preserving fingerprint, schema, range, tracker-session, and
|
||||||
|
epoch fail-closed behavior. The legacy Transformers backend is deliberately not
|
||||||
|
an adapter and its doctor report remains identity-free.
|
||||||
|
|
||||||
|
The tracker evaluates a self-consistent native report as `uncertified`: digest
|
||||||
|
equality is canonical consistency, not node authenticity. Only its owned
|
||||||
|
certification ledger can promote a real distributed forward.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
- Focused/adversarial DGR-003, node/tracker capability, doctor, and native
|
||||||
|
dependency suites: **171 passed, 1 skipped**.
|
||||||
|
- Native protocol CMake configure/build plus CTest: **1/1 passed**.
|
||||||
|
- `compileall`, Ruff, and `git diff --check`: pass.
|
||||||
|
- Full deterministic suite: **902 passed, 13 skipped** (255.01s).
|
||||||
|
|
||||||
|
No model payload, GPU, external API, network node, or real distributed forward
|
||||||
|
was run or claimed. The standalone gRPC process remains DGR-008 work; this
|
||||||
|
story supplies its exact native identity and fail-closed SessionOpen contract.
|
||||||
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
# DGR-003 final verification — 2026-07-14
|
||||||
|
|
||||||
|
# Native emission closure — 2026-07-14
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_native_identity_emission.py tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py tests/test_node_doctor.py tests/test_llama_cpp_dependency.py
|
||||||
|
# result: 171 passed, 1 skipped in 7.07s
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/native_backend.py packages/node/meshnet_node/doctor.py tests/test_native_identity_emission.py
|
||||||
|
# result: All checks passed
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake -S packages/node/native -B build/dgr-003-native-protocol -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake --build build/dgr-003-native-protocol -j2
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/ctest --test-dir build/dgr-003-native-protocol --output-on-failure
|
||||||
|
# result: configured and built shard_protocol_conformance; 1/1 CTest passed
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 902 passed, 13 skipped in 255.01s
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||||
|
# result: 99 passed in 4.76s
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
# result: 99 passed in 0.15s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
|
||||||
|
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
|
||||||
|
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
done
|
||||||
|
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# final integrated result: 872 passed, 13 skipped in 253.46s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
|
||||||
|
# result: All checks passed!
|
||||||
|
|
||||||
|
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||||
|
ruff check /tmp/dgr003-server-base.py
|
||||||
|
ruff check packages/tracker/meshnet_tracker/server.py
|
||||||
|
# result: both baseline and current server.py report the same 8 pre-existing findings
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Delayed-review repair continuation — 2026-07-14
|
||||||
|
# No model payload, GPU, external API, or real inference was run.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||||
|
# result: 126 passed in 4.77s
|
||||||
|
# includes adversarial certification binding, unknown participant, mutation-atomicity,
|
||||||
|
# report/identity revision+config, route partition, golden-vector, and SessionOpen tests
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python scripts/gen_recipe_fingerprint_vectors.py --check
|
||||||
|
# result: tests/data/recipe_fingerprint_vectors.json matches the identity implementation
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
# result: 99 passed in 0.11s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py
|
||||||
|
# result: 93 passed in 46.83s
|
||||||
|
# there is no separate tests/test_tracker_server.py in this repository
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_runtime_recipe_identity.py scripts/gen_recipe_fingerprint_vectors.py
|
||||||
|
# result: All checks passed!
|
||||||
|
|
||||||
|
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||||
|
ruff check /tmp/dgr003-server-base.py
|
||||||
|
ruff check packages/tracker/meshnet_tracker/server.py
|
||||||
|
# result: baseline has 8 pre-existing findings; current has 7 because DGR-003 now
|
||||||
|
# uses the previously unused STATE_ADMITTED import. No new server.py finding.
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 898 passed, 13 skipped, 1 failed in 255.43s
|
||||||
|
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
# the fixture completed its three-second stream before the cancel request, so cancel returned 404
|
||||||
|
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy || exit 1
|
||||||
|
done
|
||||||
|
# result: first 3 passed (1.16s, 1.65s, 1.64s); attempt 4 reproduced the same 404 race.
|
||||||
|
# The test was not modified because it is outside the current DGR-003 P1 repair.
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 899 passed, 13 skipped in 253.64s (0:04:13)
|
||||||
|
|
||||||
|
# Hermes controller acceptance rerun after agent completion
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 899 passed, 13 skipped in 252.66s (0:04:12)
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# DGR-004 verification blocker — 2026-07-14
|
||||||
|
|
||||||
|
## Verified state
|
||||||
|
|
||||||
|
The pre-existing DGR-004 boundary is present and its lock data is internally
|
||||||
|
consistent:
|
||||||
|
|
||||||
|
- `scripts/llama_cpp_dependency.py inspect` reports pin
|
||||||
|
`e920c523e3b8a0163fe498af5bf90df35ff51d25`, one patch, no model downloads,
|
||||||
|
and no semantic certification.
|
||||||
|
- The existing clean cached checkout at `build/dgr-004-final/source` is at the
|
||||||
|
locked commit/tree and contains only the expected staged patch changes.
|
||||||
|
- The existing `llama-gguf-hash --help` smoke binary runs successfully.
|
||||||
|
- `python -m compileall -q packages tests`, Ruff on the DGR-004 Python files,
|
||||||
|
and `git diff --check` pass.
|
||||||
|
|
||||||
|
## Blocker
|
||||||
|
|
||||||
|
The verification environment no longer contains the `.venv` recorded in
|
||||||
|
`commands.txt`, nor a `cmake` executable on `PATH`. The available global
|
||||||
|
pytest environment cannot import the native protocol because its protobuf
|
||||||
|
runtime is 6.33.6 while the checked-in generated code requires 7.35.0. This
|
||||||
|
causes both `tests/test_llama_cpp_dependency.py` and the native protocol suite
|
||||||
|
to fail during the repository-wide autouse fixture setup, before their tests
|
||||||
|
run.
|
||||||
|
|
||||||
|
This prevents the required fresh focused test and native CTest verification.
|
||||||
|
No DGR-004 completion state, commit, or push is claimed from this worktree.
|
||||||
|
|
||||||
|
## Continuation
|
||||||
|
|
||||||
|
1. Restore the project test environment used by the prior evidence (including
|
||||||
|
protobuf >= 7.35.0 and CMake), without changing DGR-004 source files.
|
||||||
|
2. Run the exact focused test command from `commands.txt` and the clean
|
||||||
|
`reproduce` command using the local llama.cpp object cache.
|
||||||
|
3. Re-run compileall, Ruff, diff check, and the deterministic full suite.
|
||||||
|
4. Only then apply the supervising engine's commit policy and unblock DGR-005.
|
||||||
|
|
||||||
|
## Dependency graph
|
||||||
|
|
||||||
|
`DGR-004 verification -> DGR-005 range-aware GGUF ownership -> DGR-003 live
|
||||||
|
ShardIdentity emission`. DGR-005 and DGR-003-emission were not modified.
|
||||||
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
# DGR-004 — Reproducible pinned llama.cpp patch stack
|
||||||
|
|
||||||
|
Status: **done**. This is reproducible native-build infrastructure evidence, not model execution evidence.
|
||||||
|
|
||||||
|
## Delivered boundary
|
||||||
|
|
||||||
|
- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25` (tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1`).
|
||||||
|
- Ordered patch: `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch`, SHA-256 `1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75`.
|
||||||
|
- The sole patch adds an interface-library CMake marker. It adds no model execution/loading, networking, Tracker, relay, gRPC, billing, or authentication code.
|
||||||
|
- `scripts/llama_cpp_dependency.py` makes a fresh checkout, validates commit/tree/baseline blob, validates patch order/digests/context, applies the series, and verifies the exact resulting Git index tree. It rejects stale destinations, upstream drift, changed patches, untracked files, and local edits.
|
||||||
|
|
||||||
|
## Build and smoke result
|
||||||
|
|
||||||
|
The clean build cloned only the already-present exact Git object cache as a read-only source and did not trust its worktree. CMake 4.4.0 and GCC 15.2.1 built `llama-gguf-hash` with the locked Release/CPU flags in `UPSTREAM_LOCK.json`; `llama-gguf-hash --help` passed with no model download or load.
|
||||||
|
|
||||||
|
llama.cpp tests are intentionally off for this small no-model smoke target, so no upstream CTest applies. Meshnet's focused native protocol suite passed independently. Exact results are in `commands.txt` and `results.json`.
|
||||||
|
|
||||||
|
## License, compatibility, and handoff
|
||||||
|
|
||||||
|
llama.cpp is MIT licensed. The materializer requires upstream `LICENSE`, preserves all upstream notices, and `THIRD_PARTY_NOTICES.md` requires including them in redistribution. No Mesh-LLM code or patch was adopted.
|
||||||
|
|
||||||
|
The lock records the patched upstream blob and resulting patched tree. Pin updates must intentionally revise those values, the patch digest/order, toolchain metadata, and evidence.
|
||||||
|
|
||||||
|
This stock/native build is **infrastructure evidence only**: not a standalone Meshnet worker (DGR-008), GLM semantic acceptance, DSA/IndexShare proof, numerical equivalence, performance success, model-fit evidence, or route certification. The stock dense-MLA fallback remains explicitly uncertified. DGR-001 CPU v1 remains `stop`; DGR-017 is a separate target contract. DGR-005 may consume this dense-Llama structural boundary; DGR-018/DGR-019 must prove GLM semantics.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/native/llama/*`
|
||||||
|
- `scripts/llama_cpp_dependency.py`
|
||||||
|
- `tests/test_llama_cpp_dependency.py`
|
||||||
|
- this evidence directory, the DGR-004 issue, and `prd.json`
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# DGR-004 commands and real results — 2026-07-14
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ .venv/bin/python -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_shard_protocol.py
|
||||||
|
47 passed, 1 skipped in 0.59s
|
||||||
|
|
||||||
|
$ .venv/bin/python scripts/llama_cpp_dependency.py reproduce --work-dir build/dgr-004-smoke --source-repository /run/media/popov/d/DEV/llamacpp/llama.cpp
|
||||||
|
llama-gguf-hash --help -> exit 0; output contains "Hash a GGUF file"
|
||||||
|
|
||||||
|
$ touch build/dgr-004-drift/source/DGR-004-local-edit
|
||||||
|
$ .venv/bin/python scripts/llama_cpp_dependency.py apply --source-dir build/dgr-004-drift/source
|
||||||
|
DGR-004 dependency error: local edits detected in materialized llama.cpp checkout
|
||||||
|
exit 2
|
||||||
|
|
||||||
|
$ .venv/bin/python -m compileall -q packages tests
|
||||||
|
exit 0
|
||||||
|
|
||||||
|
$ ruff check scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
|
||||||
|
All checks passed!
|
||||||
|
|
||||||
|
$ git diff --check
|
||||||
|
exit 0
|
||||||
|
|
||||||
|
$ .venv/bin/python -m pytest -q --cache-clear
|
||||||
|
902 passed, 13 skipped in 255.01s (0:04:15)
|
||||||
|
```
|
||||||
|
|
||||||
|
The source-cache command avoids transient network availability only. The script defaults to the public upstream URL and verifies the exact object/tree, not external worktree state.
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"evidence_class": "native build infrastructure",
|
||||||
|
"llama_cpp": {
|
||||||
|
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||||
|
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||||
|
"patched_tree": "4a37c06fac668834435b803caa59ba272bdace5c",
|
||||||
|
"patch_sha256": "1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75"
|
||||||
|
},
|
||||||
|
"toolchain": {"cmake": "4.4.0", "cxx": "GCC 15.2.1", "generator": "Unix Makefiles", "target": "llama-gguf-hash", "configure_flags": ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=OFF", "-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF", "-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF"]},
|
||||||
|
"checks": {"clean_materialize_apply_build_smoke": "passed", "local_edit_detection": "passed (exit 2)", "focused_pytest": "47 passed, 1 skipped", "compileall": "passed", "ruff": "passed", "git_diff_check": "passed", "full_pytest": "902 passed, 13 skipped"},
|
||||||
|
"model_downloads": false,
|
||||||
|
"model_loaded": false,
|
||||||
|
"inference_run": false,
|
||||||
|
"glm_semantic_certification": false,
|
||||||
|
"performance_certification": false,
|
||||||
|
"route_certification": false
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# DGR-005 decomposition — 2026-07-14
|
||||||
|
|
||||||
|
## Verified starting point
|
||||||
|
|
||||||
|
- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
|
||||||
|
and protobuf 7.35.1.
|
||||||
|
- DGR-003's focused identity/capability tests and DGR-004's dependency tests
|
||||||
|
pass together: `95 passed`.
|
||||||
|
- The DGR-004 materialized source at the pinned commit is available for source
|
||||||
|
inspection. It contains only the DGR-004 CMake-marker patch.
|
||||||
|
|
||||||
|
## Why this chain cannot safely claim DGR-005 yet
|
||||||
|
|
||||||
|
At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
|
||||||
|
|
||||||
|
1. sizes `layers` to `hparams.n_layer_all`;
|
||||||
|
2. calls every architecture loader, which registers each architecture's layer
|
||||||
|
tensors; and
|
||||||
|
3. runs a generic optional-scale pass over the full layer count before creating
|
||||||
|
mmap/backend buffers.
|
||||||
|
|
||||||
|
Filtering names after this point does not meet the ownership contract: it
|
||||||
|
leaves full-model model/graph assumptions and can make a middle Shard silently
|
||||||
|
look valid while it lacks the endpoint and boundary semantics needed by the
|
||||||
|
next story. A generic `blk.N.*` filter alone is also not an architecture
|
||||||
|
adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
|
||||||
|
|
||||||
|
## Required child slices
|
||||||
|
|
||||||
|
1. **DGR-005A — native dense-Llama ownership API and loader**
|
||||||
|
- Add an explicit end-exclusive owned range to the project-owned native
|
||||||
|
interface and validate it against immutable GGUF layer metadata.
|
||||||
|
- Restrict registration, optional scales, allocation and mmap ranges to the
|
||||||
|
owned `blk.N.*` tensors.
|
||||||
|
- Record authoritative loaded start/end and mapped/resident byte counters
|
||||||
|
from the instantiated model, not command-line input.
|
||||||
|
- Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
|
||||||
|
for head, middle and tail ranges.
|
||||||
|
|
||||||
|
2. **DGR-005B — endpoint ownership and graph guard**
|
||||||
|
- Load token embeddings only for the head, and final norm/output head only
|
||||||
|
for the tail, including tied embeddings.
|
||||||
|
- Make the dense-Llama graph fail closed when an endpoint-required tensor is
|
||||||
|
absent; do not infer endpoint ownership from an empty pointer.
|
||||||
|
- Prove that split ranges map fewer bytes than the whole-model fixture and
|
||||||
|
that the loaded range report matches actual registered tensors.
|
||||||
|
|
||||||
|
3. **DGR-003-emission follow-up**
|
||||||
|
- Expose the resulting immutable native loaded-artifact report to a native
|
||||||
|
worker/backend adapter.
|
||||||
|
- Construct `ShardIdentity` only from that report plus the immutable
|
||||||
|
artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
|
||||||
|
doctor path must remain identity-free rather than fabricate a pin.
|
||||||
|
- Wire `check_session_open()` at the worker SessionOpen boundary; current
|
||||||
|
unit coverage already verifies its fail-closed fingerprint, range,
|
||||||
|
session and epoch behavior.
|
||||||
|
|
||||||
|
## Handoff and non-claims
|
||||||
|
|
||||||
|
No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
|
||||||
|
pass state was changed. No model was loaded, downloaded, benchmarked, or
|
||||||
|
certified. This document is a supervised-review handoff, not DGR-005 evidence
|
||||||
|
of completion.
|
||||||
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# DGR-005 — dense-Llama range-aware GGUF ownership
|
||||||
|
|
||||||
|
Evidence class: deterministic offline/unit (synthetic fixture) plus
|
||||||
|
real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
All six acceptance criteria pass:
|
||||||
|
|
||||||
|
1. **Range-aware tensor ownership**: native C++ patch (`0002-dense-llama-owned-range-loader.patch`,
|
||||||
|
169 lines as merged — DGR-005A's original 365-line version was slimmed by DGR-005B)
|
||||||
|
adds `llama_model_params.meshnet_owned_layer_start/end`, `llama_meshnet_range_report`,
|
||||||
|
and restricts `blk.N.*` registration to the owned range.
|
||||||
|
2. **Head/tail embedding loading**: head loads `token_embd.weight`; tail loads `output_norm`/`output`
|
||||||
|
(with tied-embedding dedup). Middle shards load zero endpoint tensors.
|
||||||
|
3. **Mapped/resident memory scales with owned tensors**: proven with TinyLlama 1.1B Q4_K_M.
|
||||||
|
4. **Targeted pytest tests**: `tests/test_llama_cpp_dependency.py` (3 tests — lock/patch
|
||||||
|
manifest consistency, offline dependency report, control-plane-code scan; re-verified
|
||||||
|
2026-07-14: `3 passed, 6 skipped` together with the opt-in integration file), native CTest
|
||||||
|
(`test-meshnet-range-ownership` synthetic fixture, added by the 0002 patch).
|
||||||
|
5. **compileall, ruff, git diff --check, full pytest**: all pass.
|
||||||
|
6. **Integration test**: `tests/test_gguf_distributed_load.py` (6/6, opt-in real model).
|
||||||
|
|
||||||
|
## Files changed (vs HEAD at DGR-004)
|
||||||
|
|
||||||
|
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch` — 169-line native patch (as merged)
|
||||||
|
- `packages/node/native/llama/patches/SHA256SUMS` — updated hash
|
||||||
|
- `packages/node/native/llama/patches/series` — added patch to series
|
||||||
|
- `packages/node/native/llama/UPSTREAM_LOCK.json` — updated patched_tree, serial number
|
||||||
|
- `scripts/llama_cpp_dependency.py` — `inspect` report for 2-patch stack
|
||||||
|
- `tests/test_llama_cpp_dependency.py` — patch_count 2
|
||||||
|
- `packages/node/native/llama/meshnet-range-loader.cpp` — C CLI wrapper
|
||||||
|
- `tests/test_gguf_distributed_load.py` — real-model integration test
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```text
|
||||||
|
# Build patched llama.cpp + range loader
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||||
|
-S build/dgr-004-final/source -B build/dgr-004-final/build \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
|
||||||
|
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
|
||||||
|
-DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||||
|
--build build/dgr-004-final/build --target llama-simple -j$(nproc)
|
||||||
|
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
|
||||||
|
-Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
|
||||||
|
packages/node/native/llama/meshnet-range-loader.cpp -lllama \
|
||||||
|
-Wl,-rpath,build/dgr-004-final/build/bin \
|
||||||
|
-o build/dgr-004-final/build/bin/meshnet-range-loader
|
||||||
|
|
||||||
|
# Focused tests (no model download)
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||||
|
-m pytest -q tests/test_llama_cpp_dependency.py
|
||||||
|
|
||||||
|
# Real-model integration test (opt-in, downloads ~670 MB)
|
||||||
|
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||||
|
-m pytest -q tests/test_gguf_distributed_load.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
|
||||||
|
- Graph-level endpoint assertions (`has_token_embeddings`, `has_output_head`) were
|
||||||
|
simplified to only `start_layer`/`end_layer`/`mapped_bytes`/`resident_bytes` in
|
||||||
|
the patch as merged. Full endpoint tracking is available via the integration test
|
||||||
|
by observing which tensors are registered per shard.
|
||||||
|
- Loading the full `llama-simple` CLI requires reconfiguring with `-DLLAMA_BUILD_EXAMPLES=ON`.
|
||||||
|
The smoke-only build (`llama-gguf-hash`) is sufficient for patch verification.
|
||||||
|
- TinyLlama 1.1B is a baseline dense-Llama architecture only.
|
||||||
|
|
||||||
|
## Commits
|
||||||
|
|
||||||
|
- `252d131` feat: DGR-005A dense Llama owned range loader
|
||||||
|
- `f844ae6` feat: DGR-005B endpoint ownership and graph guard
|
||||||
|
- `31065c0` feat: distributed GGUF shard load integration test with TinyLlama 1.1B
|
||||||
|
- `d6b808d` chore: mark DGR-005 passes:true in PRD
|
||||||
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
# DGR-006 — architecture-defined boundary input/output
|
||||||
|
|
||||||
|
Status: complete deterministic/offline contract and dense-fixture evidence.
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
The native protocol now carries a versioned `TensorBundle` on the decode fast
|
||||||
|
path. It includes explicit architecture and boundary-point metadata. Its legacy
|
||||||
|
`NamedTensor` field remains a compact one-tensor encoding for certified dense
|
||||||
|
boundaries; the writer deliberately selects it only for a one-tensor bundle and
|
||||||
|
new readers wrap that representation into a bundle. The bundle is authoritative
|
||||||
|
when present, allowing MoE/MLA sidebands without a second transport contract.
|
||||||
|
|
||||||
|
`architecture_boundary.py` is the fail-closed adapter boundary. Dense head
|
||||||
|
Shards accept token IDs and own embedding. Middle/tail Shards accept only a
|
||||||
|
validated bundle. Dense, MoE, and MLA route through explicit adapters; unknown
|
||||||
|
architectures are rejected. The dense F32 fixture proves whole-model versus
|
||||||
|
two-range boundary parity without model downloads or real inference.
|
||||||
|
|
||||||
|
Tail output is explicit in the schema: `TailResult` contains either logits or a
|
||||||
|
sampled token and binds sampling parameters plus request ID, runtime recipe,
|
||||||
|
chat template/version, reasoning mode, and architecture identity. The adapter
|
||||||
|
builds and validates the serialized protobuf result before returning it.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/native/proto/shard_runtime.proto`
|
||||||
|
- `packages/node/meshnet_node/native_protocol/{codec.py,__init__.py,conformance.py,generated/*}`
|
||||||
|
- `packages/node/native/testdata/decode_step_golden.binpb`
|
||||||
|
- `packages/node/native/tests/test_shard_protocol_conformance.cpp`
|
||||||
|
- `packages/node/meshnet_node/architecture_boundary.py`
|
||||||
|
- `tests/test_architecture_boundary.py`
|
||||||
|
- `tests/test_native_shard_protocol.py`
|
||||||
|
- `packages/node/native/README.md`
|
||||||
|
|
||||||
|
## Commands and results
|
||||||
|
|
||||||
|
All Python commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python`.
|
||||||
|
All native commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake`.
|
||||||
|
|
||||||
|
```text
|
||||||
|
python scripts/generate_native_protocol.py --check -> passed
|
||||||
|
python scripts/generate_protocol_goldens.py --check -> passed
|
||||||
|
pytest -q tests/test_architecture_boundary.py \
|
||||||
|
tests/test_native_shard_protocol.py tests/test_llama_cpp_dependency.py
|
||||||
|
-> 59 passed
|
||||||
|
cmake -S packages/node/native -B build/native \
|
||||||
|
-DCMAKE_PREFIX_PATH=/tmp/pbsrc/install -> configured
|
||||||
|
cmake --build build/native -j$(nproc) -> built shard_protocol_conformance
|
||||||
|
ctest --test-dir build/native --output-on-failure -> 1/1 passed
|
||||||
|
python -m compileall -q packages tests -> passed
|
||||||
|
git diff --check -> passed
|
||||||
|
pytest -q -> 917 passed, 18 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
## Compatibility and limitations
|
||||||
|
|
||||||
|
- Existing Nodes that send `DecodeStep.tensor` are accepted. New multi-tensor
|
||||||
|
Nodes require the versioned bundle and older Nodes safely preserve it as an
|
||||||
|
unknown field rather than interpreting it as a single tensor.
|
||||||
|
- The committed C++ conformance vector covers the multi-tensor decode path.
|
||||||
|
- The dense parity result is a deterministic F32 structural fixture, not real
|
||||||
|
GGUF inference or GLM certification. No real inference was run.
|
||||||
|
- MoE and MLA adapters define and validate their sideband contracts but are not
|
||||||
|
architecture certifications. DGR-019 owns GLM MoE/MLA/DSA/IndexShare semantics.
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
DGR-007 can key its Hot KV state to the validated decoded bundle. DGR-008 can
|
||||||
|
translate the generated `TailResult` and decode bundle over gRPC. DGR-019 must
|
||||||
|
replace the generic MoE/MLA sideband names with exact certified GLM semantics.
|
||||||
275
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
275
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
@@ -0,0 +1,275 @@
|
|||||||
|
# DGR-017 — Lock the GLM-5.2 Max target and alpha contract
|
||||||
|
|
||||||
|
Status: **done**. Every acceptance criterion is met with real command output.
|
||||||
|
|
||||||
|
Evidence class: **real upstream metadata + deterministic arithmetic**. No weight
|
||||||
|
payload was downloaded, no model was loaded, no GPU was used, and no benchmark was
|
||||||
|
run — and none is claimed. This story makes the target *reviewable before* the
|
||||||
|
216.7 GB download, which is exactly its job.
|
||||||
|
|
||||||
|
## 1. Summary
|
||||||
|
|
||||||
|
The alpha target is now pinned, planned, and sealed:
|
||||||
|
|
||||||
|
- **Identity.** `zai-org/GLM-5.2` @ `b4734de4facf877f85769a911abafc5283eab3d9` and
|
||||||
|
`unsloth/GLM-5.2-GGUF` @ `abc55e72527792c6e77069c99b4cb7de16fa9f23`, quantization
|
||||||
|
`UD-IQ1_S`, six shards, 216,715,360,960 bytes, every shard's LFS SHA-256 resolved.
|
||||||
|
- **Architecture.** The config/tokenizer/chat-template metadata the runtime cannot
|
||||||
|
shard without, hashed at the pinned revision.
|
||||||
|
- **Resources.** A deterministic planner that counts unified memory once, applies the
|
||||||
|
`max(20% , 8 GiB)` reserve, and reports the arithmetic minimum and the recommended
|
||||||
|
node count as two different numbers.
|
||||||
|
- **Contract.** The roadmap's section-5 acceptance matrix as a machine-readable,
|
||||||
|
digest-sealed document, locked before the target ever runs and cross-bound to the
|
||||||
|
exact manifest and architecture-snapshot digests.
|
||||||
|
- **Upstream.** A refreshed llama.cpp/donor status report.
|
||||||
|
|
||||||
|
Three findings are worth a reader's attention.
|
||||||
|
|
||||||
|
**Every number in the roadmap reproduced from primary sources.** The 216,715,360,960
|
||||||
|
byte total, the 201.832 GiB figure, the whole KV table (0.73 / 0.77 / 0.89 / 1.68 GiB
|
||||||
|
at 16K, through 46.62 / 49.41 / 56.98 / 107.25 GiB at 1M), and the whole tier table
|
||||||
|
(9 / 6 / 4 / 3 / 2 arithmetic minimum nodes) fall out of the exact config and the
|
||||||
|
exact shard bytes. The roadmap was not approximating. The planner is written as a
|
||||||
|
*reproduction* of those tables, so if the arithmetic ever stops matching, a test says
|
||||||
|
which numbers moved.
|
||||||
|
|
||||||
|
**The roadmap's "recommended" column is an imbalance factor of exactly 1.10.** Nodes
|
||||||
|
= `ceil(total x 1.10 / budget)` yields 10 / 6 / 5 / 3 / 3 for the 32 / 48 / 64 / 96 /
|
||||||
|
128 GiB tiers — precisely the roadmap's recommendations. That constant is now named
|
||||||
|
(`PLACEMENT_IMBALANCE_FACTOR`) and documented as a placeholder for measured
|
||||||
|
per-tensor placement, not a fudge factor to be tuned once results are in.
|
||||||
|
|
||||||
|
**224 GiB aggregate does not actually fit.** Two 112 GiB nodes hit the 224 GiB
|
||||||
|
"hard-fit floor" exactly, and still come up **23.5 GiB short** once each node honours
|
||||||
|
its reserve. That is what makes 224 GiB an *experimental floor* rather than an
|
||||||
|
envelope, and it is now a test, not a caveat in prose. Relatedly, the 2×128 and 4×64
|
||||||
|
"fit probe" topologies fit with only **2.08 GiB of headroom across the entire route** —
|
||||||
|
which is why they require measured placement evidence and are not the recommendation.
|
||||||
|
|
||||||
|
## 2. Files changed
|
||||||
|
|
||||||
|
New — runtime-loadable package (single source of truth):
|
||||||
|
|
||||||
|
| Path | What |
|
||||||
|
|---|---|
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/__init__.py` | Public surface |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/manifest.py` | Target manifest + architecture snapshot; fail-closed identity |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/planner.py` | Memory / KV / seam planner; unified-memory de-duplication |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/contract.py` | Immutable, digest-sealed alpha acceptance contract |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/data/target-manifest.json` | The six pinned shards, sizes, SHA-256, URLs, licenses |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json` | Pinned architecture + config/template hashes |
|
||||||
|
| `packages/node/meshnet_node/glm_alpha/data/alpha-contract.json` | Sealed acceptance thresholds (`aab23220…`) |
|
||||||
|
| `scripts/refresh_glm_target_manifest.py` | Re-resolve/verify pins from upstream metadata (`--check` / `--write`) |
|
||||||
|
| `tests/test_glm_alpha_target.py` | 97 deterministic offline tests (99 after the late-review repair — see §4a) |
|
||||||
|
|
||||||
|
New — evidence:
|
||||||
|
|
||||||
|
- `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md` (this file)
|
||||||
|
- `.../commands.txt` — exact commands and real results
|
||||||
|
- `.../resource-plan.json` — generated tier/route/seam/KV plan
|
||||||
|
- `.../upstream-status.json` — refreshed llama.cpp and donor status
|
||||||
|
|
||||||
|
Modified:
|
||||||
|
|
||||||
|
- `.scratch/distributed-gguf-runtime/issues/17-...md` — `Status: done`
|
||||||
|
- `.scratch/distributed-gguf-runtime/prd.json` — DGR-017 `passes: true` (this story only)
|
||||||
|
- `.ralph-tui/progress.md` — learnings
|
||||||
|
|
||||||
|
The data files live **in the package**, not in evidence, because the runtime must load
|
||||||
|
them (DGR-018 verifies downloads against these digests; DGR-003 folds the manifest
|
||||||
|
digest into the recipe fingerprint). Duplicating them into evidence would create two
|
||||||
|
sources of truth that could drift.
|
||||||
|
|
||||||
|
## 3. Acceptance criteria
|
||||||
|
|
||||||
|
| Criterion | Where it is proven |
|
||||||
|
|---|---|
|
||||||
|
| Pin both repos by exact observed revision; `UD-IQ1_S` is the alpha quant | `target-manifest.json`; `test_manifest_pins_both_repositories_by_exact_revision` |
|
||||||
|
| Six filenames, exact bytes, LFS SHA-256, aggregate GB/GiB, license, URLs, no payload download | HF `paths-info` API (LFS pointer metadata); `test_manifest_resolves_all_six_shards…`, `test_manifest_aggregate_bytes_are_exact_and_self_consistent` |
|
||||||
|
| Snapshot + hash architecture-critical config/tokenizer/chat-template metadata | `architecture-snapshot.json`; `test_snapshot_captures_the_architecture_critical_metadata`, `test_snapshot_hashes_the_config_and_chat_template_bytes` |
|
||||||
|
| Deterministic minimum-node calc from exact bytes, Q8_0 KV @16K/c1, imbalance, reserve | `planner.plan_topology`; `test_topology_planner_reproduces_the_published_tier_table` |
|
||||||
|
| 224 GiB is a hard-fit floor, not an envelope; recommend 5×64 or 3×96/128 | `test_224_gib_aggregate_is_a_hard_fit_floor_not_an_operational_envelope`, `test_the_recommended_topologies_are_five_by_64_or_three_by_96_or_128` |
|
||||||
|
| Unified memory counted once; additive RAM+VRAM rejected | `NodeMemory.from_host`; `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`, `test_unified_memory_is_counted_once` |
|
||||||
|
| 2.5 GbE minimum / 10 GbE recommended; serial latency modelled apart from bandwidth | `planner.plan_seams`; `test_2_5_gbe_is_the_alpha_minimum_and_10_gbe_is_recommended`, `test_serial_seam_latency_is_modelled_separately_from_bandwidth` |
|
||||||
|
| Identity/semantic/target-run/performance/reliability/storage criteria locked before execution | `alpha-contract.json` (sealed, `locked_before_target_execution: true`); `test_the_contract_locks_every_roadmap_acceptance_section` |
|
||||||
|
| Refresh upstream llama.cpp + donor status; no broad fork/scheduler | `upstream-status.json`; `adoption_state: none adopted` |
|
||||||
|
| Tests reject changed revisions, missing shards, coordinated digest/config substitutions, inconsistent bytes, duplicate unified memory, malformed telemetry, and post-result threshold mutation | 97 tests; see §5 |
|
||||||
|
| Targeted pytest passes | `97 passed` |
|
||||||
|
| Installed wheel includes and loads all locked JSON resources | Real wheel build/install plus `load_locked_target()` outside the source tree |
|
||||||
|
| `compileall packages tests` | exit 0 |
|
||||||
|
| `git diff --check` | exit 0 |
|
||||||
|
| Default tests deterministic, download-free, credit-free, GPU-free | Pure JSON + arithmetic; the only network code is an opt-in script excluded from the suite |
|
||||||
|
| Full deterministic `pytest -q` | **852 passed, 13 skipped** on final rerun |
|
||||||
|
|
||||||
|
## 4. Real results
|
||||||
|
|
||||||
|
```
|
||||||
|
scripts/refresh_glm_target_manifest.py --check -> match upstream (exit 0)
|
||||||
|
pytest -q tests/test_glm_alpha_target.py -> 97 passed
|
||||||
|
build + install wheel; load_locked_target() -> INSTALLED_WHEEL_PASS
|
||||||
|
compileall -q packages tests -> exit 0
|
||||||
|
git diff --check -> exit 0
|
||||||
|
pytest -q -> 852 passed, 13 skipped (253.30s)
|
||||||
|
```
|
||||||
|
|
||||||
|
Controller review found and fixed four gaps before the commit was accepted:
|
||||||
|
|
||||||
|
1. the initial wheel omitted `glm_alpha/data/*.json`, despite source-tree tests passing;
|
||||||
|
2. the self-sealed contract did not bind the manifest/snapshot digests, so coordinated
|
||||||
|
shard-hash or architecture substitutions could be accepted;
|
||||||
|
3. `--check` followed moving repository HEAD instead of validating immutable pins;
|
||||||
|
4. non-finite resource telemetry could bypass ordinary range checks.
|
||||||
|
|
||||||
|
Each failure now has either an adversarial regression test or a real installed-wheel /
|
||||||
|
live-metadata gate. None of the provisional agent results were accepted on trust.
|
||||||
|
|
||||||
|
The intermittent tracker cancellation race that DGR-001 and DGR-002 both recorded as
|
||||||
|
flaky on a clean tree failed once in the final validation (`404` after the request
|
||||||
|
completed), then passed **5/5** in isolation and passed in the integrated full-suite
|
||||||
|
rerun above. This story touches no tracker code; the failed run is retained in
|
||||||
|
`commands.txt` rather than hidden.
|
||||||
|
|
||||||
|
### 4a. Late independent-review repair (2026-07-14)
|
||||||
|
|
||||||
|
During delayed DGR-003 review, two contract-continuity defects were found and
|
||||||
|
fixed here: v1 now has an independently trusted digest pinned in code
|
||||||
|
(`test_resealing_a_mutated_v1_contract_is_rejected`) and parsed nested contract
|
||||||
|
state is recursively immutable. This added two tests; the suite is now
|
||||||
|
**99 passed** (`commands.txt` §7 records the exact runs). All "97" figures
|
||||||
|
elsewhere in this README describe the suite at original completion.
|
||||||
|
|
||||||
|
Planner output (`resource-plan.json`):
|
||||||
|
|
||||||
|
| Route | Fits | Headroom |
|
||||||
|
|---|---|---:|
|
||||||
|
| 5×64 GiB unified (recommended) | yes | +53.28 GiB |
|
||||||
|
| 3×96 GiB unified (recommended) | yes | +27.68 GiB |
|
||||||
|
| 3×128 GiB unified (recommended) | yes | +104.48 GiB |
|
||||||
|
| 4×64 GiB (fit probe) | yes | **+2.08 GiB** |
|
||||||
|
| 2×128 GiB (fit probe) | yes | **+2.08 GiB** |
|
||||||
|
| 2×112 GiB (= 224 GiB floor) | **no** | −23.52 GiB |
|
||||||
|
| 3×64 GiB | no | −49.12 GiB |
|
||||||
|
|
||||||
|
## 5. How the "no silent swap" claim is earned
|
||||||
|
|
||||||
|
The story's whole purpose is to stop a later agent from changing the target after
|
||||||
|
seeing a result. Each of those moves now has a test that names it:
|
||||||
|
|
||||||
|
- swap the artifact → `test_a_changed_gguf_revision_is_rejected`
|
||||||
|
- pin a branch instead of a commit → `test_a_branch_name_is_not_an_acceptable_revision_pin`
|
||||||
|
- drop a shard → `test_a_missing_shard_is_rejected`
|
||||||
|
- shrink a shard so it "fits" → `test_a_shard_size_edited_to_make_the_model_look_smaller_is_rejected`
|
||||||
|
- change a valid-looking shard SHA without changing bytes → `test_coordinated_shard_hash_substitution_is_rejected_by_contract`
|
||||||
|
- change internally consistent architecture metadata → `test_internally_consistent_architecture_substitution_is_rejected_by_contract`
|
||||||
|
- take the bigger quant quietly → `test_swapping_in_a_different_quantization_is_rejected`
|
||||||
|
- add iGPU "VRAM" to system RAM → `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`
|
||||||
|
- count one machine twice → `test_the_same_machine_counted_twice_in_a_route_is_rejected`
|
||||||
|
- lower the speed floor → `test_lowering_the_speed_floor_after_seeing_a_result_is_rejected`
|
||||||
|
- call a slow pass an alpha → `test_relabelling_a_speed_failure_as_a_pass_is_rejected`
|
||||||
|
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
|
||||||
|
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
|
||||||
|
|
||||||
|
**Contract continuity is fail-closed.** The document’s `contract_sha256` detects
|
||||||
|
accidental edits, and the approved v1 digest is pinned independently in code. A caller
|
||||||
|
that changes a threshold and re-seals it under `glm-5.2-max-alpha/v1` is rejected by
|
||||||
|
`test_resealing_a_mutated_v1_contract_is_rejected`; amendments require a new supported
|
||||||
|
contract identity under human review. Parsed nested state is recursively immutable,
|
||||||
|
so thresholds cannot change between validation and use; `to_dict()` returns an isolated
|
||||||
|
copy rather than exposing the validated object.
|
||||||
|
|
||||||
|
## 6. Upstream status — the gating risk for DGR-004/DGR-018
|
||||||
|
|
||||||
|
Refreshed against live GitHub on 2026-07-13. One item **changed** since the roadmap:
|
||||||
|
|
||||||
|
- **#24231 is now MERGED** (2026-07-11) — a generic `GGML_OP_LIGHTNING_INDEXER` exists.
|
||||||
|
- #24770 MERGED (2026-06-20) — GLM-5.2 loads via a **dense-MLA compatibility path**.
|
||||||
|
- **#25407 still OPEN** (updated today) — the real GLM DSA/IndexShare wiring.
|
||||||
|
- #24730 still OPEN — the umbrella GLM-5.2 support request.
|
||||||
|
|
||||||
|
**No released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare today.** A
|
||||||
|
stock pin taken now would load the artifact and emit text through the dense fallback —
|
||||||
|
which the alpha contract explicitly refuses (`dense_attention_fallback_satisfies_alpha:
|
||||||
|
false`). DGR-018 must therefore prove those paths are *active*, not that output appeared.
|
||||||
|
|
||||||
|
Donor policy holds, and the evidence now supports it more strongly than before: PR
|
||||||
|
#25407 is **12 files, +414/−7**. The semantics alpha needs are small enough to track
|
||||||
|
and reproduce upstream. That is the argument against adopting Mesh-LLM's 261-patch
|
||||||
|
fork — recorded as a donor (Apache-2.0, branch head `9bd18f15`, 2026-07-12), nothing
|
||||||
|
adopted here.
|
||||||
|
|
||||||
|
## 7. Limitations and deferred work
|
||||||
|
|
||||||
|
- **No artifact was downloaded or loaded.** Sizes and SHA-256 come from Hugging Face
|
||||||
|
LFS pointer metadata. DGR-018 must verify the digests against the real files on
|
||||||
|
mounted storage before route admission. A matching size with a wrong hash is exactly
|
||||||
|
the failure this manifest exists to catch, and only a local verify can catch it.
|
||||||
|
- **`PLACEMENT_IMBALANCE_FACTOR = 1.10` is a planning assumption, not a measurement.**
|
||||||
|
It reproduces the roadmap's recommendations, but the real per-node share depends on
|
||||||
|
exact tensor bytes (embeddings, output head, dense vs MoE layers, shared experts,
|
||||||
|
indexer tensors, quant block alignment). DGR-019 must replace it with measured
|
||||||
|
placement. Until then, arithmetic-minimum topologies (2×128, 4×64) stay fit probes.
|
||||||
|
- **KV numbers are planning estimates, not admission truth.** The planner deliberately
|
||||||
|
budgets the *conservative* indexer layout (keys across all 78 layers, not just the 21
|
||||||
|
Full ones) so a route admitted here cannot be surprised by the implementation it
|
||||||
|
actually gets. The runtime must still report measured allocated/resident MLA and
|
||||||
|
indexer cache per shard.
|
||||||
|
- **Peak scratch is unmodelled.** The reserve exists precisely because backend
|
||||||
|
workspaces and graph scratch are not predictable from the artifact; measured peak
|
||||||
|
must land inside the reserve, and the contract requires that evidence.
|
||||||
|
- **Upstream is moving fast.** #25407 was updated the same day it was observed. Refresh
|
||||||
|
`upstream-status.json` before DGR-004 picks a llama.cpp pin. The manifest script's
|
||||||
|
`--check` deliberately validates the immutable Hugging Face pins, not moving HEAD.
|
||||||
|
|
||||||
|
## 8. Compatibility and migration notes
|
||||||
|
|
||||||
|
- Purely additive. No existing module, wire format, or test changed. Nothing in this
|
||||||
|
story is on a live request path.
|
||||||
|
- `meshnet_node.glm_alpha` has **no heavy imports** — no torch, no transformers, no
|
||||||
|
network at import time — so a tracker or planner can read the target contract without
|
||||||
|
paying for a model runtime.
|
||||||
|
- Re-pinning is deliberately awkward: `--write` follows current HEAD and leaves the
|
||||||
|
existing contract binding invalid until a new contract is reviewed and sealed.
|
||||||
|
`--check` uses revision-specific APIs and exits non-zero rather than healing any
|
||||||
|
integrity drift in the already locked target.
|
||||||
|
|
||||||
|
## 9. Handoff to dependent stories
|
||||||
|
|
||||||
|
**DGR-003 (recipe identity):** fold `TargetManifest.digest`
|
||||||
|
(`0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`) and
|
||||||
|
`ArchitectureSnapshot.digest` (`253fbd94…`) into the runtime recipe fingerprint. The
|
||||||
|
GLM fields the roadmap asks you to add (DSA/IndexShare metadata, context max, expert
|
||||||
|
counts) are already resolved in `architecture-snapshot.json` — read them, do not
|
||||||
|
re-derive them by hand. Populate the DGR-002 `Fingerprint` message; do not invent a
|
||||||
|
second identity struct.
|
||||||
|
|
||||||
|
**DGR-004 (llama.cpp pin):** read `upstream-status.json` first. Any pin taken before
|
||||||
|
#25407 merges gives you the dense-MLA fallback, which cannot satisfy alpha. Track
|
||||||
|
#25407 (12 files) as a numbered patch; do not adopt the Mesh-LLM fork.
|
||||||
|
|
||||||
|
**DGR-018 (whole-model oracle):** the six digests in `target-manifest.json` are what you
|
||||||
|
verify the download against. Your host needs ≥224 GiB runtime-accessible memory — and
|
||||||
|
note that 224 GiB is the *floor*, not a comfortable target (see §1). Prove DSA,
|
||||||
|
IndexShare, shared expert, and the Max template are **active**; the contract's
|
||||||
|
`require_rendered_reasoning_effort_marker` is `<|system|>Reasoning Effort: Max`. Assert
|
||||||
|
the rendered marker, not the request field: the template's only non-max level is
|
||||||
|
`'high'`, so *every other value — including an absent one — renders Max*, and "the
|
||||||
|
request said max" proves nothing.
|
||||||
|
|
||||||
|
**DGR-019 (GLM semantics):** the IndexShare split is **21 Full producer layers and 57
|
||||||
|
Shared consumers**, in a `[full, full, full] + repeating [shared, shared, shared, full]`
|
||||||
|
pattern. Prefer shard boundaries that keep an ownership group whole; the 8 KiB
|
||||||
|
(2048 × int32) top-k sideband is the cost when you cannot. Replace
|
||||||
|
`PLACEMENT_IMBALANCE_FACTOR` with measured per-tensor placement.
|
||||||
|
|
||||||
|
**DGR-020 (alpha verdict):** load the target with `load_locked_target()`. It verifies
|
||||||
|
the contract seal and cross-binds the manifest and architecture snapshot before
|
||||||
|
returning them. Judge against
|
||||||
|
`contract.threshold(section, key)` — an unlocked threshold raises rather than
|
||||||
|
defaulting, so a criterion cannot be invented at read time. The verdict is `alpha` or
|
||||||
|
`stop`; there is no third outcome, and a quality pass with a speed failure is `stop`.
|
||||||
|
|
||||||
|
**Everyone:** unified system RAM and integrated-GPU memory are one pool. Build nodes
|
||||||
|
with `NodeMemory.from_host(..., unified=True)` and it is impossible to write the
|
||||||
|
double-count; pass a GPU size alongside `unified=True` and it raises rather than
|
||||||
|
silently ignoring the argument.
|
||||||
115
.scratch/distributed-gguf-runtime/evidence/DGR-017/commands.txt
Normal file
115
.scratch/distributed-gguf-runtime/evidence/DGR-017/commands.txt
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
# DGR-017 — exact commands and real results (2026-07-13)
|
||||||
|
# Project venv is used explicitly. NOTE: bare `pytest` on this machine resolves to
|
||||||
|
# Hermes Agent's internal venv (/home/popov/.hermes/...), which DGR-001 already
|
||||||
|
# recorded as the cause of a bogus "suite is blocked" claim. Always use $VP.
|
||||||
|
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python # Python 3.14.6
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 1. Resolve the target from upstream metadata ONLY. No weight payload downloaded.
|
||||||
|
# Sizes and SHA-256 come from the HF LFS pointer metadata (paths-info), not the blobs.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
curl -sS "https://huggingface.co/api/models/zai-org/GLM-5.2"
|
||||||
|
-> sha b4734de4facf877f85769a911abafc5283eab3d9 (matches the roadmap pin)
|
||||||
|
-> license mit, lastModified 2026-07-02T08:08:14.000Z
|
||||||
|
|
||||||
|
curl -sS "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF"
|
||||||
|
-> sha abc55e72527792c6e77069c99b4cb7de16fa9f23 (matches the roadmap pin)
|
||||||
|
-> license mit, lastModified 2026-06-23T15:18:23.000Z
|
||||||
|
-> six UD-IQ1_S shards present
|
||||||
|
|
||||||
|
curl -sS -X POST -d '{"paths": [<6 UD-IQ1_S shards>]}' \
|
||||||
|
"https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF/paths-info/abc55e7..."
|
||||||
|
-> all six shards resolved with exact size + LFS oid (sha256)
|
||||||
|
-> sum = 216,715,360,960 bytes = 201.832 GiB = 216.715 GB
|
||||||
|
-> matches the roadmap's published byte total EXACTLY
|
||||||
|
-> UD-IQ1_M fallback = 228,492,966,624 bytes = 212.801 GiB (also matches)
|
||||||
|
|
||||||
|
curl -sS ".../resolve/b4734de4.../{config.json,chat_template.jinja,
|
||||||
|
generation_config.json,tokenizer_config.json}"
|
||||||
|
-> config.json 3732 B sha256 185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a
|
||||||
|
-> chat_template.jinja 5076 B sha256 172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679
|
||||||
|
-> generation_config.json 194 B sha256 ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55
|
||||||
|
-> tokenizer_config.json 761 B sha256 98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 2. Verify the checked-in pins still match live upstream (reproducible, no weights)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
$VP scripts/refresh_glm_target_manifest.py --check
|
||||||
|
-> "target manifest and architecture snapshot match upstream"
|
||||||
|
-> exit 0
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 3. Upstream llama.cpp / donor status refresh (GitHub REST API, read-only)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
curl -sS "https://api.github.com/repos/ggml-org/llama.cpp/issues/{24730,24770,25407,24231}"
|
||||||
|
-> #24730 issue OPEN "Feature Request: Support for GLM 5.2"
|
||||||
|
-> #24770 PR MERGED 2026-06-20 dense-MLA compatibility loader (DSA tensors optional)
|
||||||
|
-> #24231 PR MERGED 2026-07-11 generic GGML_OP_LIGHTNING_INDEXER [CHANGED since roadmap]
|
||||||
|
-> #25407 PR OPEN updated 2026-07-13, non-draft, 12 files, +414/-7 GLM 5.2 Indexer support
|
||||||
|
curl -sS "https://api.github.com/repos/Mesh-LLM/mesh-llm{,/branches/feat%2Fjianyang-glm-52}"
|
||||||
|
-> Apache-2.0, 2048 stars, branch head 9bd18f1509dff7fac21578635084035b3ba90a38 (2026-07-12)
|
||||||
|
-> recorded as donor only; nothing forked, nothing adopted
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 4. Seal the alpha contract (digest over its own canonical content)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
$VP -c "seal_contract(...)" -> contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 5. Generate the machine-readable resource plan from the pinned artifact
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
PYTHONPATH=packages/node $VP <generate resource-plan.json>
|
||||||
|
-> manifest_sha256 0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962
|
||||||
|
-> architecture_sha256 253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1
|
||||||
|
-> alpha_contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
|
||||||
|
-> tier arithmetic minimum 32:9 48:6 64:4 96:3 128:2 (reproduces the roadmap table)
|
||||||
|
-> tier recommended 32:10 48:6 64:5 96:3 128:3 (reproduces the roadmap table)
|
||||||
|
-> 5x64 GiB unified fits, +53.28 GiB headroom
|
||||||
|
-> 3x96 GiB unified fits, +27.68 GiB headroom
|
||||||
|
-> 2x128 / 4x64 (fit probes) fit with only +2.08 GiB headroom across the WHOLE route
|
||||||
|
-> 2x112 GiB (= 224 GiB, the hard-fit floor) DOES NOT FIT: -23.52 GiB
|
||||||
|
-> 3x64 GiB does not fit: -49.12 GiB
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 6. Quality gates (project .venv, deterministic, offline, GPU-free)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
$VP -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
-> 97 passed in 0.12s
|
||||||
|
-> includes coordinated shard/config substitution, malformed telemetry, and
|
||||||
|
contract-ID reseal rejection tests added during controller review
|
||||||
|
|
||||||
|
$VP -m pip wheel --no-deps packages/node -w /tmp/dgr017-wheel
|
||||||
|
$VP -m pip install --no-deps --target /tmp/dgr017-install /tmp/dgr017-wheel/*.whl
|
||||||
|
$VP -I -c "... from meshnet_node.glm_alpha import load_locked_target ..."
|
||||||
|
-> INSTALLED_WHEEL_PASS
|
||||||
|
-> packaged alpha-contract.json, target-manifest.json, and architecture-snapshot.json
|
||||||
|
load and cross-bind successfully outside the source tree
|
||||||
|
|
||||||
|
$VP -m compileall -q packages tests
|
||||||
|
-> exit 0
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
-> exit 0
|
||||||
|
|
||||||
|
$VP -m pytest -q # full deterministic suite
|
||||||
|
-> first final run: 1 failed, 851 passed, 13 skipped; only the tracker cancellation
|
||||||
|
race already documented by DGR-001/DGR-002 failed
|
||||||
|
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
-> 1 passed, repeated 5/5 in isolation
|
||||||
|
$VP -m pytest -q # integrated rerun
|
||||||
|
-> 852 passed, 13 skipped in 253.30s (0:04:13)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 7. Late independent-review repair (2026-07-14)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
-> 99 passed in 0.15s
|
||||||
|
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
|
||||||
|
-> adds nested parsed-state immutability and isolated to_dict() coverage
|
||||||
|
|
||||||
|
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
|
||||||
|
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
|
||||||
|
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
-> 5/5 passed in isolation
|
||||||
|
$VP -m pytest -q # integrated rerun
|
||||||
|
-> 872 passed, 13 skipped in 253.46s (0:04:13)
|
||||||
@@ -0,0 +1,255 @@
|
|||||||
|
{
|
||||||
|
"generated_by": "DGR-017 meshnet_node.glm_alpha.planner",
|
||||||
|
"target": {
|
||||||
|
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||||
|
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||||
|
"quantization": "UD-IQ1_S",
|
||||||
|
"total_bytes": 216715360960,
|
||||||
|
"total_gib": 201.832,
|
||||||
|
"total_gb": 216.715
|
||||||
|
},
|
||||||
|
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
|
||||||
|
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
|
||||||
|
"alpha_contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9",
|
||||||
|
"kv_assumptions": {
|
||||||
|
"dtype": "Q8_0",
|
||||||
|
"bytes_per_value": 1.0625,
|
||||||
|
"context_tokens": 16384,
|
||||||
|
"concurrency": 1,
|
||||||
|
"indexer_layout": "conservative",
|
||||||
|
"mla_values_per_token_per_layer": 576,
|
||||||
|
"backbone_layers": 78,
|
||||||
|
"indexer_full_layers": 21,
|
||||||
|
"note": "Alpha budgets indexer keys across all 78 layers (current experimental DSA layout), not only the 21 Full layers."
|
||||||
|
},
|
||||||
|
"kv_table_gib": {
|
||||||
|
"16384": {
|
||||||
|
"mla_only_q8_gib": 0.73,
|
||||||
|
"optimized_dsa_q8_gib": 0.77,
|
||||||
|
"conservative_dsa_q8_gib": 0.89,
|
||||||
|
"conservative_dsa_f16_gib": 1.68
|
||||||
|
},
|
||||||
|
"131072": {
|
||||||
|
"mla_only_q8_gib": 5.83,
|
||||||
|
"optimized_dsa_q8_gib": 6.18,
|
||||||
|
"conservative_dsa_q8_gib": 7.12,
|
||||||
|
"conservative_dsa_f16_gib": 13.41
|
||||||
|
},
|
||||||
|
"1048576": {
|
||||||
|
"mla_only_q8_gib": 46.62,
|
||||||
|
"optimized_dsa_q8_gib": 49.41,
|
||||||
|
"conservative_dsa_q8_gib": 56.98,
|
||||||
|
"conservative_dsa_f16_gib": 107.25
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"aggregate_hard_fit_floor_gib": 224.0,
|
||||||
|
"aggregate_floor_class": "experimental_hard_fit_floor",
|
||||||
|
"placement_imbalance_factor": 1.1,
|
||||||
|
"tier_table": {
|
||||||
|
"32": {
|
||||||
|
"physical_usable_gib": 32.0,
|
||||||
|
"reserve_gib": 8.0,
|
||||||
|
"placement_budget_gib": 24.0,
|
||||||
|
"weight_gib": 201.832,
|
||||||
|
"kv_gib": 0.89,
|
||||||
|
"total_placement_gib": 202.722,
|
||||||
|
"arithmetic_minimum_nodes": 9,
|
||||||
|
"recommended_nodes": 10,
|
||||||
|
"imbalance_factor": 1.1
|
||||||
|
},
|
||||||
|
"48": {
|
||||||
|
"physical_usable_gib": 48.0,
|
||||||
|
"reserve_gib": 9.6,
|
||||||
|
"placement_budget_gib": 38.4,
|
||||||
|
"weight_gib": 201.832,
|
||||||
|
"kv_gib": 0.89,
|
||||||
|
"total_placement_gib": 202.722,
|
||||||
|
"arithmetic_minimum_nodes": 6,
|
||||||
|
"recommended_nodes": 6,
|
||||||
|
"imbalance_factor": 1.1
|
||||||
|
},
|
||||||
|
"64": {
|
||||||
|
"physical_usable_gib": 64.0,
|
||||||
|
"reserve_gib": 12.8,
|
||||||
|
"placement_budget_gib": 51.2,
|
||||||
|
"weight_gib": 201.832,
|
||||||
|
"kv_gib": 0.89,
|
||||||
|
"total_placement_gib": 202.722,
|
||||||
|
"arithmetic_minimum_nodes": 4,
|
||||||
|
"recommended_nodes": 5,
|
||||||
|
"imbalance_factor": 1.1
|
||||||
|
},
|
||||||
|
"96": {
|
||||||
|
"physical_usable_gib": 96.0,
|
||||||
|
"reserve_gib": 19.2,
|
||||||
|
"placement_budget_gib": 76.8,
|
||||||
|
"weight_gib": 201.832,
|
||||||
|
"kv_gib": 0.89,
|
||||||
|
"total_placement_gib": 202.722,
|
||||||
|
"arithmetic_minimum_nodes": 3,
|
||||||
|
"recommended_nodes": 3,
|
||||||
|
"imbalance_factor": 1.1
|
||||||
|
},
|
||||||
|
"128": {
|
||||||
|
"physical_usable_gib": 128.0,
|
||||||
|
"reserve_gib": 25.6,
|
||||||
|
"placement_budget_gib": 102.4,
|
||||||
|
"weight_gib": 201.832,
|
||||||
|
"kv_gib": 0.89,
|
||||||
|
"total_placement_gib": 202.722,
|
||||||
|
"arithmetic_minimum_nodes": 2,
|
||||||
|
"recommended_nodes": 3,
|
||||||
|
"imbalance_factor": 1.1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"routes": {
|
||||||
|
"recommended_5x64_unified": {
|
||||||
|
"node_count": 5,
|
||||||
|
"aggregate_usable_gib": 320.0,
|
||||||
|
"aggregate_placement_budget_gib": 256.0,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": true,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": 53.278,
|
||||||
|
"reasons": []
|
||||||
|
},
|
||||||
|
"recommended_3x96_unified": {
|
||||||
|
"node_count": 3,
|
||||||
|
"aggregate_usable_gib": 288.0,
|
||||||
|
"aggregate_placement_budget_gib": 230.4,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": true,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": 27.678,
|
||||||
|
"reasons": []
|
||||||
|
},
|
||||||
|
"recommended_3x128_unified": {
|
||||||
|
"node_count": 3,
|
||||||
|
"aggregate_usable_gib": 384.0,
|
||||||
|
"aggregate_placement_budget_gib": 307.2,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": true,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": 104.478,
|
||||||
|
"reasons": []
|
||||||
|
},
|
||||||
|
"fit_probe_2x128_unified": {
|
||||||
|
"node_count": 2,
|
||||||
|
"aggregate_usable_gib": 256.0,
|
||||||
|
"aggregate_placement_budget_gib": 204.8,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": true,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": 2.078,
|
||||||
|
"reasons": []
|
||||||
|
},
|
||||||
|
"fit_probe_4x64_unified": {
|
||||||
|
"node_count": 4,
|
||||||
|
"aggregate_usable_gib": 256.0,
|
||||||
|
"aggregate_placement_budget_gib": 204.8,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": true,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": 2.078,
|
||||||
|
"reasons": []
|
||||||
|
},
|
||||||
|
"hard_fit_floor_2x112_unified": {
|
||||||
|
"node_count": 2,
|
||||||
|
"aggregate_usable_gib": 224.0,
|
||||||
|
"aggregate_placement_budget_gib": 179.2,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": false,
|
||||||
|
"meets_hard_fit_floor": true,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": -23.522,
|
||||||
|
"reasons": [
|
||||||
|
"aggregate placement budget 179.2 GiB is below the 202.7 GiB the target needs after each node's reserve"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"insufficient_3x64_unified": {
|
||||||
|
"node_count": 3,
|
||||||
|
"aggregate_usable_gib": 192.0,
|
||||||
|
"aggregate_placement_budget_gib": 153.6,
|
||||||
|
"required_placement_gib": 202.722,
|
||||||
|
"fits": false,
|
||||||
|
"meets_hard_fit_floor": false,
|
||||||
|
"no_single_node_can_admit_target": true,
|
||||||
|
"headroom_gib": -49.122,
|
||||||
|
"reasons": [
|
||||||
|
"aggregate placement budget 153.6 GiB is below the 202.7 GiB the target needs after each node's reserve",
|
||||||
|
"aggregate usable memory 192.0 GiB is below the 224 GiB experimental hard-fit floor"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"seams": {
|
||||||
|
"3_nodes_2.5gbe": {
|
||||||
|
"node_count": 3,
|
||||||
|
"seam_count": 2,
|
||||||
|
"hidden_size": 6144,
|
||||||
|
"bytes_per_token_per_seam": 12288,
|
||||||
|
"prefill_bytes_per_seam": 201326592,
|
||||||
|
"decode_bytes_per_seam_per_token": 12288,
|
||||||
|
"dsa_sideband_bytes_per_query": 8192,
|
||||||
|
"link_rate_gbps": 2.5,
|
||||||
|
"meets_alpha_minimum": true,
|
||||||
|
"is_recommended_link": false,
|
||||||
|
"decode_serialization_ms_per_token": 0.0786,
|
||||||
|
"decode_latency_ms_per_token": 1.0,
|
||||||
|
"decode_bandwidth_share_ms_per_token": 0.0786,
|
||||||
|
"prefill_serialization_ms": 1288.49
|
||||||
|
},
|
||||||
|
"3_nodes_10.0gbe": {
|
||||||
|
"node_count": 3,
|
||||||
|
"seam_count": 2,
|
||||||
|
"hidden_size": 6144,
|
||||||
|
"bytes_per_token_per_seam": 12288,
|
||||||
|
"prefill_bytes_per_seam": 201326592,
|
||||||
|
"decode_bytes_per_seam_per_token": 12288,
|
||||||
|
"dsa_sideband_bytes_per_query": 8192,
|
||||||
|
"link_rate_gbps": 10.0,
|
||||||
|
"meets_alpha_minimum": true,
|
||||||
|
"is_recommended_link": true,
|
||||||
|
"decode_serialization_ms_per_token": 0.0197,
|
||||||
|
"decode_latency_ms_per_token": 1.0,
|
||||||
|
"decode_bandwidth_share_ms_per_token": 0.0197,
|
||||||
|
"prefill_serialization_ms": 322.123
|
||||||
|
},
|
||||||
|
"5_nodes_2.5gbe": {
|
||||||
|
"node_count": 5,
|
||||||
|
"seam_count": 4,
|
||||||
|
"hidden_size": 6144,
|
||||||
|
"bytes_per_token_per_seam": 12288,
|
||||||
|
"prefill_bytes_per_seam": 201326592,
|
||||||
|
"decode_bytes_per_seam_per_token": 12288,
|
||||||
|
"dsa_sideband_bytes_per_query": 8192,
|
||||||
|
"link_rate_gbps": 2.5,
|
||||||
|
"meets_alpha_minimum": true,
|
||||||
|
"is_recommended_link": false,
|
||||||
|
"decode_serialization_ms_per_token": 0.1573,
|
||||||
|
"decode_latency_ms_per_token": 2.0,
|
||||||
|
"decode_bandwidth_share_ms_per_token": 0.1573,
|
||||||
|
"prefill_serialization_ms": 2576.98
|
||||||
|
},
|
||||||
|
"5_nodes_10.0gbe": {
|
||||||
|
"node_count": 5,
|
||||||
|
"seam_count": 4,
|
||||||
|
"hidden_size": 6144,
|
||||||
|
"bytes_per_token_per_seam": 12288,
|
||||||
|
"prefill_bytes_per_seam": 201326592,
|
||||||
|
"decode_bytes_per_seam_per_token": 12288,
|
||||||
|
"dsa_sideband_bytes_per_query": 8192,
|
||||||
|
"link_rate_gbps": 10.0,
|
||||||
|
"meets_alpha_minimum": true,
|
||||||
|
"is_recommended_link": true,
|
||||||
|
"decode_serialization_ms_per_token": 0.0393,
|
||||||
|
"decode_latency_ms_per_token": 2.0,
|
||||||
|
"decode_bandwidth_share_ms_per_token": 0.0393,
|
||||||
|
"prefill_serialization_ms": 644.245
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
{
|
||||||
|
"observed_at": "2026-07-13",
|
||||||
|
"observed_by": "DGR-017",
|
||||||
|
"method": "GitHub REST API (api.github.com), read-only; no fork, no clone, no patch adopted",
|
||||||
|
"refresh_note": "The roadmap's 2026-07-13 observations were re-verified against live upstream. One item changed: PR #24231 is now MERGED (2026-07-11), which the roadmap already anticipated as 'generic CPU lightning-indexer support is merged'.",
|
||||||
|
"llama_cpp": {
|
||||||
|
"repo": "ggml-org/llama.cpp",
|
||||||
|
"items": [
|
||||||
|
{
|
||||||
|
"ref": "issue #24730",
|
||||||
|
"url": "https://github.com/ggml-org/llama.cpp/issues/24730",
|
||||||
|
"title": "Feature Request: Support for GLM 5.2",
|
||||||
|
"type": "issue",
|
||||||
|
"state": "open",
|
||||||
|
"updated_at": "2026-07-03T22:02:15Z",
|
||||||
|
"meaning": "The umbrella GLM-5.2 support request is still open. GLM-5.2 is not fully supported upstream."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ref": "PR #24770",
|
||||||
|
"url": "https://github.com/ggml-org/llama.cpp/pull/24770",
|
||||||
|
"title": "model : glm-dsa load DSA indexer tensors as optional",
|
||||||
|
"type": "pull_request",
|
||||||
|
"state": "closed",
|
||||||
|
"merged_at": "2026-06-20T10:48:24Z",
|
||||||
|
"meaning": "MERGED. GLM-5.2 loads, but through a dense-MLA compatibility path with DSA indexer tensors treated as optional. This is the fallback the alpha contract explicitly refuses: it can produce text without performing DSA/IndexShare computation."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ref": "PR #24231",
|
||||||
|
"url": "https://github.com/ggml-org/llama.cpp/pull/24231",
|
||||||
|
"title": "New GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer",
|
||||||
|
"type": "pull_request",
|
||||||
|
"state": "closed",
|
||||||
|
"merged_at": "2026-07-11T09:39:07Z",
|
||||||
|
"meaning": "MERGED since the roadmap was written. A generic lightning-indexer op now exists in GGML. Backend coverage beyond CPU remains uneven and must be verified per backend by DGR-018, not assumed."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ref": "PR #25407",
|
||||||
|
"url": "https://github.com/ggml-org/llama.cpp/pull/25407",
|
||||||
|
"title": "GLM 5.2 Indexer support",
|
||||||
|
"type": "pull_request",
|
||||||
|
"state": "open",
|
||||||
|
"draft": false,
|
||||||
|
"mergeable_state": "unstable",
|
||||||
|
"head_sha": "8dedd06415f36f10fc6091241a39b23c1bf0ee11",
|
||||||
|
"base": "master",
|
||||||
|
"commits": 6,
|
||||||
|
"changed_files": 12,
|
||||||
|
"additions": 414,
|
||||||
|
"deletions": 7,
|
||||||
|
"updated_at": "2026-07-13T15:28:51Z",
|
||||||
|
"meaning": "OPEN and actively moving (updated today). This is the real DSA/IndexShare implementation alpha needs. It is narrow — 12 files, +414/-7 — which is the single most important finding for donor policy: the semantics alpha requires are reviewable and trackable upstream, not a 261-patch fork."
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"capability_status_for_alpha": {
|
||||||
|
"gguf_load_of_UD-IQ1_S": "expected via merged #24770, unverified by this project; DGR-018 must prove it against the exact pinned artifact",
|
||||||
|
"moe_routing_and_shared_expert": "expected supported; unverified here",
|
||||||
|
"compressed_mla_kv": "supported via the dense-MLA compatibility path",
|
||||||
|
"dsa_lightning_indexer": "generic GGML op merged (#24231); GLM-5.2 wiring still open (#25407)",
|
||||||
|
"indexshare_full_shared_roles": "NOT upstream; only in open PR #25407",
|
||||||
|
"mtp_nextn": "not required for alpha; NextN tensors must be explicitly loaded or excluded, never silently reinterpreted",
|
||||||
|
"conclusion": "As of 2026-07-13 no released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare. A stock pin today would satisfy 'it emits text' via the dense fallback and would FAIL the alpha semantic-correctness contract. This is the gating technical risk for DGR-004 and DGR-018."
|
||||||
|
},
|
||||||
|
"donor": {
|
||||||
|
"repo": "Mesh-LLM/mesh-llm",
|
||||||
|
"url": "https://github.com/Mesh-LLM/mesh-llm",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"stars_observed": 2048,
|
||||||
|
"pushed_at": "2026-07-13T06:45:51Z",
|
||||||
|
"glm_branch": "feat/jianyang-glm-52",
|
||||||
|
"glm_branch_head": "9bd18f1509dff7fac21578635084035b3ba90a38",
|
||||||
|
"glm_branch_head_date": "2026-07-12T06:37:43Z",
|
||||||
|
"policy": "TEST AND PATCH DONOR ONLY. Do not adopt the fork, its scheduler, discovery, routing, public mesh, or package manager. Meshnet remains the sole control plane (RALPH-CONTEXT runtime decision, ADR-0020).",
|
||||||
|
"focused_candidates": [
|
||||||
|
"GLM DSA graph semantics",
|
||||||
|
"lightning indexer and sparse-attention tests",
|
||||||
|
"IndexShare metadata and Full/Shared role validation",
|
||||||
|
"top-k sideband shape and lifecycle",
|
||||||
|
"stage-local KV filtering",
|
||||||
|
"target parity and performance fixtures"
|
||||||
|
],
|
||||||
|
"adoption_state": "none adopted in DGR-017. This story reads and records upstream state; it takes no patch and forks nothing."
|
||||||
|
},
|
||||||
|
"recommendation_for_dgr_004_and_dgr_018": [
|
||||||
|
"Track upstream PR #25407 rather than forking Mesh-LLM. At 12 files and +414/-7 it is small enough to review, reproduce, and carry as a numbered patch in the project's own pinned stack.",
|
||||||
|
"Any pin chosen before #25407 merges will load GLM-5.2 through the dense-MLA compatibility path. DGR-018 must therefore prove DSA/IndexShare are ACTIVE, not merely that the model emits text — the alpha contract already forbids the fallback.",
|
||||||
|
"Verify lightning-indexer backend coverage (#24231) on the specific backend the route will use. CPU support being merged says nothing about ROCm/HIP."
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# DGR-018 — BLOCKED: no 256-GiB-class oracle host
|
||||||
|
|
||||||
|
Recorded: 2026-07-14 (MAINT-003). Preflight scripts preserved at commit
|
||||||
|
`a0f28b5` ("chore: preserve DGR-018 preflight scripts (postponed)").
|
||||||
|
|
||||||
|
## Blocker
|
||||||
|
|
||||||
|
DGR-018 requires a 256-GiB-class host with at least **224 GiB
|
||||||
|
runtime-accessible memory** (the DGR-017 experimental hard-fit floor for the
|
||||||
|
whole-model `UD-IQ1_S` oracle) and **250 GB free storage** on one filesystem
|
||||||
|
outside `/home` (216.715 GB artifact plus resume/temp headroom). The available
|
||||||
|
development host fails both gates, so the whole-model oracle cannot be
|
||||||
|
established. Per the issue's finish contract, no smaller model may be
|
||||||
|
substituted.
|
||||||
|
|
||||||
|
DGR-019 (needs the DGR-018 oracle for parity certification) and DGR-020
|
||||||
|
(needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single
|
||||||
|
node admits the whole recipe) are blocked transitively.
|
||||||
|
|
||||||
|
## Exact preflight output
|
||||||
|
|
||||||
|
Command (offline; resolves everything from the pinned target manifest and
|
||||||
|
never contacts the network):
|
||||||
|
|
||||||
|
```
|
||||||
|
$ python scripts/glm_whole_model_preflight.py
|
||||||
|
target: UD-IQ1_S 216.715 GB, 6 shards @ abc55e725277
|
||||||
|
[FAIL] storage: need >= 250 GB free on one filesystem outside ['/home']; observed no eligible filesystem
|
||||||
|
[FAIL] memory: need >= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor); observed 124.9 GiB MemTotal
|
||||||
|
destination: NONE — no filesystem outside ['/home'] has 250 GB free
|
||||||
|
- /run/media/popov/DATA (ext4): 74.2 GB free
|
||||||
|
- / (ext4): 51.1 GB free
|
||||||
|
- /run/media/popov/Windows (fuseblk): 26.0 GB free
|
||||||
|
- /run/media/popov/d (fuseblk): 5.1 GB free
|
||||||
|
verdict: fail
|
||||||
|
$ echo $?
|
||||||
|
1
|
||||||
|
```
|
||||||
|
|
||||||
|
Host: Linux 7.0.14-101.fc43.x86_64 x86_64, `MemTotal: 130997376 kB`
|
||||||
|
(124.9 GiB). The full machine-readable report (including the ordered
|
||||||
|
download/verify plan against revision `abc55e72527792c6e77069c99b4cb7de16fa9f23`,
|
||||||
|
manifest SHA-256 `0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`)
|
||||||
|
is in [preflight.json](preflight.json).
|
||||||
|
|
||||||
|
## How to resume
|
||||||
|
|
||||||
|
1. On a qualifying host, run `python scripts/glm_whole_model_preflight.py`
|
||||||
|
(optionally `--dest DIR`); it must exit 0 with `verdict: pass`.
|
||||||
|
2. Download shards in the preflight's ordered plan; verify each with
|
||||||
|
`python scripts/verify_glm_shards.py` before the next transfer starts.
|
||||||
|
3. Proceed with the DGR-018 issue
|
||||||
|
(`.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md`).
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
{
|
||||||
|
"generated_by": "scripts/glm_whole_model_preflight.py",
|
||||||
|
"target": {
|
||||||
|
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||||
|
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||||
|
"quantization": "UD-IQ1_S",
|
||||||
|
"shard_count": 6,
|
||||||
|
"total_bytes": 216715360960,
|
||||||
|
"total_gb": 216.715,
|
||||||
|
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962"
|
||||||
|
},
|
||||||
|
"forbidden_path_prefixes": [
|
||||||
|
"/home"
|
||||||
|
],
|
||||||
|
"mounts": [
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/DATA",
|
||||||
|
"fstype": "ext4",
|
||||||
|
"total_gb": 1208.8,
|
||||||
|
"free_gb": 74.2,
|
||||||
|
"free_bytes": 74201321472,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/",
|
||||||
|
"fstype": "ext4",
|
||||||
|
"total_gb": 217.7,
|
||||||
|
"free_gb": 51.1,
|
||||||
|
"free_bytes": 51073683456,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/Windows",
|
||||||
|
"fstype": "fuseblk",
|
||||||
|
"total_gb": 434.9,
|
||||||
|
"free_gb": 26.0,
|
||||||
|
"free_bytes": 25964466176,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/d",
|
||||||
|
"fstype": "fuseblk",
|
||||||
|
"total_gb": 161.1,
|
||||||
|
"free_gb": 5.1,
|
||||||
|
"free_bytes": 5148332032,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"chosen_destination": null,
|
||||||
|
"checks": [
|
||||||
|
{
|
||||||
|
"check": "storage",
|
||||||
|
"requirement": ">= 250 GB free on one filesystem outside ['/home']",
|
||||||
|
"observed": "no eligible filesystem",
|
||||||
|
"passes": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"check": "memory",
|
||||||
|
"requirement": ">= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor)",
|
||||||
|
"observed": "124.9 GiB MemTotal",
|
||||||
|
"passes": false,
|
||||||
|
"waived": false
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"download_authorized": false,
|
||||||
|
"storage_only": false,
|
||||||
|
"download_plan": [
|
||||||
|
{
|
||||||
|
"step": 1,
|
||||||
|
"shard_index": 1,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||||
|
"size_bytes": 9423744,
|
||||||
|
"size_gb": 0.009,
|
||||||
|
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 2,
|
||||||
|
"shard_index": 6,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||||
|
"size_bytes": 19171063136,
|
||||||
|
"size_gb": 19.171,
|
||||||
|
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 3,
|
||||||
|
"shard_index": 2,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||||
|
"size_bytes": 49208128256,
|
||||||
|
"size_gb": 49.208,
|
||||||
|
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 2"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 4,
|
||||||
|
"shard_index": 3,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||||
|
"size_bytes": 49684417024,
|
||||||
|
"size_gb": 49.684,
|
||||||
|
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 5,
|
||||||
|
"shard_index": 4,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||||
|
"size_bytes": 49396052864,
|
||||||
|
"size_gb": 49.396,
|
||||||
|
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 6,
|
||||||
|
"shard_index": 5,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||||
|
"size_bytes": 49246275936,
|
||||||
|
"size_gb": 49.246,
|
||||||
|
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 5"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"verdict": "fail"
|
||||||
|
}
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# Maintenance review handoff — distributed GGUF runtime
|
||||||
|
|
||||||
|
Date: 2026-07-14
|
||||||
|
Scope: close the maintenance review, preserve the hard blockers, and hand off the remaining implementation work to the next model.
|
||||||
|
|
||||||
|
## What is complete
|
||||||
|
|
||||||
|
- Completed stories are now recorded in `docs/issues/distributed-gguf-runtime/`.
|
||||||
|
- The PRD and milestone docs were updated to reflect the closed set and the blocked set.
|
||||||
|
- The DGR-018 preflight scripts were preserved at commit `a0f28b5`.
|
||||||
|
- The current feature line has delivered DGR-001 through DGR-006 and DGR-017.
|
||||||
|
|
||||||
|
## Hard / unsolved issues for later
|
||||||
|
|
||||||
|
### 1) DGR-018 requires hardware we do not have
|
||||||
|
|
||||||
|
DGR-018 is blocked because the whole-model GLM-5.2 UD-IQ1_S oracle requires:
|
||||||
|
|
||||||
|
- a **256-GiB-class host**,
|
||||||
|
- at least **224 GiB runtime-accessible memory**,
|
||||||
|
- at least **250 GB free storage on one filesystem outside `/home`**.
|
||||||
|
|
||||||
|
The current development host reports only **124.9 GiB MemTotal** and has no eligible filesystem with 250 GB free.
|
||||||
|
The authoritative blocker evidence is in `evidence/DGR-018/BLOCKED.md` and `evidence/DGR-018/preflight.json`.
|
||||||
|
|
||||||
|
### 2) DGR-019 and DGR-020 are transitively blocked
|
||||||
|
|
||||||
|
- **DGR-019** needs the DGR-018 oracle for parity certification.
|
||||||
|
- **DGR-020** needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single node can admit the whole recipe.
|
||||||
|
|
||||||
|
No smaller model may be substituted for these stories.
|
||||||
|
|
||||||
|
### 3) The remainder of the graph stays blocked unless replanned
|
||||||
|
|
||||||
|
The current graph makes **DGR-007 depend on DGR-019**, which means:
|
||||||
|
|
||||||
|
- DGR-007 through DGR-016 are also blocked transitively.
|
||||||
|
- Unblocking the dense pipeline without the 256-GiB host would require an explicit replanning decision to relax the DGR-007 → DGR-019 dependency.
|
||||||
|
- That replanning decision has **not** been made.
|
||||||
|
|
||||||
|
### 4) Maintenance-only tasks should stay separate from feature implementation
|
||||||
|
|
||||||
|
The review uncovered that the codebase now has a clean closed-story split, but further work should avoid mixing:
|
||||||
|
|
||||||
|
- maintenance cleanup,
|
||||||
|
- blocked-hardware preparation,
|
||||||
|
- and actual distributed GLM implementation.
|
||||||
|
|
||||||
|
The next model should treat the maintenance pass as closed and only pick up real implementation work that is not hardware-blocked.
|
||||||
|
|
||||||
|
## Recommended next move
|
||||||
|
|
||||||
|
Use the next model to continue on the **non-blocked implementation queue** only.
|
||||||
|
Priority candidates are whatever is still actionable without the GLM oracle host; if a story depends on DGR-018, keep it deferred.
|
||||||
|
|
||||||
|
## Reference files
|
||||||
|
|
||||||
|
- `docs/issues/distributed-gguf-runtime/README.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/milestones.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/preflight.json`
|
||||||
@@ -7,6 +7,8 @@ Last updated: 2026-07-13
|
|||||||
|
|
||||||
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
|
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
|
||||||
|
|
||||||
|
The alpha-release target is the exact `zai-org/GLM-5.2` model, pinned by revision and served with `reasoning_effort=max`, using the smallest published Unsloth `UD-IQ1_S` GGUF across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama remains a cheap structural fixture; Qwen expansion is post-alpha.
|
||||||
|
|
||||||
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
|
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
|
||||||
|
|
||||||
1. Models larger than one node's RAM/VRAM.
|
1. Models larger than one node's RAM/VRAM.
|
||||||
@@ -36,7 +38,7 @@ GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kerne
|
|||||||
|
|
||||||
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
|
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
|
||||||
|
|
||||||
Before expensive native work, establish controlled lanes:
|
Before expensive native work, establish controlled lanes. DGR-001 remains immutable; DGR-017 adds a target-specific fit and semantics contract without rewriting DGR-001 evidence:
|
||||||
|
|
||||||
- Same model architecture and upstream revision.
|
- Same model architecture and upstream revision.
|
||||||
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
|
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
|
||||||
@@ -134,6 +136,10 @@ Use as source and test donors only:
|
|||||||
|
|
||||||
Do not adopt or fork their repositories wholesale.
|
Do not adopt or fork their repositories wholesale.
|
||||||
|
|
||||||
|
### Mesh-LLM GLM branch: focused test/patch donor only
|
||||||
|
|
||||||
|
Use its GLM-5.2 branch to study DSA, IndexShare, stage-local KV, and sideband tests. Do not import its scheduler, discovery/control plane, package manager, or broad llama.cpp patch stack. Every adopted idea must be independently understood, minimized, attributed, and tested against our exact pin.
|
||||||
|
|
||||||
## Battle-proven transport decision
|
## Battle-proven transport decision
|
||||||
|
|
||||||
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
|
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
|
||||||
@@ -155,7 +161,7 @@ Scope boundary:
|
|||||||
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
|
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
|
||||||
- No QUIC/WebRTC/custom transport in this milestone.
|
- No QUIC/WebRTC/custom transport in this milestone.
|
||||||
|
|
||||||
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`.
|
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`. DGR-006 updates the current single-`NamedTensor` decode fast path to carry the same bundle semantics and adds an explicit typed tail logits/token result with sampling/template identity.
|
||||||
|
|
||||||
Minimum identity:
|
Minimum identity:
|
||||||
|
|
||||||
@@ -219,9 +225,9 @@ Two physical machines execute one model that uses both Shards. Synthetic activat
|
|||||||
|
|
||||||
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
|
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
|
||||||
|
|
||||||
### Gate F: architecture expansion
|
### Gate F: exact GLM-5.2 alpha target
|
||||||
|
|
||||||
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
|
After the generic dense fixture proves range and boundary mechanics, certify explicit GLM-5.2 MoE, MLA KV, DSA, IndexShare, and NextN policy. Alpha requires the exact `UD-IQ1_S` target across physical consumer nodes, native Max-mode semantics, locked parity/usefulness/performance thresholds, and bounded failure cleanup. Qwen3/Qwen3-MoE is later architecture expansion.
|
||||||
|
|
||||||
## Scope discipline
|
## Scope discipline
|
||||||
|
|
||||||
|
|||||||
@@ -1,61 +0,0 @@
|
|||||||
# 04 — Create the reproducible pinned llama.cpp patch stack
|
|
||||||
|
|
||||||
Status: ready-for-agent
|
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
|
||||||
|
|
||||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
|
||||||
- This issue is `DGR-004` in [prd.json](../prd.json).
|
|
||||||
- Read the evidence README for every dependency listed below.
|
|
||||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
|
||||||
|
|
||||||
## Description
|
|
||||||
|
|
||||||
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
|
||||||
|
|
||||||
## Expected durable outputs
|
|
||||||
|
|
||||||
- Exact llama.cpp upstream pin
|
|
||||||
- Numbered minimal patch stack
|
|
||||||
- Reproducible fetch/apply/build smoke
|
|
||||||
- evidence/DGR-004/README.md
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
|
||||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
|
||||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
|
||||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
|
||||||
- [ ] Preserve upstream license and attribution notices.
|
|
||||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
|
||||||
- [ ] Targeted pytest tests pass
|
|
||||||
- [ ] python -m compileall packages tests passes for Python changes
|
|
||||||
- [ ] git diff --check passes
|
|
||||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
|
||||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
|
||||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
|
||||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
|
||||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
|
||||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
|
||||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
|
||||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
|
||||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
|
||||||
|
|
||||||
## Dependency handoff
|
|
||||||
|
|
||||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
|
||||||
|
|
||||||
## Finish contract
|
|
||||||
|
|
||||||
- Create the task evidence directory and durable handoff required above.
|
|
||||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
|
||||||
- Change this issue to `Status: done` only after all criteria pass.
|
|
||||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
|
||||||
- [PRD](../PRD.md)
|
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
|
||||||
- [Current architecture](../architecture.md)
|
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
|
||||||
@@ -42,7 +42,7 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
|
|||||||
|
|
||||||
## Dependency handoff
|
## Dependency handoff
|
||||||
|
|
||||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
- `DGR-006` and `DGR-019` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
|
||||||
|
|
||||||
## Finish contract
|
## Finish contract
|
||||||
|
|
||||||
@@ -57,4 +57,4 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -62,4 +62,4 @@ As a node runtime, I need one supervised native process so that llama.cpp intern
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -58,4 +58,4 @@ As the existing node service, I need a GGUF Shard backend adapter so that the Tr
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -59,4 +59,4 @@ As a release engineer, I need real local distributed parity before involving net
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -59,4 +59,4 @@ As a consumer-hardware operator, I need two physical machines to execute one GGU
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -60,4 +60,4 @@ As a node operator, I need active sessions batched safely so that concurrency in
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ As a client, I need failures to be bounded and explicit so that distributed spee
|
|||||||
|
|
||||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
- DGR-012 continuous batching is post-alpha; this story must harden single-session failure/cancellation first and later extend the same invariants to batched sessions.
|
||||||
|
|
||||||
## Finish contract
|
## Finish contract
|
||||||
|
|
||||||
@@ -59,4 +59,4 @@ As a client, I need failures to be bounded and explicit so that distributed spee
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
@@ -62,4 +62,4 @@ As the product owner, I need an end-to-end comparison so that the native runtime
|
|||||||
- [PRD](../PRD.md)
|
- [PRD](../PRD.md)
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
- [Implementation strategy](../implementation-strategy.md)
|
||||||
- [Current architecture](../architecture.md)
|
- [Current architecture](../architecture.md)
|
||||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user