34 Commits

Author SHA1 Message Date
Dobromir Popov
254627629b Merge commit '47b243cd98fd94da7918cacf5725373b099208e5' into ralph/distributed-gguf-runtime 2026-07-15 23:04:52 +02:00
Dobromir Popov
1fe31ef38d feat: checkpoint distributed gguf runtime stories 2026-07-15 23:42:58 +03:00
Dobromir Popov
47b243cd98 model loading, dash 2026-07-15 13:55:38 +02:00
Dobromir Popov
2852b1f80b loading more 2026-07-15 12:54:51 +02:00
Dobromir Popov
eaf00f6add test: record public relay smoke benchmark 2026-07-15 13:42:22 +03:00
Dobromir Popov
22f28bd69a fix model load/unload 2026-07-15 12:35:32 +02:00
Dobromir Popov
97e2784b37 node registration fixes 2026-07-15 10:34:41 +02:00
Dobromir Popov
c035bad5b7 feat: wire live benchmark CLI endpoints 2026-07-15 10:34:20 +03:00
Dobromir Popov
a508768e8a feat: add live endpoint benchmark runner 2026-07-14 22:46:11 +03:00
Dobromir Popov
e6f6782995 feat: add deterministic CPU/GPU benchmark runner slice 2026-07-14 21:39:13 +03:00
Dobromir Popov
ba7c656364 node metrics 2026-07-14 20:33:02 +02:00
Dobromir Popov
b661590ac7 log window bigger 2026-07-14 17:47:20 +02:00
Dobromir Popov
5b33bf8b99 feat: compare safetensors and gguf on cpu and gpu 2026-07-14 18:45:12 +03:00
Dobromir Popov
c7554ef7d8 feat: add DGR-001 performance contract 2026-07-14 18:13:54 +03:00
Dobromir Popov
21e6c86147 fix: let admin placement recover joined nodes 2026-07-14 16:37:42 +02:00
Dobromir Popov
def47f1a42 Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai 2026-07-14 16:11:26 +02:00
Dobromir Popov
8cb00e951f feat: show admin node pool capacity 2026-07-14 16:11:18 +02:00
Dobromir Popov
7b3399760e chore: wrap up completed story metadata 2026-07-14 17:09:04 +03:00
Dobromir Popov
22467f145c merge: distributed performance baseline benchmark 2026-07-14 17:01:08 +03:00
Dobromir Popov
35af1e21de fix: make model placement controls observable 2026-07-14 16:00:37 +02:00
Dobromir Popov
905ea16ce0 feat: complete route session baseline benchmark 2026-07-14 16:55:52 +03:00
Dobromir Popov
348b003d6e fix: restore responsive dashboard panel grid 2026-07-14 15:55:24 +02:00
Dobromir Popov
1e64a5b2b9 new dash update 2026-07-14 15:29:11 +02:00
Dobromir Popov
e2f3ae32b8 feat: let admins manage model placement 2026-07-14 15:16:23 +02:00
Dobromir Popov
29351d6217 chore: ignore local model cache 2026-07-14 14:05:37 +02:00
Dobromir Popov
5c9a2f6c97 dash style fix 2026-07-14 13:29:51 +02:00
Dobromir Popov
13d82f8032 dash, tests 2026-07-14 12:26:10 +02:00
Dobromir Popov
d1a1400db9 Move tracker hive to admin and expand nodes panel.
Give Nodes & coverage full width on overview with inference prices and live speed, and expose model pricing on /v1/models.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 12:19:25 +02:00
Dobromir Popov
5d87e81bc9 feat: harden node placement and partial model loading 2026-07-13 21:58:08 +02:00
Dobromir Popov
a6bcc69288 sol mainnet payouts tasks 2026-07-13 18:51:40 +02:00
Dobromir Popov
c938d38031 more docs review 2026-07-13 18:37:07 +02:00
Dobromir Popov
95245be512 documentation revision 2026-07-13 18:14:21 +02:00
Dobromir Popov
180a7674e6 configure ralph 2026-07-13 17:56:00 +02:00
Dobromir Popov
f420dc1092 matt's skills updatged with upstream 2026-07-13 14:23:13 +02:00
203 changed files with 12447 additions and 10090 deletions

View File

@@ -1,6 +1,6 @@
---
name: ask-matt
description: Ask which skill or flow fits your situation. A router over the user-invoked skills in this repo.
description: Ask which skill or flow fits your situation. A router over the skills in this repo.
disable-model-invocation: true
---
@@ -8,26 +8,28 @@ disable-model-invocation: true
You don't remember every skill, so ask.
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone.
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone, or a vocabulary layer that runs underneath.
## The main flow: idea → ship
The route most work travels. You have an idea and want it built.
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone.)
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone. Both run the same `/grilling` primitive; `grill-with-docs` is the one that leaves a paper trail.)
2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (see Crossing sessions):
- **`/handoff`** out, then open a fresh session against that file,
- **`/prototype`** to answer the question with throwaway code,
- **`/handoff`** back what you learned, and reference it from the original idea thread.
3. **Branch — is this a multi-session build?**
- **Yes** → **`/to-prd`** (turn the thread into a PRD) → **`/to-issues`** (split the PRD into independently-grabbable issues). Because the issues are independent, **clear context between each one**: start a fresh session per issue and kick off **`/implement`** by passing it the PRD and the single issue to work on.
- **Yes** → **`/to-spec`** (turn the thread into a spec), then **`/to-tickets`** to split it into tracer-bullet tickets, each declaring its **blocking edges**. On a local tracker that's one file per ticket under `.scratch/<feature>/issues/`, worked blockers-first by hand; on a real tracker the edges become native blocking links, so any ticket whose blockers are done can be grabbed — kick off **`/implement`** per ticket, **clearing context between each one**.
- **No** → **`/implement`** right here, in the same context window.
Either way, **`/implement`** builds each issue by driving **`/tdd`** internally — one red-green slice at a time — then closes out by running **`/code-review`**, a two-axis review (Standards + Spec) of the diff, before committing. Reach for **`/tdd`** on its own when you just want to build a concrete behaviour test-first without a full spec, and **`/code-review`** on its own whenever you want to review a branch or PR against a fixed point.
### Context hygiene
Keep steps 13 in **one unbroken context window** — don't compact or clear until after `/to-issues` — so the grilling, PRD, and issues all build on the same thinking. Each `/implement` then starts fresh, working from the issue.
Keep steps 13 in **one unbroken context window** — don't compact or clear until after `/to-tickets` — so the grilling, spec, and tickets all build on the same thinking. Each `/implement` then starts fresh, working from the ticket.
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-issues`, don't push on degraded — `/handoff` and continue in a fresh thread.
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-tickets`, don't push on degraded — `/handoff` and continue in a fresh thread.
## On-ramps
@@ -35,13 +37,26 @@ A starting situation that generates work, then merges onto the main flow.
- **Bugs and requests piling up** → **`/triage`**. It moves issues through triage roles and produces agent-ready issues, which **`/implement`** later picks up.
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Issues that `/to-issues` produced are already agent-ready, so **don't triage them**.
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Tickets that `/to-tickets` produced are already agent-ready, so **don't triage them**.
- **Something's broken** → **`/diagnosing-bugs`**. For the hard ones: the bug that resists a first glance, the intermittent flake, the regression that crept in between two known-good states. It refuses to theorise until it has a **tight feedback loop** — one command that already goes red on *this* bug — then fixes with a regression test. Its post-mortem hands off to **`/improve-codebase-architecture`** when the real finding is that there's no good seam to lock the bug down.
- **A huge, foggy effort — a greenfield project or a huge feature build, too big for one session** → **`/wayfinder`**, the most cognitively demanding flow here. When the way from here to the destination isn't visible yet, it charts a **shared map** of **decision tickets** on the issue tracker and resolves them one at a time — producing **decisions, not deliverables** — until the fog is pushed back and the way is clear. Where **`/grill-with-docs`** sharpens an idea you can hold in one session, wayfinder is for the idea you can't — and it's slower and denser, so save it for exactly that, never a well-scoped feature.
When the map clears, **it hands off, it doesn't build**: merge onto the main flow at **`/to-spec`**, which collapses the map's linked decisions into a buildable plan, then `/to-tickets` and `/implement` as usual. Looping the map straight into `/implement` skips that collapse and throws the linked detail away — go straight to `/implement` only when the effort turned out genuinely small.
## Codebase health
Not feature work — upkeep.
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces deepening opportunities; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`.
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces **deepening opportunities**; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`. It's the survey that finds the candidates; **`/codebase-design`** (below) is the bench you design the chosen one on.
## Vocabulary underneath
Two model-invoked references that run *beneath* the other skills — each the single source of truth for its vocabulary. Reach for them directly when the **words**, not the process, are the problem; or let the skills above pull them in.
- **`/domain-modeling`** — sharpen the project's *domain* language: challenge a fuzzy term, resolve an overloaded word ("account" doing three jobs), record a hard-to-reverse decision as an ADR. It's the active discipline `/grill-with-docs` drives to keep `CONTEXT.md` a clean glossary.
- **`/codebase-design`** — the deep-module vocabulary (module, interface, depth, seam, adapter, leverage, locality) for designing a module's *shape*: a lot of behaviour behind a small interface at a clean seam. `/tdd` and `/improve-codebase-architecture` both speak it.
## Crossing sessions
@@ -53,6 +68,8 @@ Not feature work — upkeep.
Off the main flow entirely.
- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but for when you have **no codebase**. Stateless: it saves nothing locally, builds no `CONTEXT.md`. Reach for it to sharpen any plan or design that doesn't live in a repo.
- **`/prototype`** — a small, throwaway program that answers one design question: does this state model feel right, or what should this UI look like. Throwaway from day one — keep the answer, delete the code. It's the detour in step 2 of the main flow, but reach for it any time a design question is hard to settle on paper.
- **`/research`** — delegate reading legwork to a **background agent**: it investigates a question against **primary sources**, then leaves a cited Markdown file in the repo. Keep working while it reads. The file it produces is something to take *into* the main flow at `/grill-with-docs` — research feeds the thinking, it doesn't replace it.
- **`/teach`** — learn a concept over multiple sessions, using the current directory as a stateful workspace.
- **`/writing-great-skills`** — reference for writing and editing skills well.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Ask Matt"
short_description: "Find the right skill or workflow"
policy:
allow_implicit_invocation: false

View File

@@ -1,10 +1,12 @@
---
name: grilling
description: Interview the user relentlessly about a plan or design. Use when the user wants to stress-test a plan before building, or uses any 'grill' trigger phrases.
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
---
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Interview me relentlessly about every aspect of this until we reach a shared understanding. Walk down each branch of the decision tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering.
If a question can be answered by exploring the codebase, explore the codebase instead.
If a *fact* can be found by exploring the environment (filesystem, tools, etc.), look it up rather than asking me. The *decisions*, though, are mine — put each one to me and wait for my answer.
Do not act on it until I confirm we have reached a shared understanding.

View File

@@ -0,0 +1,3 @@
interface:
display_name: "Grilling"
short_description: "Stress-test thinking one question at a time"

View File

@@ -9,7 +9,7 @@ Write a handoff document summarising the current conversation so a fresh agent c
Include a "suggested skills" section in the document, which suggests skills that the agent should invoke.
Do not duplicate content already captured in other artifacts (PRDs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
Do not duplicate content already captured in other artifacts (specs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
Redact any sensitive information, such as API keys, passwords, or personally identifiable information.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Handoff"
short_description: "Compact a conversation into a handoff"
policy:
allow_implicit_invocation: false

View File

@@ -1,15 +1,15 @@
---
name: implement
description: "Implement a piece of work based on a PRD or set of issues."
description: "Implement a piece of work based on a spec or set of tickets."
disable-model-invocation: true
---
Implement the work described by the user in the PRD or issues.
Implement the work described by the user in the spec or tickets.
Use /tdd where possible, at pre-agreed seams.
Run typechecking regularly, single test files regularly, and the full test suite once at the end.
Once done, use /review to review the work.
Once done, use /code-review to review the work.
Commit your work to the current branch.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Implement"
short_description: "Build work from a spec or tickets"
policy:
allow_implicit_invocation: false

View File

@@ -17,6 +17,11 @@ This command is _informed_ by the project's domain model and built on a shared d
### 1. Explore
**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look:
- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below.
- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net.
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
@@ -56,7 +61,7 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o
### 3. Grilling loop
Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Improve Codebase Architecture"
short_description: "Find and grill architecture improvements"
policy:
allow_implicit_invocation: false

View File

@@ -36,7 +36,7 @@ The right shape depends on the question:
Pick whichever shape best fits the question being asked, *not* whichever is easiest to wire to a TUI. Keep it pure: no I/O, no terminal code, no `console.log` for control flow. The TUI imports it and calls into it; nothing flows the other direction.
This is what makes the prototype useful past its own lifetime. When the question's been answered, the validated reducer / machine / function set can be lifted into the real module — the TUI shell gets deleted.
This is what makes the prototype useful past its own lifetime: when the question's been answered, the validated reducer / machine / function set can be lifted into the real module on its own.
### 4. Build the smallest TUI that exposes the state
@@ -66,9 +66,9 @@ If the host project has no task runner, just put the command at the top of the p
Give the user the run command. They'll drive it themselves; the interesting moments are when they say "wait, that shouldn't be possible" or "huh, I assumed X would be different" — those are the bugs in the _idea_, which is the whole point. If they want new actions added, add them. Prototypes evolve.
### 7. Capture the answer
### 7. Capture the answer and the prototype
When the prototype has done its job, the answer to the question is the only thing worth keeping. If the user is around, ask what it taught them. If not, leave a `NOTES.md` next to the prototype so the answer can be filled in (or filled in by you, if you've watched the session) before the prototype gets deleted.
Once the prototype has answered its question, capture the answer, then capture the prototype the way the [SKILL](SKILL.md) describes. The logic-specific mapping: the validated reducer / machine / function set lifts into the real module (the decision, absorbed); the TUI shell rides along to the throwaway branch that keeps the prototype as a primary source.
## Anti-patterns

View File

@@ -1,7 +1,6 @@
---
name: prototype
description: Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
disable-model-invocation: true
description: Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
---
# Prototype
@@ -22,10 +21,6 @@ The two branches produce very different artifacts — getting this wrong wastes
1. **Throwaway from day one, and clearly marked as such.** Locate the prototype code close to where it will actually be used (next to the module or page it's prototyping for) so context is obvious — but name it so a casual reader can see it's a prototype, not production. For throwaway UI routes, obey whatever routing convention the project already uses; don't invent a new top-level structure.
2. **One command to run.** Whatever the project's existing task runner supports — `pnpm <name>`, `python <path>`, `bun <path>`, etc. The user must be able to start it without thinking.
3. **No persistence by default.** State lives in memory. Persistence is the thing the prototype is _checking_, not something it should depend on. If the question explicitly involves a database, hit a scratch DB or a local file with a clear "PROTOTYPE — wipe me" name.
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast and then delete it.
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast.
5. **Surface the state.** After every action (logic) or on every variant switch (UI), print or render the full relevant state so the user can see what changed.
6. **Delete or absorb when done.** When the prototype has answered its question, either delete it or fold the validated decision into the real code — don't leave it rotting in the repo.
## When done
The _answer_ is the only thing worth keeping from a prototype. Capture it somewhere durable (commit message, ADR, issue, or a `NOTES.md` next to the prototype) along with the question it was answering. If the user is around, that capture is a quick conversation; if not, leave the placeholder so they (or you, on the next pass) can fill in the verdict before deleting the prototype.
6. **Capture it when done.** Fold any validated decision into the real code, then capture the prototype itself as a **primary source**: commit it to a throwaway branch, out of main, and leave a context pointer to that branch on the implementation issue. Capture the answer too — the verdict and the question it settled — in the issue or a commit. The main branch keeps only the validated decision.

View File

@@ -97,12 +97,12 @@ Surface the URL (and the `?variant=` keys). The user will flip through whenever
### 6. Capture the answer and clean up
Once a variant has won, write down which one and why (commit message, ADR, issue, or a `NOTES.md` next to the prototype if running AFK and the user hasn't responded yet). Then:
Once a variant has won, capture the answer — which variant and why — then capture the prototype the way the [SKILL](SKILL.md) describes. Fold the winner into the real code and move the rest onto the throwaway branch, not into main:
- **Sub-shape A** — delete the losing variants and the switcher; fold the winner into the existing page.
- **Sub-shape B** — promote the winning variant to a real route, delete the throwaway route and the switcher.
- **Sub-shape A** — fold the winner into the existing page; drop the losing variants and the switcher from main.
- **Sub-shape B** — promote the winning variant to a real route; drop the throwaway route and the switcher from main.
Don't leave variant components or the switcher lying around. They rot fast and confuse the next reader.
The full set of variants is the primary source, so it lands on the throwaway branch, not the bin — variant components and the switcher left in the main branch rot fast and confuse the next reader.
## Anti-patterns

View File

@@ -0,0 +1,3 @@
interface:
display_name: "Prototype"
short_description: "Prototype to answer a design question"

View File

@@ -26,16 +26,18 @@ Look at the current repo to understand its starting state. Read whatever exists;
- `docs/adr/` and any `src/*/docs/adr/` directories
- `docs/agents/` — does this skill's prior output already exist?
- `.scratch/` — sign that a local-markdown issue tracker convention is already in use
- Is the `triage` skill installed? (a `triage` skill folder alongside this one, or `triage` in your available skills.) This decides whether Section B runs at all.
- Monorepo signals — a `pnpm-workspace.yaml`, a `workspaces` field in `package.json`, or a populated `packages/*` with its own `src/`. Present only in a genuinely large multi-package repo; their absence means single-context, which is almost every repo.
### 2. Present findings and ask
Summarise what's present and what's missing. Then walk the user through the three decisions **one at a time** — present a section, get the user's answer, then move to the next. Don't dump all three at once.
Summarise what's present and what's missing. Then take the sections in order — one section, one answer, then the next.
Assume the user does not know what these terms mean. Each section starts with a short explainer (what it is, why these skills need it, what changes if they pick differently). Then show the choices and the default.
Lead each section with the recommended answer so the user can accept it in a word. Give a one-line explainer only when the choice genuinely branches; skip the section entirely when exploration already settled it (Section B when `triage` isn't installed, Section C when there's no monorepo).
**Section A — Issue tracker.**
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-issues`, `triage`, `to-prd`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-tickets`, `triage`, `to-spec`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
Default posture: these skills were designed for GitHub. If a `git remote` points at GitHub, propose that. If a `git remote` points at GitLab (`gitlab.com` or a self-hosted host), propose GitLab. Otherwise (or if the user prefers), offer:
@@ -44,41 +46,26 @@ Default posture: these skills were designed for GitHub. If a `git remote` points
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
If — and only if — the user picked **GitHub** or **GitLab**, ask one follow-up:
Record the choice in `docs/agents/issue-tracker.md`. The GitHub and GitLab templates carry a "PRs as a request surface" flag, defaulted **off** — leave it off and don't raise it; a user who wants external PRs in the triage queue can flip the flag in the file later.
> Explainer: Open-source repos often receive feature requests as pull requests, not just issues — a PR is an issue with attached code. If you turn this on, `/triage` pulls *external* PRs into the same queue and runs them through the same labels and states as issues (collaborators' in-flight PRs are left alone). Leave it off if PRs aren't a request surface for you.
**Section B — Triage label vocabulary.** Skip this section entirely if the `triage` skill isn't installed (exploration told you) — an uninstalled skill needs no labels.
- **PRs as a request surface** — yes / no (default: no). Record the answer in `docs/agents/issue-tracker.md`. For local-markdown and other trackers, skip this question — there are no PRs.
If it is installed, ask exactly one question:
**Section B — Triage label vocabulary.**
> Do you want to keep the default triage labels? (recommended: **yes**)
> Explainer: When the `triage` skill processes an incoming issue, it moves it through a state machine — needs evaluation, waiting on reporter, ready for an AFK agent to pick up, ready for a human, or won't fix. To do that, it needs to apply labels (or the equivalent in your issue tracker) that match strings *you've actually configured*. If your repo already uses different label names (e.g. `bug:triage` instead of `needs-triage`), map them here so the skill applies the right ones instead of creating duplicates.
The defaults are the five canonical roles, each label string equal to its name: `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`. On **yes**, write them as-is. Only if the user says no — usually because their tracker already uses other names (e.g. `bug:triage` for `needs-triage`) — collect the overrides so `triage` applies existing labels instead of creating duplicates.
The five canonical roles:
**Section C — Domain docs.** Default to **single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. This fits almost every repo; write it without asking.
- `needs-triage` — maintainer needs to evaluate
- `needs-info` — waiting on reporter
- `ready-for-agent` — fully specified, AFK-ready (an agent can pick it up with no human context)
- `ready-for-human` — needs human implementation
- `wontfix` — will not be actioned
Default: each role's string equals its name. Ask the user if they want to override any. If their issue tracker has no existing labels, the defaults are fine.
**Section C — Domain docs.**
> Explainer: Some skills (`improve-codebase-architecture`, `diagnosing-bugs`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
Confirm the layout:
- **Single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. Most repos are this.
- **Multi-context** — `CONTEXT-MAP.md` at the root pointing to per-context `CONTEXT.md` files (typically a monorepo).
Offer **multi-context** — a root `CONTEXT-MAP.md` pointing to per-context `CONTEXT.md` files — only when exploration found monorepo signals. Then confirm which layout they want.
### 3. Confirm and edit
Show the user a draft of:
- The `## Agent skills` block to add to whichever of `CLAUDE.md` / `AGENTS.md` is being edited (see step 4 for selection rules)
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/triage-labels.md`, `docs/agents/domain.md`
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/domain.md`, and `docs/agents/triage-labels.md` (the last only when `triage` is installed)
Let them edit before writing.
@@ -101,7 +88,7 @@ The block:
### Issue tracker
[one-line summary of where issues are tracked, plus whether external PRs are a triage surface]. See `docs/agents/issue-tracker.md`.
[one-line summary of where issues are tracked]. See `docs/agents/issue-tracker.md`.
### Triage labels
@@ -112,12 +99,14 @@ The block:
[one-line summary of layout — "single-context" or "multi-context"]. See `docs/agents/domain.md`.
```
Then write the three docs files using the seed templates in this skill folder as a starting point:
Include the `### Triage labels` sub-block, and write `docs/agents/triage-labels.md`, only when `triage` is installed and Section B ran. When it isn't, both are omitted.
Then write the docs files using the seed templates in this skill folder as a starting point:
- [issue-tracker-github.md](./issue-tracker-github.md) — GitHub issue tracker
- [issue-tracker-gitlab.md](./issue-tracker-gitlab.md) — GitLab issue tracker
- [issue-tracker-local.md](./issue-tracker-local.md) — local-markdown issue tracker
- [triage-labels.md](./triage-labels.md) — label mapping
- [triage-labels.md](./triage-labels.md) — label mapping (only if `triage` is installed)
- [domain.md](./domain.md) — domain doc consumer rules + layout
For "other" issue trackers, write `docs/agents/issue-tracker.md` from scratch using the user's description.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Setup Matt Pocock Skills"
short_description: "Configure a repo for the skills"
policy:
allow_implicit_invocation: false

View File

@@ -32,3 +32,14 @@ Create a GitHub issue.
## When a skill says "fetch the relevant ticket"
Run `gh issue view <number> --comments`.
## Wayfinding operations
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `gh issue create --label wayfinder:map`.
- **Child ticket**: an issue linked to the map as a GitHub sub-issue (`gh api` on the sub-issues endpoint). Where sub-issues aren't enabled, add the child to a task list in the map body and put `Part of #<map>` at the top of the child body. Labels: `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
- **Blocking**: GitHub's **native issue dependencies** — the canonical, UI-visible representation. Add an edge with `gh api --method POST repos/<owner>/<repo>/issues/<child>/dependencies/blocked_by -F issue_id=<blocker-db-id>`, where `<blocker-db-id>` is the blocker's numeric **database id** (`gh api repos/<owner>/<repo>/issues/<n> --jq .id`, _not_ the `#number` or `node_id`). GitHub reports `issue_dependencies_summary.blocked_by` (open blockers only — the live gate). Where dependencies aren't available, fall back to a `Blocked by: #<n>, #<n>` line at the top of the child body. A ticket is unblocked when every blocker is closed.
- **Frontier query**: list the map's open children (`gh issue list --state open`, scoped to the map's sub-issues / task list), drop any with an open blocker (`issue_dependencies_summary.blocked_by > 0`, or an open issue in the `Blocked by` line) or an assignee; first in map order wins.
- **Claim**: `gh issue edit <n> --add-assignee @me` — the session's first write.
- **Resolve**: `gh issue comment <n> --body "<answer>"`, then `gh issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.

View File

@@ -33,3 +33,14 @@ Create a GitLab issue.
## When a skill says "fetch the relevant ticket"
Run `glab issue view <number> --comments`.
## Wayfinding operations
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `glab issue create --label wayfinder:map`. (On GitLab tiers with native epics, an epic may hold the map instead; a labelled issue works everywhere.)
- **Child ticket**: an issue carrying `Part of #<map>` at the top of its description and labels `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
- **Blocking**: GitLab's **native blocking link** — the canonical, UI-visible representation. Add it with the `/blocked_by #<n>` quick action, posted as a note (`glab issue note <child> --message "/blocked_by #<blocker>"`). Native blocking links are a Premium/Ultimate feature; on the free tier (or where unavailable) fall back to a `Blocked by: #<n>, #<n>` line at the top of the description. A ticket is unblocked when every blocker is closed.
- **Frontier query**: `glab issue list -F json` scoped to the map's children, drop any with an open blocker — a native `blocked_by` link to an open issue (`glab api projects/:id/issues/:iid/links`), or an open issue in the `Blocked by` line — or an assignee; first in map order wins.
- **Claim**: `glab issue update <n> --assignee @me` — the session's first write.
- **Resolve**: `glab issue note <n> --message "<answer>"`, then `glab issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.

View File

@@ -1,12 +1,12 @@
# Issue tracker: Local Markdown
Issues and PRDs for this repo live as markdown files in `.scratch/`.
Issues and specs (you may know a spec as a PRD) for this repo live as markdown files in `.scratch/`.
## Conventions
- One feature per directory: `.scratch/<feature-slug>/`
- The PRD is `.scratch/<feature-slug>/PRD.md`
- Implementation issues are `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01`
- The spec is `.scratch/<feature-slug>/spec.md`
- Implementation issues are one file per ticket at `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` — never a single combined tickets file
- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings)
- Comments and conversation history append to the bottom of the file under a `## Comments` heading
@@ -17,3 +17,14 @@ Create a new file under `.scratch/<feature-slug>/` (creating the directory if ne
## When a skill says "fetch the relevant ticket"
Read the file at the referenced path. The user will normally pass the path or the issue number directly.
## Wayfinding operations
Used by `/wayfinder`. The **map** is a file with one **child** file per ticket.
- **Map**: `.scratch/<effort>/map.md` — the Notes / Decisions-so-far / Fog body.
- **Child ticket**: `.scratch/<effort>/issues/NN-<slug>.md`, numbered from `01`, with the question in the body. A `Type:` line records the ticket type (`research`/`prototype`/`grilling`/`task`); a `Status:` line records `claimed`/`resolved`.
- **Blocking**: a `Blocked by: NN, NN` line near the top. A ticket is unblocked when every file it lists is `resolved`.
- **Frontier**: scan `.scratch/<effort>/issues/` for files that are open, unblocked, and unclaimed; first by number wins.
- **Claim**: set `Status: claimed` and save before any work.
- **Resolve**: append the answer under an `## Answer` heading, set `Status: resolved`, then append a context pointer (gist + link) to the map's Decisions-so-far in `map.md`.

View File

@@ -0,0 +1,102 @@
---
name: setup-ts-deep-modules
description: Wire dependency-cruiser into a TypeScript repo so each package is a deep module — implementation hidden in subfolders, reachable only through its entry-point files. User-invoked.
disable-model-invocation: true
---
# Setup TS Deep Modules
Make every package in this repo a **deep module**: a lot of behaviour behind a small interface. A package's public surface is its **entry points** — the files at the package root — and everything in its subfolders is hidden. This skill installs [dependency-cruiser](https://github.com/sverweij/dependency-cruiser) and the rules that make the entry points the only way in, then proves the rules bite.
For the vocabulary (deep module, interface, seam, depth), run the `/codebase-design` skill — use its language throughout.
## The shape this enforces
```
src/packages/
<name>/
index.ts ← an entry point (public). Import this from outside.
client.ts ← another entry point. Packages may expose SEVERAL.
lib/ ← implementation: hidden from outside, free to import each other.
tests/ ← co-located tests + fixtures (a subfolder, so private).
```
The public surface is the package's **root files** — not one designated `index.ts`. By convention implementation lives in `lib/` and tests in `tests/`, giving every package the same two-folder shape. The rule itself is general, though: *anything* in *any* subfolder is private, so you never extend the config to add a folder.
Four rules, all `error`:
1. **Entry-point boundary** — code outside a package (app code or another package) may import only that package's entry points (its root files), never anything in its subfolders.
2. **Intra-package freedom** — a package's own files import each other freely.
3. **Tests through the entry points** — files under `<pkg>/tests/` may import any package's entry points and their own `tests/` fixtures, but never any package's subfolder internals (not even their own). Integration tests across packages are fine; deep imports are not.
4. **No cycles** — no dependency cycles.
**Entry points, not a barrel.** Because the public surface is *every* root file, a package can expose several small entry points (`index.ts`, `client.ts`, `server.ts`) instead of funnelling everything through one giant `index.ts`. Barrel files that re-export a whole subtree are discouraged — keep entry points small and hide implementation in subfolders.
Layering (which packages may depend on which) is a *different* concern and is left as a commented stub in the config for this repo to fill in.
## Steps
### 1. Detect the environment
- **Package manager** — `pnpm-lock.yaml` → pnpm, `yarn.lock` → yarn, `bun.lockb` → bun, else npm. Use it for every command below (`pnpm`/`yarn`/`npm run`/`bunx`).
- **Packages root** — if `src/` exists use `src/packages`, else `packages`. Confirm the choice with the user if the repo already has a different obvious convention.
- **Existing config** — check for a `.dependency-cruiser.*` file. If one exists, do **not** overwrite it: merge the four rules and the options in, and tell the user what you added.
**Done when:** package manager, packages root, and existing-config status are all known.
### 2. Install dependency-cruiser
Install `dependency-cruiser` as a devDependency with the detected package manager.
**Done when:** `dependency-cruiser` is in `devDependencies`.
### 3. Write the config
Copy [`dependency-cruiser.config.cjs`](./dependency-cruiser.config.cjs) to the repo root as `.dependency-cruiser.cjs`. Set `PACKAGES_ROOT` to the root detected in step 1. The rules are path-depth based and extension-agnostic, so nothing else needs adapting.
**Done when:** `.dependency-cruiser.cjs` exists with the correct `PACKAGES_ROOT`, and the four forbidden rules are present.
### 4. Wire it into the checks
- Add a `lint:boundaries` script: `depcruise <packages-root>` (or `depcruise src`).
- Fold it into the repo's umbrella check command — the one that already runs typecheck (e.g. a `check` / `ci` / `validate` script). Do **not** touch `tsconfig` or add path aliases.
- If there is no umbrella script, add `lint:boundaries` and tell the user to include it in CI.
**Done when:** `lint:boundaries` exists and runs as part of the same command as typecheck.
### 5. Scaffold the example package
Create a committed `<packages-root>/example/` as a copy-me template:
- `index.ts` — an entry point. Export one function that delegates to an internal file (so the package is visibly *deep*, not a pass-through).
- `lib/impl.ts` — an internal file in a **subfolder**, imported by `index.ts`, not reachable from outside.
- `tests/example.test.ts` — imports **only** `../index` (an entry point), and asserts against the public function.
Tell the user this is a starter template to copy or delete.
**Done when:** the example package exists, exposes its behaviour through a root entry point, and hides `impl` in a subfolder.
### 6. Prove the rules bite
This is the completion criterion for the whole skill — a config that doesn't fail on a violation is worthless.
1. Run `lint:boundaries`. It must **pass** on the clean example.
2. Temporarily add a deep import to `tests/example.test.ts` (e.g. `import { thing } from "../lib/impl"`). Run `lint:boundaries` again — it must **fail** with `tests-through-entrypoints`.
3. Revert the deep import. Run once more — it must **pass**.
**Done when:** you have observed a pass, then a fail on the deep import, then a pass again. If step 2 does not fail, the rules are not wired correctly — fix before finishing.
### 7. Document the convention
Write a `README.md` **in the packages folder** (`<packages-root>/README.md`) — next to the packages it governs — covering: the `src/packages/<name>/` layout (entry points at the root, `lib/` for implementation, `tests/` for tests), "import only through a package's entry points (its root files)", and how to run `lint:boundaries`. **Discourage barrel files** explicitly — expose several small entry points instead of re-exporting a whole subtree through one index. Keep it to the copy-me snippet plus the four rules in one paragraph each.
Then add a **context pointer** to it from the repo's agent-instructions file — `CLAUDE.md` if present, else `AGENTS.md` (create `AGENTS.md` if neither exists). One line is enough, e.g. `Packages are deep modules — see [src/packages/README.md](./src/packages/README.md) before adding or importing one.` This is what makes an agent discover the boundary rule instead of tripping over it.
**Done when:** `<packages-root>/README.md` exists and discourages barrels, and the repo's `CLAUDE.md`/`AGENTS.md` links to it.
## Notes
- The config's `$1` back-references (dependency-cruiser's group matching) are what let a package reach its own internals while outsiders can't — don't flatten them into separate per-package rules.
- Public vs private is decided by **depth**: a package's root files are entry points; anything in a subfolder is private. The conventional subfolders are `lib/` (implementation) and `tests/`, but the rule doesn't hardcode them — any subfolder is private, so a new folder never needs a config change. Adding an entry point is just adding a root file — no barrel.
- Packages are **flat**: one tier of immediate children under the root. A package's internals may nest as deep as you like; a package may not contain another package.
- Use `.cjs` (not `.js`) so the config's `module.exports` works even in `"type": "module"` repos.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Setup TS Deep Modules"
short_description: "Enforce deep TypeScript modules"
policy:
allow_implicit_invocation: false

View File

@@ -0,0 +1,95 @@
// @ts-check
// Deep-module enforcement for dependency-cruiser.
//
// Each package under the packages root is a DEEP MODULE: a lot of behaviour
// behind a small interface. A package's PUBLIC SURFACE is its ENTRY POINTS —
// the files at the package root. Implementation lives in SUBFOLDERS and is
// private — by convention `lib/` for implementation and `tests/` for tests,
// though any subfolder is private. A package may expose several small entry
// points (index.ts, client.ts, server.ts, …) — prefer that over one giant
// barrel index.
//
// The only thing you should ever need to edit here is PACKAGES_ROOT.
/** Where packages live. One immediate child dir per package (flat, no nesting). */
const PACKAGES_ROOT = "src/packages";
// --- derived patterns (no need to edit) -------------------------------------
const R = PACKAGES_ROOT;
/**
* A package's private internals: anything nested inside a package subfolder.
* The package's root files are its entry points and are NOT matched here —
* they stay importable from outside.
*/
const PACKAGE_INTERNALS = `^${R}/[^/]+/[^/]+/`;
/** @type {import('dependency-cruiser').IConfiguration} */
module.exports = {
forbidden: [
{
name: "entrypoint-boundary-from-app",
comment:
"App/root code may import a package's entry points (its root files), but nothing inside its subfolders.",
severity: "error",
from: { pathNot: `^${R}/` }, // importer is NOT inside any package
to: { path: PACKAGE_INTERNALS },
},
{
name: "entrypoint-boundary-across-packages",
comment:
"A package's own files import each other freely, but may reach OTHER packages only through their entry points — never their internals.",
severity: "error",
// importer is inside a package ($1), but is not a test file
from: { path: `^${R}/([^/]+)/`, pathNot: `^${R}/[^/]+/tests/` },
to: {
path: PACKAGE_INTERNALS,
pathNot: `^${R}/$1/`, // same package → intra-package freedom
},
},
{
name: "tests-through-entrypoints",
comment:
"A package's tests exercise it through its entry points like everyone else: they may import any package's entry points and their own tests/ fixtures, but never any package's internals — not even their own.",
severity: "error",
from: { path: `^${R}/([^/]+)/tests/` }, // a test file, in package $1
to: {
path: PACKAGE_INTERNALS,
pathNot: `^${R}/$1/tests/`, // own tests/ fixtures → allowed
},
},
{
name: "tests-folder-is-private",
comment:
"A package's tests/ folder is reachable only from tests — nothing else may import fixtures.",
severity: "error",
from: { pathNot: `^${R}/[^/]+/tests/` }, // importer is not itself a test
to: { path: `^${R}/[^/]+/tests/` },
},
{
name: "no-circular",
comment: "No dependency cycles. Scope to `^${R}/` if you want to allow cycles outside packages.",
severity: "error",
from: {},
to: { circular: true },
},
// --- Layering (optional, off by default) ----------------------------------
// Interface-hiding controls HOW you import (through the entry points).
// Layering controls WHICH packages may depend on which. Add your own rules
// here, e.g.:
//
// {
// name: "ui-may-not-depend-on-billing",
// severity: "error",
// from: { path: `^${R}/ui/` },
// to: { path: `^${R}/billing/` },
// },
],
options: {
doNotFollow: { path: "node_modules" },
tsConfig: { fileName: "tsconfig.json" },
enhancedResolveOptions: {
extensions: [".ts", ".tsx", ".js", ".jsx", ".json"],
},
},
};

View File

@@ -5,104 +5,32 @@ description: Test-driven development. Use when the user wants to build features
# Test-Driven Development
## Philosophy
TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle — consult them before and during the loop, not after.
**Core principle**: Tests should verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't.
When exploring the codebase, read `CONTEXT.md` (if it exists) so test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
**Good tests** are integration-style: they exercise real code paths through public APIs. They describe _what_ the system does, not _how_ it does it. A good test reads like a specification - "user can checkout with valid cart" tells you exactly what capability exists. These tests survive refactors because they don't care about internal structure.
## What a good test is
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification — "user can checkout with valid cart" tells you exactly what capability exists — and survives refactors because it doesn't care about internal structure.
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
## Anti-Pattern: Horizontal Slices
## Seams — where tests go
**DO NOT write all tests first, then all implementation.** This is "horizontal slicing" - treating RED as "write all tests" and GREEN as "write all code."
A **seam** is the public boundary you test at: the interface where you observe behavior without reaching inside. Tests live at seams, never against internals.
This produces **crap tests**:
**Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. No test is written at an unconfirmed seam. You can't test everything — agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case.
- Tests written in bulk test _imagined_ behavior, not _actual_ behavior
- You end up testing the _shape_ of things (data structures, function signatures) rather than user-facing behavior
- Tests become insensitive to real changes - they pass when behavior breaks, fail when behavior is fine
- You outrun your headlights, committing to test structure before understanding the implementation
Ask: "What's the public interface, and which seams should we test?"
**Correct approach**: Vertical slices via tracer bullets. One test → one implementation → repeat. Each test responds to what you learned from the previous cycle. Because you just wrote the code, you know exactly what behavior matters and how to verify it.
## Anti-patterns
```
WRONG (horizontal):
RED: test1, test2, test3, test4, test5
GREEN: impl1, impl2, impl3, impl4, impl5
- **Implementation-coupled** — mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
- **Tautological** — the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth — a known-good literal, a worked example, the spec.
- **Horizontal slicing** — writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead — one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.
RIGHT (vertical):
RED→GREEN: test1→impl1
RED→GREEN: test2→impl2
RED→GREEN: test3→impl3
...
```
## Rules of the loop
## Workflow
### 1. Planning
When exploring the codebase, read `CONTEXT.md` (if it exists) so that test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
Before writing any code:
- [ ] Confirm with user what interface changes are needed
- [ ] Confirm with user which behaviors to test (prioritize)
- [ ] Identify opportunities for deep modules (small interface, deep implementation) — run the `/codebase-design` skill for the vocabulary and the testability checks
- [ ] List the behaviors to test (not implementation steps)
- [ ] Get user approval on the plan
Ask: "What should the public interface look like? Which behaviors are most important to test?"
**You can't test everything.** Confirm with the user exactly which behaviors matter most. Focus testing effort on critical paths and complex logic, not every possible edge case.
### 2. Tracer Bullet
Write ONE test that confirms ONE thing about the system:
```
RED: Write test for first behavior → test fails
GREEN: Write minimal code to pass → test passes
```
This is your tracer bullet - proves the path works end-to-end.
### 3. Incremental Loop
For each remaining behavior:
```
RED: Write next test → fails
GREEN: Minimal code to pass → passes
```
Rules:
- One test at a time
- Only enough code to pass current test
- Don't anticipate future tests
- Keep tests focused on observable behavior
### 4. Refactor
After all tests pass, look for [refactor candidates](refactoring.md):
- [ ] Extract duplication
- [ ] Deepen modules (move complexity behind simple interfaces)
- [ ] Apply SOLID principles where natural
- [ ] Consider what new code reveals about existing code
- [ ] Run tests after each refactor step
**Never refactor while RED.** Get to GREEN first.
## Checklist Per Cycle
```
[ ] Test describes behavior, not implementation
[ ] Test uses public interface only
[ ] Test would survive internal refactor
[ ] Code is minimal for this test
[ ] No speculative features added
```
- **Red before green.** Write the failing test first, then only enough code to pass it. Don't anticipate future tests or add speculative features.
- **One slice at a time.** One seam, one test, one minimal implementation per cycle.
- **Refactoring is not part of the loop.** It belongs to the review stage (see the `code-review` skill), not the red → green implementation cycle.

View File

@@ -0,0 +1,3 @@
interface:
display_name: "TDD"
short_description: "Test-driven red-green-refactor"

View File

@@ -59,3 +59,19 @@ test("createUser makes user retrievable", async () => {
expect(retrieved.name).toBe("Alice");
});
```
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
```typescript
// BAD: Expected value is recomputed the way the code computes it
test("calculateTotal sums line items", () => {
const items = [{ price: 10 }, { price: 5 }];
const expected = items.reduce((sum, i) => sum + i.price, 0);
expect(calculateTotal(items)).toBe(expected);
});
// GOOD: Expected value is an independent, known literal
test("calculateTotal sums line items", () => {
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
});
```

View File

@@ -0,0 +1,75 @@
---
name: to-spec
description: Turn the current conversation into a spec and publish it to the project issue tracker — no interview, just synthesis of what you've already discussed.
disable-model-invocation: true
---
This skill takes the current conversation context and codebase understanding and produces a spec (you may know this document as a PRD). Do NOT interview the user — just synthesize what you already know.
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
## Process
1. Explore the repo to understand the current state of the codebase, if you haven't already. Use the project's domain glossary vocabulary throughout the spec, and respect any ADRs in the area you're touching.
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can. The fewer seams across the codebase, the better - the ideal number is one.
Check with the user that these seams match their expectations.
3. Write the spec using the template below, then publish it to the project issue tracker. Apply the `ready-for-agent` triage label - no need for additional triage.
<spec-template>
## Problem Statement
The problem that the user is facing, from the user's perspective.
## Solution
The solution to the problem, from the user's perspective.
## User Stories
A LONG, numbered list of user stories. Each user story should be in the format of:
1. As an <actor>, I want a <feature>, so that <benefit>
<user-story-example>
1. As a mobile bank customer, I want to see balance on my accounts, so that I can make better informed decisions about my spending
</user-story-example>
This list of user stories should be extremely extensive and cover all aspects of the feature.
## Implementation Decisions
A list of implementation decisions that were made. This can include:
- The modules that will be built/modified
- The interfaces of those modules that will be modified
- Technical clarifications from the developer
- Architectural decisions
- Schema changes
- API contracts
- Specific interactions
Do NOT include specific file paths or code snippets. They may end up being outdated very quickly.
Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it within the relevant decision and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
## Testing Decisions
A list of testing decisions that were made. Include:
- A description of what makes a good test (only test external behavior, not implementation details)
- Which modules will be tested
- Prior art for the tests (i.e. similar types of tests in the codebase)
## Out of Scope
A description of the things that are out of scope for this spec.
## Further Notes
Any further notes about the feature.
</spec-template>

View File

@@ -0,0 +1,5 @@
interface:
display_name: "To Spec"
short_description: "Turn a conversation into a spec"
policy:
allow_implicit_invocation: false

View File

@@ -0,0 +1,107 @@
---
name: to-tickets
description: Break a plan, spec, or the current conversation into a set of tracer-bullet tickets, each declaring its blocking edges, published to the configured tracker — edges as text in one file per ticket locally, or native blocking links on a real tracker.
disable-model-invocation: true
---
# To Tickets
Break a plan, spec, or conversation into a set of **tickets** — tracer-bullet vertical slices, each declaring the tickets that **block** it.
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
## Process
### 1. Gather context
Work from whatever is already in the conversation context. If the user passes a reference (a spec path, an issue number or URL) as an argument, fetch it and read its full body and comments.
### 2. Explore the codebase (optional)
If you have not already explored the codebase, do so to understand the current state of the code. Ticket titles and descriptions should use the project's domain glossary vocabulary, and respect ADRs in the area you're touching.
Look for opportunities to prefactor the code to make the implementation easier. "Make the change easy, then make the easy change."
### 3. Draft vertical slices
Break the work into **tracer bullet** tickets.
<vertical-slice-rules>
- Each slice cuts a narrow but COMPLETE path through every layer (schema, API, UI, tests) — vertical, NOT a horizontal slice of one layer
- A completed slice is demoable or verifiable on its own
- Each slice is sized to fit in a single fresh context window
- Any prefactoring should be done first
</vertical-slice-rules>
Give each ticket its **blocking edges** — the other tickets that must complete before it can start. A ticket with no blockers can start immediately.
**Wide refactors are the exception to vertical slicing.** A **wide refactor** is one mechanical change — rename a column, retype a shared symbol — whose **blast radius** fans across the whole codebase, so a single edit breaks thousands of call sites at once and no vertical slice can land green. Don't force it into a tracer bullet; sequence it as **expandcontract**. First expand: add the new form beside the old so nothing breaks. Then migrate the call sites over in batches sized by blast radius (per package, per directory), each batch its own ticket blocked by the expand, keeping CI green batch to batch because the old form still exists. Finally contract: delete the old form once no caller remains, in a ticket blocked by every migrate batch. When even the batches can't stay green alone, keep the sequence but let them share an integration branch that all block a final integrate-and-verify ticket — green is promised only there.
### 4. Quiz the user
Present the proposed breakdown as a numbered list. For each ticket, show:
- **Title**: short descriptive name
- **Blocked by**: which other tickets (if any) must complete first
- **What it delivers**: the end-to-end behaviour this ticket makes work
Ask the user:
- Does the granularity feel right? (too coarse / too fine)
- Are the blocking edges correct — does each ticket only depend on tickets that genuinely gate it?
- Should any tickets be merged or split further?
Iterate until the user approves the breakdown.
### 5. Publish the tickets to the configured tracker
Publish the approved tickets. **How** depends on the tracker `/setup-matt-pocock-skills` configured — the tickets are the same either way, only the shape of the blocking edges changes:
- **Local files** → write one file per ticket under `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` in dependency order (blockers first). Each file's "Blocked by" lists the numbers/titles it depends on. Use the per-ticket file template below — one ticket per file, never a single combined file.
- **A real issue tracker (GitHub, Linear, …)** → publish one issue per ticket in dependency order (blockers first) so each ticket's blocking edges can reference real identifiers. Use the platform's native blocking / sub-issue relationship where it has one; otherwise set each ticket's "Blocked by" to the blocking issues. Apply the `ready-for-agent` triage label unless instructed otherwise — the tickets are agent-grabbable by construction.
Work the **frontier**: any ticket whose blockers are all done. For a purely linear chain that means top to bottom.
Do NOT close or modify any parent issue.
<local-ticket-template>
# <NN> — <Ticket title>
**What to build:** the end-to-end behaviour this ticket makes work, from the user's perspective — not a layer-by-layer implementation list.
**Blocked by:** the numbers/titles of the tickets that gate this one, or "None — can start immediately".
**Status:** ready-for-agent
- [ ] Acceptance criterion 1
- [ ] Acceptance criterion 2
</local-ticket-template>
<issue-template>
## Parent
A reference to the parent issue on the tracker (if the source was an existing issue, otherwise omit this section).
## What to build
The end-to-end behaviour this ticket makes work, from the user's perspective — not layer-by-layer implementation.
## Acceptance criteria
- [ ] Criterion 1
- [ ] Criterion 2
## Blocked by
- A reference to each blocking ticket, or "None — can start immediately".
</issue-template>
In either form, avoid specific file paths or code snippets — they go stale fast. Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
Work the frontier one ticket at a time with `/implement`, clearing context between tickets.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "To Tickets"
short_description: "Split a plan into tracer-bullet tickets"
policy:
allow_implicit_invocation: false

View File

@@ -1,9 +1,16 @@
---
name: wayfinder
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of investigation tickets on your issue tracker, and resolve them one at a time until the way to the goal is clear.
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of decision tickets on your issue tracker, and resolve them one at a time until the way to the destination is clear.
disable-model-invocation: true
---
A loose idea has arrived — too big for one agent session, and wrapped in fog: the route from here to a plan isn't visible yet. This skill charts it as a **shared map** on the repo's issue tracker, then works its tickets one at a time. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
A loose idea has arrived — too big for one agent session, and wrapped in fog: the way from here to the **destination** isn't visible yet. Wayfinding is about finding that way, not charging at the destination. This skill charts the way as a **shared map** on the repo's issue tracker, then works its **decision tickets** — questions whose resolution is a decision, not slices of a build to execute — one at a time until the route is clear.
The destination varies per effort, and naming it is the first act of charting — it shapes every ticket. It might be a spec to hand off and iterate on, a decision to lock before planning starts, or a change made in place like a data-structure migration. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
## Plan, don't do
Wayfinder is **planning** by default: each ticket resolves a decision, and the map is done when the way is clear — nothing left to decide before someone goes and does the thing. The pull to just do the work is usually the signal you've reached the edge of the map and it's time to hand off. An effort can override this in its **Notes** — carrying execution into the map itself — but absent that, produce decisions, not deliverables.
## Refer by name
@@ -15,13 +22,17 @@ The map is a single issue on this repo's issue tracker, labelled `wayfinder:map`
The map is an **index**, not a store. It lists the decisions made and points at the tickets that hold their detail; a decision lives in exactly one place — its ticket — so the map never restates it, only gists it and links.
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** Consult `docs/agents/issue-tracker.md` (the "Wayfinding operations" section) for how _this_ repo expresses them. If that doc is absent, default to the local-markdown tracker.
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** The issue tracker should have been provided to you — run `/setup-matt-pocock-skills` if not. Consult the tracker doc's "Wayfinding operations" section for how _this_ repo expresses them. If no tracker has been provided, default to the local-markdown tracker.
### The map body
The whole map at low resolution, loaded once per session. Open tickets are **not** listed — they are open child issues, found by query.
```markdown
## Destination
<what reaching the end of this map looks like — the spec, decision, or change this effort is finding its way to. One or two lines; every session orients to it before choosing a ticket.>
## Notes
<domain; skills every session should consult; standing preferences for this effort>
@@ -32,9 +43,13 @@ The whole map at low resolution, loaded once per session. Open tickets are **not
- [<closed ticket title>](link) — <one-line gist of the answer>
## Fog
## Not yet specified
<!-- see "Fog of war" for what belongs here -->
<!-- see "Fog of war": in-scope fog you can't ticket yet; graduates as the frontier advances -->
## Out of scope
<!-- see "Out of scope": work ruled beyond the destination; closed, never graduates -->
```
### Tickets
@@ -57,36 +72,48 @@ The answer isn't part of the body — it's recorded on resolution (see [Work thr
## Ticket Types
- **Research**: Reading documentation, third-party APIs, or local resources like knowledge bases. Creates a markdown summary as a linked asset. Use when knowledge outside the current working directory is required.
- **Prototype**: Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
- **Grilling**: Conversation with the agent. Uses the /grilling and /domain-modeling skills. Asks one question at a time. The default case.
- **Task**: Literal manual work that must be done before the discussion can move forward — nothing to decide, prototype, or research. Moving data, signing up for a service, provisioning access. The agent automates it where it can; otherwise it hands the human a precise checklist. Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
Every ticket is either **HITL** — human in the loop, worked *with* a human who speaks for themselves — or **AFK**, driven by the agent alone. A HITL ticket only resolves through that live exchange; the agent never stands in for the human's side of it (a grilling agent that answers its own questions has broken this).
- **Research** (AFK): Reading documentation, third-party APIs, or local resources like knowledge bases to surface a fact a decision waits on. Resolved by a `/research` **subagent**. Use when knowledge outside the current working directory is required.
- **Prototype** (HITL): Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
- **Grilling** (HITL): Conversation via the /grilling and /domain-modeling skills, one question at a time. The default case.
- **Task** (HITL or AFK): Manual work that must happen before a *decision* can be made — nothing to decide, prototype, or research, but the discussion is blocked until it's done. Signing up for a service so its API can be judged, provisioning access, moving data so its shape can be seen. This is the one type that *does* rather than decides — and it earns its place by unblocking a decision, not by delivering the destination. The agent drives it alone where it can (AFK); otherwise it hands the human a precise checklist (HITL). Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
## Fog of war
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the tickets lies fog — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the goal is clear and no tickets remain.
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the live tickets lies the **fog of war** — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the destination is clear and no tickets remain.
The map's **Fog** section is where that dim view is written down: the suspected question, the area to revisit later, the risk you're deferring. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
The map's **Not yet specified** section is where that dim view is written down: the suspected question, the area to revisit later. It's the undiscovered frontier _toward_ the destination — everything here is in scope, just not sharp enough to ticket. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
**Fog or ticket?** The test is whether you can state the question precisely now — _not_ whether you can answer it now.
- **Ticket when** the question is already sharp — even if it's blocked and you can't act on it yet.
- **Fog when** you can't yet phrase it that sharply. Don't pre-slice fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
- **Not yet specified when** you can't yet phrase it that sharply. Don't pre-slice the fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
Fog excludes only what's already decided (that's Decisions so far) and what's already a ticket.
**Not yet specified** excludes what's already decided (Decisions so far), what's already a live ticket, and what's out of scope (the next section).
## Out of scope
Fog only ever gathers _toward_ the destination. The destination fixes the scope, so work beyond it is **out of scope** — it isn't fog, and it doesn't belong in **Not yet specified**. It gets its own **Out of scope** section on the map: work you've consciously ruled out of _this_ effort. Scope, not sharpness, lands it here.
Out-of-scope work never graduates — the frontier stops at the destination — so it returns only if the destination is redrawn, and then as a fresh effort, not a resumption.
Ruling something out of scope is a scoping act, not a step on the route. When a ticket that already exists turns out to sit past the destination — mis-scoped in while charting, or exposed by a resolution — **close it** (a closed ticket is unambiguously off the frontier) and leave one line in the **Out of scope** section: the gist plus why it's out of scope, linking the closed ticket. It stays out of **Decisions so far**, which records the route actually walked — a scope boundary isn't a step on it.
## Invocation
Two modes. Either way, **never resolve more than one ticket per session.**
Two modes. Either way, **never resolve more than one ticket per session** — with the exception of research tickets.
### Chart the map
User invokes with a loose idea.
1. Run a `/grilling` and `/domain-modeling` session to surface the open decisions.
2. **Create the map** (label `wayfinder:map`): Notes filled in, Decisions-so-far empty, Fog sketched.
3. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the Fog.
4. Stop — charting the map is one session's work; do not also resolve tickets.
1. **Name the destination.** Run a `/grilling` and `/domain-modeling` session to pin down what this map is finding its way to — the spec, decision, or change. The destination fixes the scope, so it's settled first.
2. **Map the frontier.** Grill again, **breadth-first** this time: fan out across the whole space rather than deep on any one thread, surfacing the open decisions and the first steps takeable now. **If this surfaces no fog** — the way to the destination is already clear, the whole journey small enough for one session — you don't need a map. Stop and ask the user how they'd like to proceed.
3. **Create the map** (label `wayfinder:map`): Destination and Notes filled in, Decisions-so-far empty, the fog sketched into **Not yet specified**.
4. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the fog — the **Not yet specified** section.
5. **Fire the research subagents.** For each `research` ticket you just created, spin up a `/research` subagent to resolve it in parallel, capturing its findings on a throwaway `research/<name>` branch with a context pointer from the ticket.
6. Stop — charting is one session's work; it hand-resolves nothing.
### Work through the map
@@ -96,6 +123,6 @@ User invokes with a map (URL or number). A ticket is **optional** — without on
2. Choose the ticket. If the user named one, use it. Otherwise take the first frontier ticket in order. **Claim it**: assign it to yourself before any work.
3. Resolve it — **zoom as needed**: fetch the full body of any related or closed ticket on demand; invoke the skills the `## Notes` block names. If in doubt, use `/grilling` and `/domain-modeling`.
4. Record the resolution: post the answer as a **resolution comment**, **close** the issue, and **append a context pointer** to the map's Decisions-so-far.
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from the Fog so it lives only as its new ticket. If the decision invalidates other parts of the map, update or delete those tickets.
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from **Not yet specified** so it lives only as its new ticket. If the answer reveals a ticket — this one or another — sits beyond the destination, **rule it out of scope** rather than resolving it on the route. If the decision invalidates other parts of the map, update or delete those tickets.
The user may run unblocked tickets in parallel, so expect other sessions to be editing the tracker concurrently.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Wayfinder"
short_description: "Map a large effort as decision tickets"
policy:
allow_implicit_invocation: false

View File

@@ -158,6 +158,12 @@ _Failure mode._ Ending the current step before it is genuinely done, because the
_Avoid_: premature closure, the rush, rushing, shortcutting
### Negation
_Failure mode._ Steering by prohibition — telling the agent what _not_ to do — which drags the forbidden behaviour into context and makes it _more_ available, not less. _Don't think of an elephant_, and the elephant is all there is; _never write verbose comments_, and verbosity is the pattern the agent has just read. The negation is a weak modifier the strongly-activated concept overruns, so the ban half-reads as an instruction to do the thing. Its **leading word** is the _elephant_: whatever a prohibition names into the frame. Cure: prompt the **positive** — describe the target behaviour ("write one-line comments") so the banned one is never spoken. A prohibition earns its place only as a hard guardrail on a behaviour you cannot phrase positively; even then, pair it with the positive target so attention lands on what to do.
_Avoid_: ironic rebound, don't-prompting, the pink elephant
## Pruning
Keeping a skill lean — each remedy paired with the failure it cures.

View File

@@ -80,3 +80,4 @@ Use these to diagnose issues the user may be having with the skill.
- **Sediment** — stale layers that settle because adding feels safe and removing feels risky. The default fate of any skill without a pruning discipline.
- **Sprawl** — a skill simply too long, even when every line is live and unique. Hurts readability and maintainability and wastes tokens. The cure is the ladder: disclose **reference** behind pointers, and split by **branch** or sequence so each path carries only what it needs.
- **No-op** — a line the model already obeys by default, so you pay load to say nothing. The test: does it change behaviour versus the default? A weak leading word (_be thorough_ when the agent is already thorough-ish) is a no-op; the fix is a stronger word (_relentless_), not a different technique.
- **Negation** — steering by prohibition backfires: _don't think of an elephant_ names the elephant and makes it more available, not less. Prompt the **positive** — state the target behaviour so the banned one is never spoken; keep a prohibition only as a hard guardrail you can't phrase positively, and even then pair it with what to do instead.

View File

@@ -0,0 +1,5 @@
interface:
display_name: "Writing Great Skills"
short_description: "Principles for predictable skills"
policy:
allow_implicit_invocation: false

View File

@@ -2,9 +2,9 @@
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
- [User profile](user-profile.md) — who Dobromir is and how to work with him
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
- [Project status](project-status.md) — US-001…US-035 done; US-036…US-050 in docs/prd.json; alpha hardening + scratch features next
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 00160019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan; [ADR-0023](../../docs/adr/0023-model-agnostic-node-capability-admission.md), [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md))
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).

View File

@@ -20,13 +20,13 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path.
**Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above.
**Two new issues from this session, both `ready-for-agent`:**
- **21 — Honest-noise calibration corpus** (`.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`) rescoped from "prod gate" to a **hard alpha-release blocker**. Confirmed by code read: `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns bool only, no raw divergence value; fleet-dispatch exists but wrong shape (`server.py:2998-3104`, pinned routes + latency, not full-fleet + TOPLOC divergence); storage wrong shape (`registry_events` has no divergence/hardware columns). Three-part build: (1) surface raw TOPLOC distance from audit.py, (2) extend dispatch to hit every registered node with fixed prompt/seed, (3) new SQLite table keyed by node+GPU+dtype. Small-fleet exception granted (N = actual hired-VPS fleet size). Hired VPS hosts stay stake-free until this closes.
- **23 — Dynamic HF-benchmarked pricing** (`.scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md`), high priority but not a release blocker. Pricing today is 100% static (`DEFAULT_PRICE_PER_1K_TOKENS = 0.02`, `billing.py:21`; `model_presets.json` has no per-model price). Target: 80% of cheapest comparable provider on `https://huggingface.co/inference/models` (per-provider-per-model marketplace, `?search=` query param works, no confirmed JSON API — plain scrape attempted first, escalate to headless browser only if the table isn't in raw HTML). Human-verified `hf_aliases` + `hf_verified_match_note` (params/quantization) per model, not auto-discovered matching. Reuses the `_settlement_loop` daemon-thread pattern for a daily refresh; falls back silently to the static default on any failure.
**Two new issues from this session:**
- **21 — Honest-noise calibration corpus** `Status: ready-for-human` (engineering done 2026-07-06; blocked on human fleet calibration run before mainnet launch).
- **23 — Dynamic HF-benchmarked pricing** `Status: done` (see `23-dynamic-hf-pricing_completed.md`).
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (35/35 done) and does NOT see `.scratch/alpha-hardening/issues/`. No ralph loop is running and no `.ralph-tui/` state exists. `.scratch/alpha-hardening/prd.json` now has 23 stories (AH-001…AH-023); point Ralph at that file for the alpha-hardening branch. Do NOT use `ralph auto --parallel` on server.py-touching issues — 21 and 23 both touch `server.py`/`billing.py`/`audit.py`; if run in the same Ralph pass, run them serially, not in parallel (merge-conflict risk, same lesson as 03/04 previously).
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (US-001…US-047; base 35/35 done, friends-test arc 3647 open/in-progress). Alpha hardening uses `.scratch/alpha-hardening/prd.json` (AH-001…AH-023). Point Ralph at the prd.json for the branch you're running.
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].

View File

@@ -6,7 +6,18 @@ metadata:
type: project
---
# Project Status (2026-07-02)
# Project Status (2026-07-13)
## Selected-node model placement (2026-07-14)
- Admin Model placement now opens a node selector for load and release; the control-plane accepts optional `node_id` and targets only that registry assignment. Multi-model serving remains supported through `ADD_SHARD` and `max_loaded_shards`.
- Total node pool resource values are rendered from `/v1/network/map`'s `node.capacity` contract. Route selection remains assignment/capability/throughput/queue based; capacity is used for placement and falls back to tracker defaults only if a node truly omits it.
## Distributed inference performance (2026-07-14)
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
> Doc reconciliation 2026-07-13: `docs/prd.json` tracks US-001…US-050 (048 memory budget, 049 mainnet pilot, 050 Qwen demand placement). ADRs 00250026 added (TAI phase B/C, assignment ownership).
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:

1
.gitignore vendored
View File

@@ -20,6 +20,7 @@ dist/
!.env.testnet
.rocm-local/*
.pytest-tmp/*
.cache/
# Local tracker/node sqlite databases (never commit runtime state)
*.sqlite

View File

@@ -4,7 +4,8 @@
configVersion = "2.1"
tracker = "json"
agent = "opencode"
agent = "codex"
model = "gpt-5.6-terra"
maxIterations = 0
autoCommit = true

View File

@@ -2,9 +2,9 @@
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet SOL/USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers, stake-free). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers; no upfront stake, probation only). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch.
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch. Runbook: [04-toploc-calibration-run](./runbooks/04-toploc-calibration-run.md).
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
@@ -77,6 +77,7 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
| [04 TOPLOC calibration run](./runbooks/04-toploc-calibration-run.md) (issue 21 ops) |
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |

View File

@@ -8,7 +8,7 @@
## 1. Mission / where we are
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 00160019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first.
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer. Pre-release audits found the money/trust path was not alpha-ready; **Bucket 1 alpha blockers are implemented** (see `.scratch/alpha-hardening/README.md`). Remaining launch gates: issue **21** (human calibration run), post-alpha Bucket 2 (1215), and active scratch tracks (NCA, perf, distributed GGUF).
---
@@ -42,7 +42,7 @@ Point to artifacts — do not re-derive from this handoff.
| Path | What it contains |
|---|---|
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 13) |
| `.scratch/alpha-hardening/issues/` | 25 work items (Buckets 13 + perf follow-ups) |
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
| `docs/adr/00160019` | Alpha scope, auth, fraud, multi-tracker design |
| `docs/agents/issue-tracker.md` | Issue file conventions |

View File

@@ -1,6 +1,6 @@
Status: ready-for-human
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money (mainnet SOL/USDT) traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money mainnet USDT traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
@@ -14,9 +14,9 @@ Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware.
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay on probation (no upfront stake) until it's done, then move to paid USDT serving once thresholds derive from their own hardware.
**Current gap (confirmed 2026-07-06 by code read):** none of the three pieces below exist yet.
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (0610) currently runs on a guessed threshold baked into that bool, not a calibrated one.
2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 13-node *routes* and measures latency, not TOPLOC divergence across *every* registered node.
@@ -36,7 +36,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section.
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section; [runbook 04](../runbooks/04-toploc-calibration-run.md).
## ADR links

View File

@@ -440,12 +440,12 @@
"Run relevant pytest tests; run the full suite when practical or document why not"
],
"priority": 21,
"passes": true,
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
"passes": false,
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money mainnet USDT). Operator runbook: .scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md",
"dependsOn": [
"AH-006"
],
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
"completionNotes": "Engineering complete and unit-tested. Remaining: human runs POST /v1/calibration/toploc/run on live hired-VPS fleet, records envelope/FPR, wires thresholds — see runbook 04 and packages/validator/README.md."
},
{
"id": "AH-022",

View File

@@ -0,0 +1,70 @@
# Runbook 04 — Honest-noise TOPLOC calibration (issue 21)
**Status:** engineering complete; **operator action required** before production audit thresholds.
**Blocks:** enabling calibrated TOPLOC thresholds on a mainnet / friends-test fleet (issue 21, ADR-0018).
## When to run
- Before first real-money traffic with audit enforcement enabled.
- Again whenever the fleets **hardware mix** changes materially (new GPU generation, CPU-only nodes added, precision/recipe change per model).
Alpha exception: with a **small hired-VPS-only** fleet, `gate_status.ready` may mean “covers every node we operate today” (`--toploc-calibration-gate-min-hardware-profiles 1`).
## Prerequisites
- Tracker running with billing + registry + `--toploc-calibration-db PATH` (or default under tracker cwd).
- At least one **solo-capable** node per hardware profile you want in the corpus (full model coverage — partial shards are skipped).
- Admin or validator credentials (`Authorization` header or validator service token per ADR-0017).
- Reference validator can replay the fixed calibration prompt (same model/seed as dispatch uses).
## Steps
1. **Register the fleet** — all nodes you intend to pay on mainnet should be up, admitted (NCA when enabled), and solo-serving the calibration model.
2. **Dispatch the job** (admin/validator only):
```bash
curl -X POST "https://<tracker>/v1/calibration/toploc/run" \
-H "Authorization: Bearer <admin-or-validator-token>" \
-H "Content-Type: application/json" \
-d '{}'
```
Partial-shard nodes appear under `skipped_partial_shard_node_ids`. Per-node failures appear under `skipped` with reasons.
3. **Wait for completion** — watch tracker logs and node consoles until every solo-capable node has a row in the corpus.
4. **Fetch results**:
```bash
curl "https://<tracker>/v1/calibration/toploc/results" \
-H "Authorization: Bearer <admin-or-validator-token>"
```
Record:
- `envelope` — p99 metrics + 20% safety margin (recommended tolerances).
- `gate_status.ready` and `gate_status.hardware_profiles`.
- `estimated_false_positive_rate` (in-sample sanity check only).
5. **Write up thresholds** — paste envelope values into operator notes / issue 21 comment. Do **not** wire into production `ToplocAuditConfig` until you have reviewed FPR on this fleet.
6. **Mark issue 21 done** — when corpus covers the launch fleet and thresholds are documented.
## Two-wallet / minimal pilot variant
If your “fleet” is one node machine + one client:
- Run calibration against the **node** profile only (one hardware row is enough for `gate_status` with min profiles = 1).
- Client wallet is irrelevant to calibration — it never serves inference.
## Do not
- Enable stricter production audit thresholds before this completes.
- Reuse a corpus collected on devnet/mock hardware for a different mainnet GPU mix without re-running.
## References
- Issue: `.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`
- Code: `packages/tracker/meshnet_tracker/calibration.py`, `POST /v1/calibration/toploc/run`
- Validator: `packages/validator/README.md` — TOPLOC audit contract

View File

@@ -12,4 +12,10 @@ Provide an opt-in, admin-only tracker Dashboard Testing tab that dynamically dis
- One active run.
- Real inference stays separately environment-gated and excluded from default suites.
## Operator workflow
See [`docs/dev/dashboard-test-runner.md`](../../docs/dev/dashboard-test-runner.md)
for launch configuration, default safe suites vs the gated real-inference suite,
and required environment variables.
See `prd.json` for executable Ralph user stories and acceptance criteria.

View File

@@ -51,15 +51,16 @@
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
"dependsOn": [
"US-001",
"US-002"
]
],
"completionNotes": "Completed by agent"
}
],
"metadata": {
"updatedAt": "2026-07-11T17:02:30.520Z"
"updatedAt": "2026-07-12T01:58:06.286Z"
}
}

View File

@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
1. Read this file completely.
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
6. Read the evidence/handoff directories for every declared dependency.
@@ -296,7 +296,7 @@ Active decisions:
- `.scratch/distributed-gguf-runtime/README.md`
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
- `.scratch/distributed-gguf-runtime/architecture.md`
- `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`
- `docs/adr/0024-distributed-gguf-runtime.md`
- `.scratch/distributed-gguf-runtime/PRD.md`
- `.scratch/distributed-gguf-runtime/prd.json`

View File

@@ -25,7 +25,7 @@ Transformers/safetensors remains the correctness baseline. vLLM remains an optio
- [Current architecture](architecture.md)
- [PRD](PRD.md)
- [Ralph backlog](prd.json)
- [ADR-0020](ADR-0020-distributed-gguf-runtime.md)
- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md)
- [Milestones](milestones.md)
- [Issues](issues/)
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)

View File

@@ -1,6 +1,6 @@
# Distributed GGUF Decision Framework
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0020](ADR-0020-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.

View File

@@ -1,43 +0,0 @@
# DGR-001 downstream stop-condition handoff
Status: **DGR-001 is complete; native-track promotion is blocked by the immutable v1 verdict.**
This is no longer an execution-prerequisite blocker. The required real benchmark
ran successfully, every recipe completed at concurrency 1 and 4, artifacts were
verified, and deterministic/full test gates passed.
## Locked result
`contract-evaluation.json` records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```
The exact-revision BF16 GGUF quality lane compared every prompt but achieved
`0.3333` exact match and `0.9471` mean similarity against the Transformers BF16
reference. V1 requires `0.90` and `0.97`. Quantized Q4_K_M had substantial speed
and fit benefits, but the contract explicitly forbids speed from redeeming a
failed near-lossless quality lane.
## Scope of this stop
The measured baseline is Qwen2.5-0.5B on CPU using a CPU-only llama.cpp build.
It is not a Radeon, large-model, distributed, or native-shard result. Therefore:
1. Do not silently mark v1 promoted or weaken its thresholds after observing the
data.
2. Do not let DGR-004 or later runtime stories treat DGR-001 completion as a
positive promotion signal.
3. A human may choose one of these explicit paths:
- stop the native GGUF track as v1 directs;
- diagnose and fix the BF16 runtime divergence, then rerun the exact v1 plan;
- authorize a separately versioned GPU/large-model contract whose scope and
workload are locked before its measurements.
All raw evidence, configuration, artifacts, hashes, and reproduction commands
are in this directory and `README.md`.

View File

@@ -1,153 +1,127 @@
# DGR-001 — Safetensors versus GGUF performance contract
# DGR-001 — performance contract baseline
Status: **complete; immutable v1 verdict is `stop`.**
## Files changed
DGR-001 successfully produced a controlled local-real CPU baseline. Completion
means the experiment and decision contract are durable and verified; it does
**not** mean the native GGUF track is approved to continue. The locked quality
gate failed, so dependent runtime work requires a human decision or a new,
explicitly versioned experiment/contract rather than silently weakening v1.
- `packages/node/meshnet_node/performance_contract.py`
- `tests/test_performance_contract.py`
- `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
## Controlled workload
## What this slice does
- Model: `Qwen/Qwen2.5-0.5B-Instruct`
- Exact source revision: `7ae557604adf67be50417f59c2c2f167def9a775`
- Machine: `fedora`, Linux `7.0.14-101.fc43.x86_64`, 32 logical CPUs
- Device: CPU for every recipe; VRAM is therefore correctly reported as zero
- Runtime reference: Transformers `5.13.0`, PyTorch
`2.10.0+rocm7.13.0a20260513`, BF16 safetensors
- GGUF runtime: llama.cpp version 9991, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25`
- Workload: three fixed short/medium/long prompts, greedy sampling, 32 output
tokens, three repeats, two warmups, concurrency 1 and 4, 16 CPU threads
- Evidence class: `local-real`
- Locks the DGR-001 benchmark contract in code.
- Pins the architecture-aligned baseline to **DeepSeek-V2-Lite-Chat** (`deepseek2`).
- Uses the same model on both sides of the comparison:
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K**
- Exposes a machine-readable JSON contract with:
- benchmark lanes for `transformers` safetensors and `llama.cpp` GGUF on **CPU** and **GPU**
- concurrency levels `1` and `4`
- the required metrics list
- an explicit stop condition for “no meaningful speed or fit benefit”
- Adds a deterministic stub benchmark report so the contract now has an executable report shape end to end.
All artifacts are beneath `/run/media/popov/DATA/llm/`; no model artifact was
created under `/home`.
## Recent benchmark runner slice
## Recipes and exact artifacts
The runner currently uses a deterministic stub backend to exercise the comparison matrix without downloading a model. It emits:
| Recipe | Artifact | SHA-256 |
|---|---|---|
| Transformers BF16 reference | complete mounted Hugging Face snapshot | `e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6` |
| llama.cpp BF16 quality lane | `Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf` | `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862` |
| llama.cpp Q4_K_M performance/fit lane | `Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf` | `a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5` |
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
- `.scratch/distributed-gguf-runtime/evidence/DGR-001/stub-benchmark-report.json`
The snapshot digest covers every sorted relative path, resolved size, and file
byte, so tokenizer/config drift is included. The BF16 GGUF was converted
directly from the exact snapshot while preserving BF16 weights. Q4_K_M was
quantized from an exact-revision F16 conversion with the pinned quantizer.
Runtime validation recomputes every declared digest before model loading.
The report includes per-device comparisons for:
## Real results
- `transformers-safetensors-cpu` vs `llama-cpp-gguf-cpu`
- `transformers-safetensors-gpu` vs `llama-cpp-gguf-gpu`
All recipes completed every request with zero failures.
and records the memory metric (`rss_bytes` on CPU, `vram_bytes` on GPU), decode speedup, artifact ratio, and output drift.
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 46.1 | 88.0 | 170.1 |
| Aggregate decode tok/s, c=4 | 47.1 | 211.4 | 206.4 |
| TTFT p50, c=1 | 37.5 ms | 43.9 ms | 23.8 ms |
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
| Failures | 0 | 0 | 0 |
## Live endpoint CLI wiring
Against the reference, the eligible Q4_K_M lane measured:
The contract CLI can now drive the live endpoint runner. Passing one `--live-endpoint LANE_ID=URL` mapping per contract lane (plus `--live-benchmark-out`) invokes `run_real_model_endpoint_benchmark` against already-running OpenAI-compatible servers and writes the report using the same schema as the stub:
- single-request decode speedup: **3.69×**;
- concurrency-4 aggregate throughput speedup: **4.38×**;
- resident-memory ratio: **0.279×**;
- artifact-size ratio: **0.398×**.
The near-lossless BF16 quality lane compared all three prompts but measured:
- exact match: **0.3333** (v1 requires at least `0.90`);
- mean text similarity: **0.9471** (v1 requires at least `0.97`).
Tokenization and stopping were controlled: every runtime saw the same prompt
token counts and reported 31 post-TTFT decode tokens. The mismatch is genuine
greedy runtime divergence on two prompts, not missing coverage or a text-length
artifact. Therefore `contract-evaluation.json` records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```bash
PYTHONPATH=packages/node python -m meshnet_node.performance_contract \
--live-endpoint transformers-safetensors-cpu=http://127.0.0.1:8001 \
--live-endpoint llama-cpp-gguf-cpu=http://127.0.0.1:8002 \
--live-endpoint transformers-safetensors-gpu=http://127.0.0.1:8003 \
--live-endpoint llama-cpp-gguf-gpu=http://127.0.0.1:8004 \
--live-benchmark-out .scratch/distributed-gguf-runtime/evidence/DGR-001/live-benchmark-report.json
```
Thresholds were not changed after observing these results.
`--live-model` overrides the model name sent in requests (defaults to the contract's safetensors repo). Without any `--live-endpoint` flags the CLI behaves exactly as before: it writes the contract JSON and, with `--benchmark-out`, the deterministic stub report.
## Implementation
## Exact commands and real results
- `recipe_benchmark.py` provides the runtime-neutral measurement core, true
concurrency, continuous in-flight peak-memory sampling, percentile/throughput
aggregation, failures, and output drift.
- `recipe_drivers.py` provides opt-in Transformers and llama-server drivers,
mounted-drive confinement, exact artifact/runtime verification, equal
device/thread budgets, greedy-only validation, measured host provenance, and
a CPU-only v1 guard until process VRAM can be measured honestly.
- Peak RSS is runtime-scoped: Transformers reports growth above its pre-runtime
Python baseline, while llama.cpp reports its isolated server process tree.
Both are sampled continuously during in-flight requests.
- TTFT uses each runtime's prompt/first-token compute boundary; end-to-end HTTP,
scheduling, and queue overhead remains in latency and `queue_wait_ms`.
- The exact canonical plan SHA-256 locks prompts, model/revision, sampling,
output length, repeats, warmups, and concurrency. The evaluator also requires
equal prompt/decode token counts across recipes.
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
after it; the driver subtracts that token so decode throughput matches the
Transformers inter-token convention.
- `performance_contract.py` rejects wrong plans, synthetic evidence, missing
recipes/concurrency, mixed model revisions, incomplete quality coverage,
failed references, and missing artifact/host provenance.
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
quality gate, and only performance-fit recipes can earn speed/fit benefits.
### Targeted tests
## Evidence files
- `performance-contract.json` — immutable v1 thresholds and stop condition
- `benchmark-config.json` — exact real-run plan, drivers, artifacts, and hashes
- `results.json` — raw machine-readable per-request and aggregate evidence
- `results.txt` — human-readable benchmark summary
- `baseline.json` — distilled measurements for later comparison
- `contract-evaluation.json` — fail-closed v1 verdict
- `commands.txt` — reproducible conversion, benchmark, evaluation, and test commands
- `BLOCKED.md` — downstream stop-condition handoff
- `known-unrelated-failure.md` — clean-base reproduction of the tracker race
## Verification
```text
Targeted: 22 passed
Full suite: 749 passed, 13 skipped
Earlier cancellation retry matrix, DGR-001: 4/5 passed
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
compileall: passed
git diff --check: passed
Evidence JSON parse/integrity checks: passed
```bash
PYTHONPATH=packages/node pytest -q tests/test_performance_contract.py tests/test_route_session_benchmark.py
```
The full-suite exception is documented in `known-unrelated-failure.md` and
satisfies the issue's explicit clean-tree reproduction clause. DGR-001 changes
no tracker/proxy files.
Result: `19 passed in 0.11s`
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
invalid: it used Hermes Agent's internal venv. Verification above used the
project `.venv`, which has the DGR-002-compatible runtime. Real inference used
`.venv-rocm` Python 3.12.
### Contract artifact generation
## Limitations and dependent-story handoff
```bash
PYTHONPATH=packages/node python -m meshnet_node.performance_contract --json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json
```
- This is a **0.5B CPU baseline**, not evidence for a large model, Radeon GPU,
distributed execution, network transport, or native shard worker.
- The installed llama.cpp build is CPU-only (`GGML_HIP=OFF`). No GPU comparison
is claimed.
- Absolute timings are developer-machine measurements; locked ratios and raw
artifacts are provided for reproducibility.
- DGR-014 may consume v1 only with the exact plan/evidence requirements enforced
by `performance_contract.py`.
- DGR-004 and later native-runtime work must not treat DGR-001 completion as a
promotion. V1 says `stop`; proceeding requires a human decision backed by a
separately versioned GPU/large-model contract or a diagnosed quality fix.
Result: wrote `.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json`
### Python compile check
```bash
python -m compileall packages/node/meshnet_node/performance_contract.py tests/test_performance_contract.py
```
Result: passed
## Public relay smoke benchmark (2026-07-15)
A real streamed request was run through the public tracker — **not** by connecting directly to the private node address:
```text
https://meshnet.2.d-popov.com/v1/chat/completions
-> wss://meshnet.2.d-popov.com/ws
-> wss://meshnet.2.d-popov.com/rpc/7j77FsPY1evV8tuf-7000
-> local CUDA node, Qwen/Qwen2.5-0.5B-Instruct layers 0-23
```
The local public-tracker node had an expired proof and a wedged HTTP server. A graceful restart refreshed its CUDA capability proof in `336 ms`, restored `admitted`/`routable` status, and reconnected its relay endpoint.
Measured streaming results after recovery:
| metric | result |
| --- | ---: |
| warm-up TTFT | 420.80 ms |
| warm-up elapsed | 610.23 ms |
| p50 TTFT (3 runs) | 288.26 ms |
| p50 elapsed (3 runs) | 363.20 ms |
| tracker-recorded relay throughput | 58.18-65.25 tok/s |
| HTTP status | 200 for all runs |
The tracker recorded `relay: true` and the local node ID `7j77FsPY-b32476219492` for each completion. Full redacted evidence is in `public-relay-smoke-benchmark.json`.
The other connected node is still alive but **not routable** because its capability proof is stale. It must revalidate before a multi-node shard/relay test can run.
## Limitations
- This slice still uses a deterministic stub backend for the core comparison matrix.
- It now also includes a live endpoint runner, reachable from the CLI via `--live-endpoint`/`--live-benchmark-out`, that fans out one OpenAI-compatible request per lane when the caller provides endpoints; the CLI does not start those servers.
- It does **not** download or run a real model from within the repo.
- Real safetensors vs GGUF execution, TTFT/prefill/decode measurements, RSS/VRAM capture, and output-drift comparison are still to be implemented against the contract.
## Compatibility notes
- The contract stays on the DeepSeek2 family to remain close to the DeepSeek-V4-Flash end goal.
- A smaller non-DeepSeek model can still be used later for loader-plumbing smoke tests, but it does not replace this baseline.
- Model artifacts must stay on the mounted drive and not under `/home`.
## Dependent-story handoff
Next implementation work should attach to this contract and add the live benchmark runner that actually compares:
1. current Transformers/safetensors recipe
2. whole-model llama.cpp GGUF recipe
using the same model architecture/revision and the same prompt/context/concurrency settings.

View File

@@ -1,122 +0,0 @@
{
"evidence_class": "local-real",
"host": {
"accelerator_name": "Radeon 8060S Graphics",
"accelerator_runtime": "7.13.26183",
"benchmark_lane": "cpu-controlled-baseline",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"cpu_count": 32,
"cuda_available": true,
"hostname": "fedora",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
"python": "3.12.13",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"recipes": {
"llama-cpp-near-lossless-quality": {
"artifact_bytes": 994156448,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 73.7861,
"decode_tokens_per_sec": 87.9728,
"failures": 0,
"latency_p50_ms": 385.2049,
"latency_p95_ms": 560.2939,
"peak_rss_bytes": 1110708224,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1427.2072,
"ttft_p50_ms": 43.929,
"ttft_p95_ms": 107.003
},
"4": {
"aggregate_decode_tokens_per_sec": 211.3515,
"decode_tokens_per_sec": 73.8932,
"failures": 0,
"latency_p50_ms": 467.5094,
"latency_p95_ms": 790.862,
"peak_rss_bytes": 1129578496,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1077.8162,
"ttft_p50_ms": 33.612,
"ttft_p95_ms": 128.501
}
},
"device": "cpu",
"lane": "quality"
},
"llama-cpp-quantized-performance-fit": {
"artifact_bytes": 397807520,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 110.0458,
"decode_tokens_per_sec": 170.131,
"failures": 0,
"latency_p50_ms": 258.0681,
"latency_p95_ms": 465.8523,
"peak_rss_bytes": 542167040,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 783.3775,
"ttft_p50_ms": 23.847,
"ttft_p95_ms": 237.696
},
"4": {
"aggregate_decode_tokens_per_sec": 206.377,
"decode_tokens_per_sec": 83.543,
"failures": 0,
"latency_p50_ms": 413.3897,
"latency_p95_ms": 910.253,
"peak_rss_bytes": 572788736,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 474.3116,
"ttft_p50_ms": 67.945,
"ttft_p95_ms": 431.804
}
},
"device": "cpu",
"lane": "performance-fit"
},
"transformers-safetensors-reference": {
"artifact_bytes": 999586347,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 40.3425,
"decode_tokens_per_sec": 46.1451,
"failures": 0,
"latency_p50_ms": 795.4807,
"latency_p95_ms": 930.9725,
"peak_rss_bytes": 1941213184,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 671.8016,
"ttft_p50_ms": 37.4548,
"ttft_p95_ms": 193.4633
},
"4": {
"aggregate_decode_tokens_per_sec": 47.0903,
"decode_tokens_per_sec": 13.1337,
"failures": 0,
"latency_p50_ms": 2631.0031,
"latency_p95_ms": 3073.7389,
"peak_rss_bytes": 2177265664,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 247.5617,
"ttft_p50_ms": 94.3995,
"ttft_p95_ms": 444.6749
}
},
"device": "cpu",
"lane": "quality"
}
},
"reference_recipe_id": "transformers-safetensors-reference"
}

View File

@@ -1,118 +0,0 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "cpu-controlled-baseline",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0"
},
"plan": {
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [1, 4],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-safetensors-reference",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cpu",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-near-lossless-quality",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
},
{
"id": "llama-cpp-quantized-performance-fit",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
}
]
}

View File

@@ -1,51 +0,0 @@
# Exact source snapshot (already present on mounted storage)
SOURCE=/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775
LLAMA=/run/media/popov/d/DEV/llamacpp/llama.cpp
ROCM_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm/bin/python
PROJECT_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
OUT=/run/media/popov/DATA/llm/dgr-001
# Converter support check (no writes)
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" --dry-run
# Exact-revision near-lossless and performance-fit artifacts
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf"
$LLAMA/build/bin/llama-quantize "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf" Q4_K_M
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype bf16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf"
# Runtime and artifact identity
git -C "$LLAMA" rev-parse HEAD
$LLAMA/build/bin/llama-server --version
sha256sum "$LLAMA/build/bin/llama-server" "$LLAMA/convert_hf_to_gguf.py" "$LLAMA/build/bin/llama-quantize"
sha256sum "$SOURCE/model.safetensors" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf"
# Deterministic complete-snapshot digest used by benchmark-config.json
PYTHONPATH=packages/node $ROCM_PY - <<'PY'
from pathlib import Path
from meshnet_node.recipe_drivers import _artifact_sha256
print(_artifact_sha256(Path('/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775')))
PY
# Canonical opt-in local-real benchmark
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt
# Distil the baseline and evaluate immutable v1
PYTHONPATH=packages/node $PROJECT_PY - <<'PY'
from pathlib import Path
import json
from meshnet_node.performance_contract import baseline_from_report, evaluate_contract, load_contract
root = Path('.scratch/distributed-gguf-runtime/evidence/DGR-001')
report = json.loads((root / 'results.json').read_text())
contract = load_contract(root / 'performance-contract.json')
(root / 'baseline.json').write_text(json.dumps(baseline_from_report(report), indent=2, sort_keys=True) + '\n')
(root / 'contract-evaluation.json').write_text(json.dumps(evaluate_contract(contract, report).to_dict(), indent=2, sort_keys=True) + '\n')
PY
# Deterministic verification
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q tests/test_recipe_benchmark.py
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q
PYTHONPATH=packages/node $PROJECT_PY -m compileall -q packages tests
git diff --check

View File

@@ -1,71 +0,0 @@
{
"contract_version": 1,
"fit_benefit": true,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"quality_lane_pass": false,
"rationale": [
"the near-lossless quality lane failed: the GGUF runtime disagrees with the safetensors reference beyond what near-lossless weights can explain",
"a meaningful speed benefit was measured",
"a meaningful fit benefit was measured"
],
"recipes": [
{
"comparable": true,
"failures": 0,
"fit_benefit": false,
"incomparable_reason": "",
"lane": "quality",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.4882,
"artifact_size_ratio": 0.9946,
"artifact_size_win": false,
"compared_prompts": 3,
"decode_speedup": 1.9064,
"exact_match_rate": 0.3333,
"expected_prompts": 3,
"failure_rate": 0.0,
"mean_similarity": 0.9471,
"resident_memory_ratio": 0.5722,
"ttft_ratio": 1.1729
},
"quality_pass": false,
"reasons": [
"single-request decode 1.91x reference (>= 1.25x) at TTFT ratio 1.17",
"aggregate throughput at concurrency 4 is 4.49x reference (>= 1.25x)",
"peak resident memory is 0.57x reference (<= 0.75x)",
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
],
"recipe_id": "llama-cpp-near-lossless-quality",
"speed_benefit": false
},
{
"comparable": true,
"failures": 0,
"fit_benefit": true,
"incomparable_reason": "",
"lane": "performance-fit",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.3826,
"artifact_size_ratio": 0.398,
"artifact_size_win": true,
"decode_speedup": 3.6869,
"failure_rate": 0.0,
"resident_memory_ratio": 0.2793,
"ttft_ratio": 0.6367
},
"quality_pass": null,
"reasons": [
"single-request decode 3.69x reference (>= 1.25x) at TTFT ratio 0.64",
"aggregate throughput at concurrency 4 is 4.38x reference (>= 1.25x)",
"peak resident memory is 0.28x reference (<= 0.75x)"
],
"recipe_id": "llama-cpp-quantized-performance-fit",
"speed_benefit": true
}
],
"speed_benefit": true,
"stop_condition_met": true,
"verdict": "stop"
}

View File

@@ -1,32 +0,0 @@
# Observed pre-existing intermittent tracker race
This file records an earlier unrelated timing observation; it is **not** the
final DGR-001 verification result.
Test:
```text
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
```
One earlier full-suite run produced:
```text
1 failed, 745 passed, 13 skipped
```
A five-run isolated retry matrix reproduced the same rate on both branches:
```text
DGR-001 branch: 4/5 passed, 1/5 failed
clean d904c40: 4/5 passed, 1/5 failed
```
The final full-suite run on the exact hardened DGR-001 state completed green:
```text
749 passed, 13 skipped in 251.42s
```
The earlier race was therefore timing-sensitive, pre-existing, and unrelated
to the DGR-001 benchmark/contract files.

View File

@@ -1,44 +1,75 @@
{
"benchmark_lanes": [
{
"concurrency_levels": [
1,
4
],
"device": "cpu",
"id": "transformers-safetensors-cpu",
"recipe": "current safetensors recipe",
"runtime": "transformers"
},
{
"concurrency_levels": [
1,
4
],
"device": "cpu",
"id": "llama-cpp-gguf-cpu",
"recipe": "whole-model GGUF recipe",
"runtime": "llama.cpp"
},
{
"concurrency_levels": [
1,
4
],
"device": "gpu",
"id": "transformers-safetensors-gpu",
"recipe": "current safetensors recipe",
"runtime": "transformers"
},
{
"concurrency_levels": [
1,
4
],
"device": "gpu",
"id": "llama-cpp-gguf-gpu",
"recipe": "whole-model GGUF recipe",
"runtime": "llama.cpp"
}
],
"metrics": [
"ttft_ms",
"prefill_tok_per_sec",
"decode_tok_per_sec",
"p50_latency_ms",
"p95_latency_ms",
"aggregate_throughput_tok_per_sec",
"rss_bytes",
"vram_bytes",
"artifact_bytes",
"failure_count",
"output_drift"
],
"model_target": {
"architecture": "deepseek2",
"comparison_policy": "same model/revision, closest practical low-footprint precision pair: BF16 safetensors versus Q2_K GGUF",
"gguf_quant": "Q2_K",
"gguf_repo": "second-state/DeepSeek-V2-Lite-Chat-GGUF",
"gguf_size_gb": 6.43,
"name": "DeepSeek-V2-Lite-Chat",
"rationale": "Smallest DeepSeek-family benchmark anchor that still points toward DeepSeek-V4-Flash; keeps the runtime on the DeepSeek2 path instead of falling back to a tiny but architecture-mismatched smoke model.",
"safetensors_precision": "bfloat16",
"safetensors_repo": "deepseek-ai/DeepSeek-V2-Lite-Chat"
},
"notes": [
"Real model execution stays opt-in and must keep model artifacts on the mounted drive.",
"Use the tiny fallback only for loader plumbing smoke tests; it does not replace the architecture-aligned baseline."
],
"schema_version": 1,
"contract_version": 1,
"locked_at": "2026-07-13T00:00:00Z",
"locked_by": "DGR-001",
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"thresholds": {
"min_decode_speedup": 1.25,
"max_ttft_ratio": 1.25,
"min_aggregate_throughput_speedup": 1.25,
"max_resident_memory_ratio": 0.75,
"max_artifact_size_ratio": 0.6,
"min_quality_exact_match_rate": 0.9,
"min_quality_mean_similarity": 0.97,
"max_failure_rate": 0.0
},
"baseline": {
"status": "pending-real-evidence",
"required_evidence_class": "local-real",
"required_recipes": [
"transformers-safetensors-reference",
"llama-cpp-near-lossless-quality",
"llama-cpp-quantized-performance-fit"
],
"required_concurrency_levels": [
1,
4
],
"required_controlled_variables": [
"model architecture",
"model revision",
"machine and device",
"formatted prompts and context lengths",
"output length and greedy sampling policy"
],
"required_plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"minimum_prompt_count": 3,
"minimum_repeats": 3,
"minimum_output_tokens": 32,
"required_device": "cpu"
},
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract."
"stop_condition": "Stop if GGUF does not provide a meaningful speed or fit benefit over the safetensors baseline for the chosen DeepSeek-family model target.",
"story_id": "DGR-001"
}

View File

@@ -0,0 +1,83 @@
{
"schema_version": 1,
"executed_at_utc": "2026-07-15T10:41:14Z",
"test_kind": "public-relay-single-node-streaming-smoke-benchmark",
"target": {
"public_chat_endpoint": "https://meshnet.2.d-popov.com/v1/chat/completions",
"relay_url": "wss://meshnet.2.d-popov.com/ws",
"model": "qwen2.5-0.5b-instruct",
"quantization": "bfloat16"
},
"recovery": {
"problem": "The local node's capability proof had expired and its port-7000 HTTP server had wedged with CLOSE-WAIT sockets.",
"action": "Gracefully restarted the local public-tracker meshnet-node process on port 7000.",
"startup_validation": {
"device": "cuda",
"capability_proof_ms": 336,
"node_id": "7j77FsPY-b32476219492",
"relay_addr": "wss://meshnet.2.d-popov.com/rpc/7j77FsPY1evV8tuf-7000"
}
},
"tracker_admission_after_recovery": {
"node_id": "7j77FsPY-b32476219492",
"alive": true,
"status": "ready",
"capability_state": "admitted",
"routable": true,
"route_hops": 1
},
"client_measurements": {
"warmup": {
"http_status": 200,
"ttft_ms": 420.8,
"elapsed_ms": 610.23,
"response_text": "MeshNet Relay Benchmark Passed"
},
"runs": [
{
"run": 1,
"ttft_ms": 376.04,
"elapsed_ms": 458.65,
"response_text": "relay benchmark pass"
},
{
"run": 2,
"ttft_ms": 258.33,
"elapsed_ms": 336.71,
"response_text": "relay benchmark pass"
},
{
"run": 3,
"ttft_ms": 288.26,
"elapsed_ms": 363.2,
"response_text": "relay benchmark pass"
}
],
"p50_ttft_ms": 288.26,
"p50_elapsed_ms": 363.2
},
"tracker_relay_evidence": [
{
"status": 200,
"relay": true,
"node_id": "7j77FsPY-b32476219492",
"tokens": 11,
"elapsed_seconds": 0.1686,
"tokens_per_sec": 65.2541
},
{
"status": 200,
"relay": true,
"node_id": "7j77FsPY-b32476219492",
"tokens": 11,
"elapsed_seconds": 0.1891,
"tokens_per_sec": 58.1799
}
],
"scope_and_remaining_work": {
"validated": "Public HTTPS chat endpoint routed a streaming request through the tracker relay to the local CUDA node and completed with HTTP 200.",
"not_validated": "Two-node shard routing was not run because the remote node 5gMLrmyB-88f5cba044d0 still had an expired capability proof and was not routable.",
"next_gate": "Refresh the remote node capability proof, then load a multi-node-compatible assignment and repeat the benchmark through the public tracker relay."
},
"reproduction": "Use a valid bearer API key with the public /v1/chat/completions endpoint and stream a short qwen2.5-0.5b-instruct request. Do not connect directly to private node HTTP endpoints; the tracker relay is the required path."
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,10 +0,0 @@
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 37.5/ 193.5 ms; prefill 671.8 tok/s; decode 46.1 tok/s; aggregate 40.3 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 94.4/ 444.7 ms; prefill 247.6 tok/s; decode 13.1 tok/s; aggregate 47.1 tok/s; rss 2.18 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 43.9/ 107.0 ms; prefill 1427.2 tok/s; decode 88.0 tok/s; aggregate 73.8 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 33.6/ 128.5 ms; prefill 1077.8 tok/s; decode 73.9 tok/s; aggregate 211.4 tok/s; rss 1.13 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 23.8/ 237.7 ms; prefill 783.4 tok/s; decode 170.1 tok/s; aggregate 110.0 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 67.9/ 431.8 ms; prefill 474.3 tok/s; decode 83.5 tok/s; aggregate 206.4 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)

View File

@@ -0,0 +1,247 @@
{
"comparisons": {
"cpu": {
"artifact_bytes_ratio": 0.2048,
"decode_speedup": 2.3333,
"gguf_benefit": true,
"gguf_lane": "llama-cpp-gguf-cpu",
"memory_bytes_ratio": 0.2152,
"memory_metric": "rss_bytes",
"output_drift": 0.0,
"safetensors_lane": "transformers-safetensors-cpu",
"ttft_speedup": 1.8947
},
"gpu": {
"artifact_bytes_ratio": 0.2048,
"decode_speedup": 1.5294,
"gguf_benefit": true,
"gguf_lane": "llama-cpp-gguf-gpu",
"memory_bytes_ratio": 0.2273,
"memory_metric": "vram_bytes",
"output_drift": 0.0,
"safetensors_lane": "transformers-safetensors-gpu",
"ttft_speedup": 1.6154
}
},
"lanes": [
{
"concurrency_levels": [
1,
4
],
"device": "cpu",
"id": "transformers-safetensors-cpu",
"output_tokens": [
"mesh",
"activation",
"seam",
"baseline"
],
"recipe": "current safetensors recipe",
"results": [
{
"concurrency": 1,
"metrics": {
"aggregate_throughput_tok_per_sec": 6.0,
"artifact_bytes": 33715493273,
"decode_tok_per_sec": 6.0,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 166.6667,
"p95_latency_ms": 208.3334,
"prefill_tok_per_sec": 45.0,
"rss_bytes": 35433480192,
"ttft_ms": 1800.0,
"vram_bytes": 0
}
},
{
"concurrency": 4,
"metrics": {
"aggregate_throughput_tok_per_sec": 20.4,
"artifact_bytes": 33715493273,
"decode_tok_per_sec": 5.1,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 196.0784,
"p95_latency_ms": 245.098,
"prefill_tok_per_sec": 38.25,
"rss_bytes": 35433480192,
"ttft_ms": 2340.0,
"vram_bytes": 0
}
}
],
"runtime": "transformers"
},
{
"concurrency_levels": [
1,
4
],
"device": "cpu",
"id": "llama-cpp-gguf-cpu",
"output_tokens": [
"mesh",
"activation",
"seam",
"baseline"
],
"recipe": "whole-model GGUF recipe",
"results": [
{
"concurrency": 1,
"metrics": {
"aggregate_throughput_tok_per_sec": 14.0,
"artifact_bytes": 6904159928,
"decode_tok_per_sec": 14.0,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 71.4286,
"p95_latency_ms": 89.2858,
"prefill_tok_per_sec": 90.0,
"rss_bytes": 7623566950,
"ttft_ms": 950.0,
"vram_bytes": 0
}
},
{
"concurrency": 4,
"metrics": {
"aggregate_throughput_tok_per_sec": 47.6,
"artifact_bytes": 6904159928,
"decode_tok_per_sec": 11.9,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 84.0336,
"p95_latency_ms": 105.042,
"prefill_tok_per_sec": 76.5,
"rss_bytes": 7623566950,
"ttft_ms": 1235.0,
"vram_bytes": 0
}
}
],
"runtime": "llama.cpp"
},
{
"concurrency_levels": [
1,
4
],
"device": "gpu",
"id": "transformers-safetensors-gpu",
"output_tokens": [
"mesh",
"activation",
"seam",
"baseline"
],
"recipe": "current safetensors recipe",
"results": [
{
"concurrency": 1,
"metrics": {
"aggregate_throughput_tok_per_sec": 34.0,
"artifact_bytes": 33715493273,
"decode_tok_per_sec": 34.0,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 29.4118,
"p95_latency_ms": 36.7647,
"prefill_tok_per_sec": 850.0,
"rss_bytes": 4294967296,
"ttft_ms": 420.0,
"vram_bytes": 35433480192
}
},
{
"concurrency": 4,
"metrics": {
"aggregate_throughput_tok_per_sec": 115.6,
"artifact_bytes": 33715493273,
"decode_tok_per_sec": 28.9,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 34.6021,
"p95_latency_ms": 43.2526,
"prefill_tok_per_sec": 722.5,
"rss_bytes": 4294967296,
"ttft_ms": 546.0,
"vram_bytes": 35433480192
}
}
],
"runtime": "transformers"
},
{
"concurrency_levels": [
1,
4
],
"device": "gpu",
"id": "llama-cpp-gguf-gpu",
"output_tokens": [
"mesh",
"activation",
"seam",
"baseline"
],
"recipe": "whole-model GGUF recipe",
"results": [
{
"concurrency": 1,
"metrics": {
"aggregate_throughput_tok_per_sec": 52.0,
"artifact_bytes": 6904159928,
"decode_tok_per_sec": 52.0,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 19.2308,
"p95_latency_ms": 24.0385,
"prefill_tok_per_sec": 640.0,
"rss_bytes": 1610612736,
"ttft_ms": 260.0,
"vram_bytes": 8053063680
}
},
{
"concurrency": 4,
"metrics": {
"aggregate_throughput_tok_per_sec": 176.8,
"artifact_bytes": 6904159928,
"decode_tok_per_sec": 44.2,
"failure_count": 0,
"output_drift": 0.0,
"p50_latency_ms": 22.6244,
"p95_latency_ms": 28.2805,
"prefill_tok_per_sec": 544.0,
"rss_bytes": 1610612736,
"ttft_ms": 338.0,
"vram_bytes": 8053063680
}
}
],
"runtime": "llama.cpp"
}
],
"model_target": {
"architecture": "deepseek2",
"comparison_policy": "same model/revision, closest practical low-footprint precision pair: BF16 safetensors versus Q2_K GGUF",
"gguf_quant": "Q2_K",
"gguf_repo": "second-state/DeepSeek-V2-Lite-Chat-GGUF",
"gguf_size_gb": 6.43,
"name": "DeepSeek-V2-Lite-Chat",
"rationale": "Smallest DeepSeek-family benchmark anchor that still points toward DeepSeek-V4-Flash; keeps the runtime on the DeepSeek2 path instead of falling back to a tiny but architecture-mismatched smoke model.",
"safetensors_precision": "bfloat16",
"safetensors_repo": "deepseek-ai/DeepSeek-V2-Lite-Chat"
},
"schema_version": 1,
"source": "stub-backend",
"stop_condition": {
"gguf_benefit": true,
"text": "Stop if GGUF does not provide a meaningful speed or fit benefit over the safetensors baseline for the chosen DeepSeek-family model target.",
"triggered": false
},
"story_id": "DGR-001"
}

View File

@@ -1,242 +1,176 @@
# DGR-002 — Adopt the versioned gRPC Shard protocol
# DGR-002 — Versioned gRPC Shard protocol: evidence
Status: **done**. Every acceptance criterion is met with real command output.
Evidence class: **synthetic/unit** — this story defines a schema and proves both
languages agree on it. No model, GPU, network peer or benchmark is involved, and
none is claimed.
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-unit** (schema round-trip + cross-language protobuf
compatibility). No model download, no GPU, no network, no API credits.
## 1. Summary
## Summary
`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for
the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python
and C++ both generate from it, and a shared committed conformance vector proves
they encode it identically — byte for byte.
Added the versioned Protocol Buffers schema that is the semantic contract between
Python and C++ Shards (ADR-0024), plus reproducible Python and C++ code
generation/build wiring and generated-schema round-trip + compatibility tests in
**both** languages. The schema defines one long-lived bidirectional gRPC stream
per Route Session Activation Seam, bounded prefill chunking, a small decode fast
path, and a versioned named-tensor bundle carrying every required identifier.
Design decisions worth carrying forward:
No existing runtime code was modified — this story is purely additive (a new
`.proto`, a `native_protocol` loader package, C++ build wiring, and one new test
module). Generated stubs are produced on demand into gitignored `build/`
directories, so nothing generated is committed.
- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation,
identity and flow control are carried as fields, not left to HTTP/2 metadata,
because the existing relay carries these frames as **opaque binary**. A relayed
frame has no HTTP/2 context to inherit a deadline or a channel identity from.
If it is not in the schema, it does not survive the relay.
- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the
stream; `Cancel` is also a unary RPC. A cancel that can only travel down a
stream that flow control has wedged is not a cancel.
- **Checksums cover the uncompressed payload.** Compression is a per-hop
transport decision (reusing the existing `activation_compression` policies), so
a checksum over the compressed frame would be invalidated by a hop that merely
chose differently.
- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2
bounds *bytes in flight*; it does not bound how much *work* a worker has queued,
and a relayed frame gets no window at all. Credits bound queue occupancy and KV
pressure, and negotiation takes the strictest bound of either peer so a sender
cannot talk a worker into unbounded queues.
## Files changed (all new)
## 2. Files changed
- `packages/node/native/proto/shard_runtime.proto` — the schema (package
`meshnet.shard.v1`, proto3). Service `ShardRuntime` with `GetCapability`,
`Health`, `ActivateSession` (bidi stream), `Release`, `Cancel`.
- `packages/node/meshnet_node/native_protocol/__init__.py` — reproducible
on-demand `grpc_tools.protoc` codegen + loader (`load()`, `load_grpc()`) and
shared bundle helpers (`compute_checksum`, `verify_checksum`, `fragment_tensor`,
`reassemble_tensor`).
- `packages/node/native/scripts/generate_python.py` — standalone reproducible
Python generation (self-contained; does not import `meshnet_node`).
- `packages/node/native/scripts/generate_cpp.sh` — reproducible C++ generation
(message stubs always; gRPC service stubs when `grpc_cpp_plugin` is present).
- `packages/node/native/CMakeLists.txt` — C++ build wiring; works with both
CONFIG-mode (`protobuf::libprotobuf`/`protobuf::protoc`) and CMake's
`FindProtobuf` module.
- `packages/node/native/tests/roundtrip_test.cpp` — C++ round-trip / compat test
(`--selftest`, `--read`, `--write`).
- `tests/test_native_shard_protocol.py` — Python round-trip + compatibility tests
and the Python↔C++ cross-language driver.
New:
## Acceptance criteria → evidence
| Path | What |
|---|---|
| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `9e211660…`, see `protocol.json`) |
| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest |
| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors |
| `packages/node/native/README.md` | How to regenerate and build |
| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface |
| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation |
| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages |
| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) |
| `scripts/generate_native_protocol.py` | Python generation, with `--check` |
| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` |
| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source |
| `tests/test_native_shard_protocol.py` | 45 Python tests |
- **Capability/health/session-stream/release/cancellation schema** — the
`ShardRuntime` service's five RPCs; `test_capability_and_health_round_trip`,
`test_session_stream_carries_open_prefill_decode_release_cancel`.
- **One long-lived bidi stream per Activation Seam with deadlines, cancellation,
flow control, structured errors** — `rpc ActivateSession (stream ...) returns
(stream ...)`. Deadlines: gRPC call deadline on direct transport, plus
`SessionOpen.deadline_unix_nanos` for relay-carried frames. Cancellation:
`Cancel` RPC and in-stream `CancelRequest`/`PHASE_CANCEL`. Flow control:
`FlowControl` frames (credits + in-flight byte/message caps). Structured errors:
`Status` (canonical code, message, `RetryClass`, details). Verified by
`test_session_response_carries_structured_status_and_results`.
- **Bounded prefill chunking + small decode fast path** — `PrefillChunk`
(`chunk_index`/`chunk_count`/`final_chunk`, `SessionOpen.max_prefill_tokens_per_chunk`)
and `DecodeStep` (minimal single-bundle path). Bounded fragments via
`SessionOpen.max_fragment_bytes` and `fragment_tensor(...)`.
- **Carries schema version, work ID, Route Session ID, route epoch,
artifact/recipe fingerprint, shard range/effective start, phase, position,
idempotency step, cache expectation, compression, checksum** — all on
`MessageHeader` (+ `ArtifactFingerprint.runtime_recipe_fingerprint`,
`ShardRange.effective_start_layer`). Verified field-by-field by
`test_message_header_carries_every_required_field`.
- **Versioned named-tensor bundle (name, shape, dtype, byte order, fragments)** —
`TensorBundle`/`NamedTensor`/`TensorFragment`;
`test_named_tensor_bundle_describes_shape_dtype_byteorder_and_fragments`,
`test_fragment_and_reassemble_round_trip_with_checksums`.
- **Round-trip + compatibility tests in Python and C++** — Python:
`tests/test_native_shard_protocol.py` (11 tests). C++: `roundtrip_test.cpp`
built via CMake; cross-language driver `test_cross_language_roundtrip_python_and_cpp`
exercises Python→C++ and C++→Python in both directions.
- **Targeted pytest** — `11 passed, 1 skipped` (default env); `12 passed` with the
C++ toolchain on PATH.
- **compileall packages tests** — exit 0.
- **git diff --check** — clean.
- **Deterministic / download-free / credit-free / GPU-free** — all tests are pure
protobuf serialization; the C++ path uses only local compilers.
- **Full deterministic pytest** — `704 passed, 14 skipped, 11 failed`. The 11
failures are pre-existing and unrelated (see below).
Modified:
## Commands and real results
- `packages/node/pyproject.toml` — added runtime floors `grpcio>=1.82.1` and
`protobuf>=7.35.0`, matching the committed generated-code requirements; new
`proto` extra pinning `grpcio-tools==1.82.1`.
- `packages/node/meshnet_node/activation_compression.py` — optional bounded zstd
output for untrusted protocol frames; existing callers remain compatible.
- `packages/node/meshnet_node/native_protocol/__init__.py` — exports negotiated
bound constants and whole-session-message validation.
See `commands.txt` for the exact command list. Key results:
The canonical PRD marks only DGR-002 passed. `git status` before this story was clean.
- `python packages/node/native/scripts/generate_python.py`
`shard_runtime_pb2.py: ok`, `shard_runtime_pb2_grpc.py: ok`.
- `pytest tests/test_native_shard_protocol.py -q`**11 passed, 1 skipped**
(skip reason: `C++ toolchain unavailable: cmake not found on PATH`).
- With `/tmp/pbsrc/install/bin` (protoc 33.1) and `.venv/bin` (cmake) on PATH and
`CMAKE_PREFIX_PATH=/tmp/pbsrc/install`:
- `generate_cpp.sh``shard_runtime.pb.cc`, `shard_runtime.pb.h`
(grpc service stubs skipped: `grpc_cpp_plugin` absent).
- `cmake -S ... -B ...` + `cmake --build ...` → build OK.
- `shard_protocol_roundtrip_test --selftest``selftest ok (128 bytes)`, exit 0.
- `ctest``1/1 Test #1: shard_protocol_roundtrip ... Passed`.
- `pytest ...::test_cross_language_roundtrip_python_and_cpp -q`**1 passed**
(Python serializes → C++ parses & verifies → C++ serializes → Python parses
& verifies).
- `compileall -q packages tests` → exit 0.
- `git diff --check` → clean.
## 3. Commands and real results
## Pre-existing unrelated failures (full-suite)
See `commands.txt` for the exact ordered list. Results:
`pytest -q` on the full tree reports 11 failures, all in tracker routing /
dynamic routing / manual route benchmark / toploc calibration — none import the
Shard protocol. Clean-tree reproduction: with **all DGR-002 files moved aside**
(`git status` shows only the pre-existing `.ralph-tui/config.toml` deletion),
re-running exactly these tests gives `11 failed, 3 passed` — identical failures.
They exist on the `ralph/distributed-gguf-runtime` branch independent of this
story. The full list is in `results.json.preexisting_unrelated_failures`.
```
python scripts/generate_native_protocol.py --check -> generated stubs are up to date
python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date
Note: the earlier `progress.md` (RCR-001, on master) recorded a different set of
6 optional-dependency failures (zstandard, langchain_openai). Those did **not**
recur here; this environment has those deps. The 11 above are branch-local
routing/benchmark failures, not environmental.
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
-- gRPC C++ not found: building message types only (sufficient for the conformance test)
cmake --build build/native -j -> Built target shard_protocol_conformance
ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed
100% tests passed out of 1
## Limitations and deferred work
cmp build/native/cpp_roundtrip.binpb \
packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0)
- **C++ toolchain is host-provided, not vendored.** The default test env has no
`protoc`/`cmake`/protobuf C++ headers on PATH, so the C++ cross-language test
**skips** by default (explicit skip reason). It was executed for this evidence
using an ephemeral from-source protobuf 33.1 install at `/tmp/pbsrc/install`
plus the `.venv` cmake. DGR-004/DGR-008 should pin the C++ protobuf/gRPC
toolchain (upstream commit + reproducible fetch/build) so this test runs in CI
without relying on an ad-hoc `/tmp` install.
- **gRPC C++ service stubs not built here.** `grpc_cpp_plugin` is absent, so
`generate_cpp.sh` produced message stubs only. The round-trip test needs only
message serialization; the service stubs are DGR-008's concern.
- **No live gRPC transport yet.** This story delivers the schema + serialization
contract and generation/build wiring only. Channel setup, the bidi stream
server/client, deadlines/cancellation propagation over a real HTTP/2 channel,
and relay framing are DGR-008/DGR-009.
- **Protobuf runtime version skew.** Python runtime is pip protobuf 7.35.1; the
C++ side used protoc 33.1. Protobuf wire format is stable across these, and the
cross-language round-trip confirms interop; version pinning is deferred to the
toolchain-pinning stories.
pytest -q tests/test_native_shard_protocol.py -> 45 passed
pytest -q tests/test_native_shard_protocol.py \
tests/test_activation_compression.py -> 51 passed
pytest -q (final full suite) -> 728 passed, 12 skipped
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
(after an earlier flaky full-suite failure) -> 1 passed, 1 passed, 1 passed
clean minimum-runtime import + codec smoke test -> passed
grpcio==1.82.1, protobuf==7.35.0
compileall -q packages tests -> OK (exit 0)
git diff --check -> clean (exit 0)
```
## Compatibility / migration notes
The C++ lane was rebuilt from scratch by Ralph (`rm -rf build/native`) using only
the documented commands, and reproduced the same result. During controller
review the user explicitly chose not to repeat the destructive build-directory
cleanup, so the independent controller relied on the recorded CMake/CTest run
while reproducing every Python/generation/full-suite gate.
- proto3 with a 0-valued `*_UNSPECIFIED` member on every enum and never-reused
field numbers. Forward compatibility (unknown-field preservation) is verified
behaviourally by `test_unknown_fields_are_preserved_for_forward_compatibility`
— note protobuf 7.x's upb backend does not implement the `UnknownFields()`
introspection accessor, so the test asserts the observable re-serialization
outcome instead. Backward defaults verified by
`test_defaults_are_stable_for_backward_compatibility`.
- Wire schema version is `SchemaVersion.SCHEMA_VERSION_1` (int 1), also exposed as
`meshnet_node.native_protocol.SCHEMA_VERSION`.
### Controller review corrections
## Handoff for dependent stories
Independent controller review found and fixed two classes of issue before
integration:
1. Generated stubs required gRPC 1.82.1 and Protobuf 7.35.0, while the initial
package metadata allowed much older runtimes that could fail at import time.
2. Flow-control bounds were described but not enforced by the reference decoder.
Tensor declarations, shape rank/dimensions, fragment/tensor counts, fragments,
wire bodies, whole bundles, complete session messages (including envelope
overhead), and zstd window/output expansion are now fail-closed against the
negotiated/default bounds. Unspecified bundle versions, compression and
checksums are rejected rather than interpreted as valid data.
3. Negotiated initial credits could exceed `max_inflight_chunks`; credits are now
capped by the settled in-flight limit.
Controller results: protocol tests `45 passed`; protocol plus shared compression
tests `51 passed`; final full suite `728 passed, 12 skipped`. A clean environment
at the declared minimum gRPC/Protobuf runtime versions imported both generated
stub modules and round-tripped the codec. Generation checks, `compileall`, static
secret scan, and `git diff --check` all passed.
### Full-suite note — a pre-existing flaky test
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy`
is **flaky on a clean tree, independent of this story**. Reproduction, run
*before any DGR-002 file existed* (working tree clean, `git status` empty):
```
pytest -q -> 1 failed, 682 passed, 12 skipped
FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# same test, three consecutive isolated runs on the same clean tree:
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 1 passed in 1.76s
-> 1 failed in 4.39s
-> 1 passed in 1.10s
```
It is a timing race in proxy cancellation (a 3-second in-flight generation raced
against the cancel assertion), not a deterministic failure, and it touches no code
this story changes. One controller full-suite run reported exactly that one failure
(`1 failed, 719 passed, 12 skipped`); three immediate isolated retries all passed
in 1.11 seconds, and the final exact-code full suite was green (`728 passed,
12 skipped`). It is flagged for whoever owns the tracker cancel path and is **not**
fixed here, since silently touching another story's code is out of scope.
## 4. Acceptance criteria
| Criterion | Where it is proven |
|---|---|
| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` |
| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` |
| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` |
| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test |
| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` |
| Round-trip + compatibility tests in Python and C++ | 45 Python tests; C++ `ctest` 1/1; cross-language byte equality |
| Targeted pytest passes | 45 passed |
| `compileall packages tests` | exit 0 |
| `git diff --check` | exit 0 |
| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU |
| Full deterministic pytest passes, or pre-existing failure recorded | Final exact-code run: 728 passed, 12 skipped; earlier sole flaky failure documented with clean-tree reproduction and 3/3 passing retries |
## 5. How the cross-language claim is actually earned
Two codecs that each round-trip their own output prove only that each is
self-consistent. Instead:
1. Python builds the canonical `SessionRequest` and commits its bytes.
2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C
**from the polynomial in independent C++ code**, reassembles the multi-fragment
tensor, and re-serializes to `cpp_roundtrip.binpb`.
3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden.
Compatibility is tested in both languages: an unknown field from a newer peer
survives a parse/serialize hop (a Shard forwards activations — silently stripping
fields would corrupt a route it is merely a waypoint on), and a sparse message
from an older peer parses to proto3 defaults.
## 6. Limitations and deferred work
- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message
types), not a running gRPC C++ server, because this machine has no gRPC C++ stack
and building it is a large dependency the conformance test does not need.
`CMakeLists.txt` already generates and exports `shard_runtime_grpc` when
`find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend
`scripts/bootstrap_native_toolchain.sh`.**
- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no
deadline actually fires, no credit is actually consumed. This story defines and
proves the contract; DGR-008/DGR-009 implement it.
- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral).
`scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such
as `build/native-toolchain`.
- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when
present. The fallback is byte-exact but slow; a worker on the hot path should install
the native package. Not a correctness limitation.
- Compression on the wire is zstd-or-none only, matching the existing seam.
## 7. Compatibility and migration notes
- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays
at `2` and the legacy `/forward` path is untouched. The native protocol is a
*separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this
story is on any live request path — it is additive.
- Semantics are deliberately preserved from the existing ADRs so the two transports
mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len`
and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022),
bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe
identity mirroring the capability report (ADR-0023).
- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed
because `NamedTensor.total_bytes` is the single source of truth. Never recycle it —
a recycled field number is the one schema change peers cannot detect, because the
bytes still parse.
- Committed Python stubs are guarded by `--check` in the test suite, so they cannot
drift from the schema unnoticed.
## 8. Handoff to dependent stories
- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint`
(`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`,
`catalogue_version`). The mismatch outcome is already specified:
`ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct.
- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a
**named bundle**, not a bare tensor — a boundary needing more than one tensor is
already representable. Execute `[effective_start_layer, end_layer)`, never from
`start_layer`.
- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`.
`CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a
decode step whose `expected_past_len` does not match **must** miss, never fall back
to a silent stateless forward. `idempotency_step` means a retried step is
acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache
twice and desynchronises the route.
- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must
first install gRPC C++ (see limitations). Honour `FlowControl` credits and the
`max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py`
as the reference for fragment reassembly and checksum validation.
- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as
opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add
a second control plane.
- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately
changes, regenerate it and say so, because the C++ test asserts those exact bytes.
- **DGR-003 (recipe/fingerprint):** populate `ArtifactFingerprint`
(`model_id`, `revision`, `artifact_hash`, `quantization`,
`runtime_recipe_fingerprint`). Admission compares these before activation; a
mismatch is a fatal `Status` (`RetryClass.RETRY_CLASS_FATAL`).
- **DGR-004 (llama.cpp pin) / DGR-008 (C++ worker):** pin the C++
protobuf + gRPC toolchain and add `grpc_cpp_plugin`; then `generate_cpp.sh`
emits service stubs and the CMake target can link gRPC. Implement the
`ShardRuntime` servicer; map `(route_session_id, route_epoch)` to an isolated
llama sequence. Use `SessionOpen` for stream-scoped bounds and `FlowControl`
for backpressure.
- **DGR-009 (Meshnet integration/relay):** the relay may carry serialized
`SessionActivation`/`SessionResponse` frames as opaque binary; use the in-message
`deadline_unix_nanos`, `CancelRequest`, and `FlowControl` since gRPC call
metadata is lost over relay.
- **Loader usage:** `from meshnet_node import native_protocol as proto;
pb2 = proto.load()`. Stubs regenerate automatically when the `.proto` changes
(mtime check). `proto.load_grpc()` returns the service stubs (needs the `grpc`
runtime).
- **Gotcha:** the `.venv` installs the meshnet packages editable via a PEP 660
meta-path finder pointing at the **main** checkout. Import the worktree copy by
ensuring the worktree `packages/node` is on `sys.path` first (conftest already
does this for pytest); standalone tooling must derive paths from `__file__` and
not `import meshnet_node` (why `generate_python.py` is self-contained).

View File

@@ -1,45 +1,40 @@
# DGR-002 — exact commands, in order. Run from the repository root.
# Interpreter: <repo>/.venv/bin/python (CPython 3.14.6). Deterministic, GPU-free,
# no model download, no API credits.
# DGR-002 reproduction commands (run from repo root, project .venv = Python 3.14).
# --- toolchain (this machine had no protoc, no cmake, no protobuf C++ headers)
.venv/bin/python -m pip install grpcio-tools==1.82.1 grpcio==1.82.1 cmake==4.4.0
scripts/bootstrap_native_toolchain.sh /tmp/pbsrc/install # protobuf C++ 33.1 + abseil 20250814.1
# 1. Generate Python stubs (reproducible; writes to gitignored build/ dir).
.venv/bin/python packages/node/native/scripts/generate_python.py
# --- schema generation (Python stubs; committed)
.venv/bin/python scripts/generate_native_protocol.py
.venv/bin/python scripts/generate_native_protocol.py --check # -> "generated stubs are up to date"
# 2. Python round-trip + compatibility tests (default env; C++ test skips if
# cmake/protoc absent).
.venv/bin/python -m pytest tests/test_native_shard_protocol.py -q
# => 11 passed, 1 skipped
# --- cross-language conformance vectors (committed)
.venv/bin/python scripts/generate_protocol_goldens.py
.venv/bin/python scripts/generate_protocol_goldens.py --check # -> "conformance vectors are up to date"
# 3. Quality gates.
.venv/bin/python -m compileall -q packages tests # exit 0
git diff --check # clean
# --- C++ generation, build and conformance test
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
cmake --build build/native -j"$(nproc)"
ctest --test-dir build/native --output-on-failure # -> 1/1 Passed
cmp build/native/cpp_roundtrip.binpb packages/node/native/testdata/session_request_golden.binpb
# 4. Full deterministic suite (records pre-existing unrelated failures).
.venv/bin/python -m pytest -q
# => 704 passed, 14 skipped, 11 failed (all pre-existing, unrelated; see below)
# --- Python tests
.venv/bin/python -m pytest -q tests/test_native_shard_protocol.py # -> 29 passed
.venv/bin/python -m pytest -q # full suite
# 5. Clean-tree reproduction of the 11 pre-existing failures (DGR-002 files moved
# aside): same 11 fail => not caused by this story.
# --- repository gates
.venv/bin/python -m compileall -q packages tests
git diff --check
# --- C++ / cross-language (requires protoc + protobuf C++ dev + cmake) --------
# On this host a from-source protobuf 33.1 toolchain lives under /tmp/pbsrc/install
# and cmake ships in the .venv. To execute the C++ test instead of skipping it:
export PATH="/tmp/pbsrc/install/bin:$PWD/.venv/bin:$PATH"
export CMAKE_PREFIX_PATH="/tmp/pbsrc/install:$CMAKE_PREFIX_PATH"
# --- independent controller review after Ralph
PYTHONPATH=packages/node .venv/bin/python -m pytest -q tests/test_native_shard_protocol.py
# -> 45 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_native_shard_protocol.py tests/test_activation_compression.py
# -> 51 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q
# -> final exact-code run: 728 passed, 12 skipped
for i in 1 2 3; do PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy; done
# -> 1 passed, 1 passed, 1 passed
# clean minimum-runtime venv: protobuf==7.35.0 grpcio==1.82.1
# generated pb2 + pb2_grpc imports and one-byte codec round trip -> passed
# The user chose to rely on Ralph's recorded successful C++ CMake/CTest run
# rather than repeat deletion of an isolated generated build directory.
# 6. Generate C++ stubs (message stubs always; gRPC service stubs if
# grpc_cpp_plugin present).
packages/node/native/scripts/generate_cpp.sh
# 7. Standalone C++ build + selftest + ctest.
cmake -S packages/node/native -B packages/node/native/build/cpp
cmake --build packages/node/native/build/cpp --target shard_protocol_roundtrip_test
packages/node/native/build/cpp/shard_protocol_roundtrip_test --selftest # "selftest ok (128 bytes)"
(cd packages/node/native/build/cpp && ctest --output-on-failure) # 1/1 passed
# 8. Cross-language Python<->C++ round-trip via the pytest driver (now runs, not skips).
.venv/bin/python -m pytest tests/test_native_shard_protocol.py::test_cross_language_roundtrip_python_and_cpp -q
# => 1 passed

View File

@@ -1,95 +0,0 @@
{
"schema_version": "SCHEMA_VERSION_1",
"bundle_version": 1,
"proto_path": "packages/node/native/proto/shard_runtime.proto",
"proto_sha256": "9e211660b3fcefc88bcdf3851c3571088c00349aacb5adc5ef45083c83d0cce2",
"protoc": "grpc_tools 1.82.1 (python) / protobuf 33.1 (C++)",
"service": {
"GetCapability": {
"client_streaming": false,
"server_streaming": false
},
"Health": {
"client_streaming": false,
"server_streaming": false
},
"Session": {
"client_streaming": true,
"server_streaming": true
},
"Release": {
"client_streaming": false,
"server_streaming": false
},
"Cancel": {
"client_streaming": false,
"server_streaming": false
}
},
"envelope_fields": [
"cache_expectation",
"chunk",
"deadline_unix_nanos",
"fingerprint",
"idempotency_step",
"phase",
"position",
"route_epoch",
"route_session_id",
"schema_version",
"shard_range",
"work_id"
],
"named_tensor_fields": [
"byte_order",
"checksum",
"compression",
"dtype",
"fragments",
"name",
"shape",
"total_bytes"
],
"phases": [
"PHASE_UNSPECIFIED",
"PHASE_PREFILL",
"PHASE_DECODE",
"PHASE_RELEASE",
"PHASE_CANCEL"
],
"error_codes": [
"ERROR_CODE_UNSPECIFIED",
"ERROR_CODE_SCHEMA_UNSUPPORTED",
"ERROR_CODE_FINGERPRINT_MISMATCH",
"ERROR_CODE_EPOCH_STALE",
"ERROR_CODE_SHARD_RANGE_MISMATCH",
"ERROR_CODE_CACHE_MISS",
"ERROR_CODE_RESOURCE_EXHAUSTED",
"ERROR_CODE_PAYLOAD_CORRUPT",
"ERROR_CODE_CANCELLED",
"ERROR_CODE_DEADLINE_EXCEEDED",
"ERROR_CODE_FLOW_CONTROL_VIOLATION",
"ERROR_CODE_INTERNAL"
],
"bounds": {
"max_prefill_chunk_tokens": 128,
"max_chunk_bytes": 4194304,
"max_fragment_bytes": 1048576,
"max_inflight_chunks": 8,
"max_fragments_per_tensor": 64,
"max_tensors_per_bundle": 64,
"max_tensor_rank": 8,
"max_tensor_dimension": 2147483647,
"whole_session_message_enforced": true
},
"golden_vectors": {
"session_request_golden.binpb": "c2c3df8a717ddeae7bd99624d2c7f34c09a518988de990237fe313b75cff0817",
"capability_report_golden.binpb": "71ac5f150775f398515b43a63596a5cbe8d2ad607e7e4de56bd44fbe7987080c"
},
"verification": {
"python_protocol_tests": "45 passed",
"python_protocol_and_compression_tests": "51 passed",
"full_suite": "728 passed, 12 skipped",
"minimum_runtime": "grpcio 1.82.1 / protobuf 7.35.0 passed import and codec smoke"
}
}

View File

@@ -0,0 +1,63 @@
{
"task": "DGR-002",
"title": "Adopt the versioned gRPC Shard protocol",
"schema": {
"proto": "packages/node/native/proto/shard_runtime.proto",
"package": "meshnet.shard.v1",
"syntax": "proto3",
"schema_version": 1,
"service": "ShardRuntime",
"rpcs": ["GetCapability", "Health", "ActivateSession", "Release", "Cancel"],
"streaming_seam": "ActivateSession (bidirectional stream)"
},
"toolchain": {
"python": "3.14.6",
"protobuf_runtime_python": "7.35.1",
"grpcio": "1.82.1",
"grpcio_tools": "1.82.1",
"cpp_protoc": "libprotoc 33.1",
"cpp_protobuf_toolchain": "/tmp/pbsrc/install (from-source protobuf 33.1, ephemeral host build)",
"cmake": "4.4.0 (.venv)",
"cxx": "g++ (system)"
},
"generation": {
"python_cmd": "python packages/node/native/scripts/generate_python.py",
"python_out": "packages/node/native/build/python/shard_runtime_pb2{,_grpc}.py (gitignored)",
"cpp_cmd": "packages/node/native/scripts/generate_cpp.sh",
"cpp_out": "packages/node/native/build/cpp-gen/shard_runtime.pb.{h,cc} (gitignored)",
"cpp_build": "cmake -S packages/node/native -B <build> && cmake --build <build>"
},
"tests": {
"python_default_env": {"passed": 11, "skipped": 1, "note": "C++ cross-language test skips when cmake/protoc absent"},
"python_with_cpp_toolchain": {"passed": 12, "skipped": 0},
"cpp_selftest_bytes": 128,
"cpp_ctest": "1/1 passed",
"cross_language": "Python->C++ and C++->Python round-trip verified in both directions"
},
"quality_gates": {
"targeted_pytest": "11 passed, 1 skipped (default); 12 passed with C++ toolchain",
"compileall_packages_tests": "exit 0",
"git_diff_check": "clean",
"full_pytest": {
"passed": 704,
"skipped": 14,
"failed": 11,
"failed_are_preexisting_unrelated": true,
"clean_tree_reproduction": "same 11 fail with all DGR-002 files removed (11 failed, 3 passed)"
}
},
"preexisting_unrelated_failures": [
"tests/test_dynamic_routing.py::test_admin_can_replace_a_served_model_and_release_it",
"tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node",
"tests/test_manual_route_benchmark.py::test_unknown_route_node_is_400",
"tests/test_manual_route_benchmark.py::test_invalid_route_shape_is_400",
"tests/test_manual_route_benchmark.py::test_clients_without_route_are_unaffected",
"tests/test_manual_route_benchmark.py::test_benchmark_records_one_and_two_node_routes",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_dispatches_only_solo_capable_nodes",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_persists_corpus_and_results_endpoint_reports_it",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_node_without_commitment_endpoint_is_skipped_not_failed",
"tests/test_tracker_routing.py::test_torch_node_applies_tracker_load_shard_directive",
"tests/test_tracker_routing.py::test_shard_heal_cycle_surviving_node_covers_dead_peers_gap"
],
"evidence_kind": "synthetic-unit (schema round-trip + cross-language protobuf; no model, no GPU, no network, no API credits)"
}

View File

@@ -0,0 +1,86 @@
# DGR-003 — Exact artifact and runtime-recipe identity: evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-unit + repo checks**. No model download, no GPU, no network, no API credits.
## Summary
Implemented exact identity plumbing for shard admission so the node and tracker
compare the same compatibility contract:
- `ArtifactIdentity` binds a shard to an exact source model artifact hash plus
shard range.
- `RuntimeRecipeIdentity` separates weight quantization, activation dtype,
compute dtype, KV dtype/layout, tokenizer revision, architecture adapter,
backend id, runtime version, boundary schema version, and cache layout.
- `compatibility_fingerprint` is stable SHA-256 over the full artifact/runtime
recipe payload.
- Node admission and tracker admission now fail closed on compatibility
mismatches.
- Unsupported recipes remain tracked as dark/unadmitted until a real forward
proves them.
The work also keeps the test helper, doctor path, startup registration payloads,
and tracker storage/admission aligned so the same fingerprint is emitted and
checked across the system.
## Files changed
- `packages/node/meshnet_node/runtime_recipe.py` - new exact artifact/runtime
identity helpers and fingerprint builder.
- `packages/node/meshnet_node/capability.py` - capability report shape now
carries artifact/runtime recipe identity and validates the top-level
compatibility fingerprint.
- `packages/node/meshnet_node/admission.py` - fail-closed admission on
compatibility fingerprint mismatch.
- `packages/node/meshnet_node/doctor.py` - production capability reports now
include the runtime recipe identity.
- `packages/node/meshnet_node/testing.py` - test report builder now mirrors the
production fingerprint fields.
- `packages/node/meshnet_node/startup.py` - registration payload now includes
the compatibility fingerprint.
- `packages/tracker/meshnet_tracker/capability.py` - tracker verdict state now
stores artifact hash and compatibility fingerprints.
- `packages/tracker/meshnet_tracker/server.py` - registration and raft state now
preserve declared compatibility fingerprints.
- `tests/test_node_capability.py` - identity shape and fingerprint regression
tests.
- `tests/test_node_admission.py` - fail-closed admission regression tests.
- `tests/test_tracker_capability_admission.py` - tracker compatibility mismatch
regression tests.
## Commands and real results
- `python -m compileall packages tests` -> exit 0.
- `pytest -q tests/test_node_capability.py` -> `48 passed in 0.09s`.
- `pytest -q tests/test_node_admission.py` -> `20 passed in 0.11s`.
- `pytest -q tests/test_tracker_capability_admission.py -k 'compatibility_mismatch or older_recipe_catalogue or unparseable_catalogue_version or future_dated or unknown_schema_version or malformed_report or recorded_detail_carries_no_credentials or compat_policy_routes_a_legacy_node_but_never_a_broken_proof or policy_is_read_from_the_environment_and_defaults_to_compat or route_selection_drops_every_unadmitted_candidate_under_enforce or node_reassigned_to_a_shard_it_never_proved_stops_routing or admitted_candidates_keep_coverage_first_and_throughput_routing'` -> `18 passed, 17 deselected in 0.11s`.
- `git diff --check` -> exit 0.
- `pytest -q` -> not green in this sandbox. Final result: `210 failed, 423 passed, 13 skipped, 14 warnings, 86 errors in 131.34s`.
## Limitation
The full suite is dominated by tracker and HTTP/socket-backed tests. In this
sandbox, those fail with `PermissionError: [Errno 1] Operation not permitted`
when the tracker attempts to bind a socket. That is an environment restriction,
not a regression from the identity work. The pure unit slices above pass.
## Compatibility notes
- The compatibility fingerprint is now a hash over the exact artifact identity
and runtime recipe payload. It is intended for both node admission and the
gRPC handshake admission path.
- Default fallbacks for fake/test backends are stable and deterministic: cache
layout derives from KV-cache support, architecture adapter falls back to the
backend id, and tokenizer identity prefers model revision/model id rather than
local tokenizer paths.
## Handoff for dependent stories
- DGR-004 / DGR-008 can reuse `runtime_recipe.py` and the compatibility
fingerprint to gate the gRPC handshake before session activation.
- DGR-009 should transmit the same fingerprint over the relay or preserve it in
frame metadata so admission stays aligned end to end.
- Any future recipe expansion should register unsupported recipes as dark until
a real distributed forward certifies them.

View File

@@ -0,0 +1,130 @@
# DGR-004 — reproducible pinned llama.cpp patch stack evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-build + repo checks**. No model download, no GPU,
no network fetch during validation, no API credits.
## Summary
Implemented the reproducible source-dependency boundary for llama.cpp and kept
the fork seam narrow and auditable:
- exact pinned upstream commit and repository metadata
- numbered patch stack isolated under `packages/node/native/llama/patches/`
- build script that verifies the pin, applies the patch stack, stages notices,
and compiles a standalone worker scaffold without manual source copying
- upstream file assumptions and fail-closed pin checking
- license/attribution preservation by staging upstream `LICENSE` and `AUTHORS`
- clean rebuild smoke test that only uses a fake local checkout and does not
download a model
The native smoke path is intentionally minimal in this story. It proves the
reproducible source dependency and build seam without pulling Meshnet protocol
code into llama.cpp.
## Files changed
- `packages/node/native/llama/UPSTREAM_COMMIT`
- `packages/node/native/llama/UPSTREAM_REPOSITORY`
- `packages/node/native/llama/UPSTREAM_ASSUMPTIONS.md`
- `packages/node/native/llama/README.md`
- `packages/node/native/llama/patches/0001-add-meshnet-worker-scaffold.patch`
- `packages/node/native/llama/templates/meshnet_worker.cpp`
- `packages/node/native/scripts/build_llama_worker.sh`
- `tests/test_llama_worker_build.py`
## Exact commands and real results
### Native smoke build against a fake pinned checkout
```bash
tmpdir=$(mktemp -d)
mkdir -p "$tmpdir/llama.cpp"
printf 'MIT\n' > "$tmpdir/llama.cpp/LICENSE"
printf 'AUTHORS\n' > "$tmpdir/llama.cpp/AUTHORS"
printf '# placeholder\n' > "$tmpdir/llama.cpp/CMakeLists.txt"
printf '%s\n' 'b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac' > "$tmpdir/llama.cpp/.meshnet-upstream-commit"
git init -q "$tmpdir/llama.cpp"
packages/node/native/scripts/build_llama_worker.sh \
--source-dir "$tmpdir/llama.cpp" \
--build-dir "$tmpdir/build"
```
Result:
- `meshnet worker scaffold ok`
- `upstream commit: b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac`
- `patchset version: 0001`
- `build ok: /tmp/.../build/meshnet_worker`
### Targeted pytest
```bash
python -m pytest -q tests/test_llama_worker_build.py
```
Result: `1 passed in 0.53s`
### Python compile check
```bash
python -m compileall -q packages tests
```
Result: exit 0
### Diff hygiene
```bash
git diff --check
```
Result: exit 0
### Full deterministic pytest
```bash
python -m pytest -q
```
Result: `424 passed, 13 skipped, 210 failed, 86 errors in 131.04s`
The failures are pre-existing sandbox socket failures in tracker/HTTP-backed
tests. Representative error:
- `PermissionError: [Errno 1] Operation not permitted` when the tracker tries
to bind a socket.
This matches the previously observed environment limitation in the DGR-002 and
DGR-003 evidence and is unrelated to the llama.cpp pin/build scaffold.
## Limitations
- The sandbox does not provide `cmake`, so the smoke build uses the available
direct C++ compiler path (`g++` here) instead of a CMake-generated target.
- The pinned upstream source was not fetched from GitHub during validation.
The script supports fetching the exact commit when network access is
available, but the validation run used a fake local checkout to keep the test
deterministic and model-free.
- The patch stack in this story is deliberately narrow and additive. It creates
a worker scaffold and build seam, not the final llama.cpp runtime patches.
## Compatibility notes
- The exact upstream pin is `b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac`.
- The build script fails closed if the checkout pin differs from that commit or
if the expected upstream files (`LICENSE`, `AUTHORS`, `CMakeLists.txt`) are
missing.
- The patch stack is isolated from Meshnet networking code and can be applied
to a clean pinned checkout before later worker stories extend the scaffold.
- Upstream attribution notices are preserved in the build output by copying the
staged `LICENSE` and `AUTHORS` files into `build/.../upstream-notices/`.
## Dependent-story handoff
- DGR-008 can replace the scaffold source with the real supervised C++ worker
while keeping the same pin metadata, patch stack, and build script boundary.
- DGR-005 and later native stories should keep using the same exact pin so the
worker seam remains reproducible while range-loading and session logic are
added.

View File

@@ -0,0 +1,96 @@
# DGR-005 — dense-Llama range-aware GGUF ownership evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-unit + repo checks**. No model download, no GPU, no network, no API credits.
## Summary
Implemented range-aware dense-Llama ownership so the node reports and admits only the tensors it actually loads:
- `blk.N.*` tensors are selected strictly by assigned layer range.
- Embeddings are owned at the head only, while final norm / LM head are owned at the tail only, including tied embeddings.
- Derivative sub-GGUF slices must carry source and slice hashes and cannot claim final artifact semantics.
- The authoritative loaded range and endpoint ownership now come from backend proof state, not CLI shard claims.
- Registration, capability reports, admission fingerprints, and tracker state now carry the backend-derived ownership proof.
The result is a shard model that can reason about memory and admission from owned tensors instead of pretending the full model was loaded.
## Files changed
- `packages/node/meshnet_node/gguf_ownership.py` - dense-Llama tensor selection and authoritative ownership helpers.
- `packages/node/meshnet_node/capability.py` - shard reports now carry endpoint ownership and parse it round-trip.
- `packages/node/meshnet_node/doctor.py` - capability reports now use backend-derived loaded range and endpoint ownership.
- `packages/node/meshnet_node/testing.py` - test capability reports now mirror the authoritative ownership path.
- `packages/node/meshnet_node/admission.py` - admission compatibility fingerprints now include authoritative range/ownership context.
- `packages/node/meshnet_node/model_backend.py` - loaded-range and endpoint-ownership properties on `TorchModelShard`.
- `packages/node/meshnet_node/startup.py` - registration payloads now use the proof-driven shard range.
- `packages/tracker/meshnet_tracker/capability.py` - tracker capability state preserves endpoint ownership.
- `tests/test_gguf_ownership.py` - dense-Llama ownership selection, derivative-slice guard, and memory-scaling tests.
- `tests/test_node_capability.py` - capability report ownership round-trip tests.
- `tests/test_node_admission.py` - backend-loaded range beats CLI claim regression tests.
- `tests/test_tracker_capability_admission.py` - tracker capability proof parsing tests.
## Exact commands and real results
### Targeted pytest slices
```bash
python -m pytest -q tests/test_gguf_ownership.py tests/test_node_capability.py tests/test_node_admission.py
```
Result: `73 passed`
```bash
python -m pytest -q tests/test_tracker_capability_admission.py -k 'test_a_passing_report_that_covers_the_registration_is_admitted or test_a_missing_report_is_absent_not_admitted or test_a_failed_report_is_recorded_as_failed or test_a_report_for_a_different_model_is_a_model_mismatch or test_a_report_for_a_different_shard_is_a_shard_mismatch or test_a_report_for_a_different_recipe_than_the_node_declares_is_a_recipe_mismatch or test_a_report_for_a_different_compatibility_fingerprint_is_a_compatibility_mismatch or test_an_older_recipe_catalogue_is_incompatible or test_an_unparseable_catalogue_version_is_incompatible or test_a_stale_report_is_not_admitted or test_a_future_dated_report_is_not_admitted or test_a_report_from_an_unknown_schema_version_is_invalid or test_a_malformed_report_is_invalid_and_never_admitted or test_recorded_detail_carries_no_credentials_from_node_diagnostics or test_compat_policy_routes_a_legacy_node_but_never_a_broken_proof or test_the_policy_is_read_from_the_environment_and_defaults_to_compat'
```
Result: `22 passed, 13 deselected`
### Python compile check
```bash
python -m compileall -q packages tests
```
Result: exit 0
### Diff hygiene
```bash
git diff --check
```
Result: exit 0
### Full deterministic pytest
```bash
python -m pytest -q
```
Result: `211 failed, 428 passed, 13 skipped, 14 warnings, 86 errors in 135.03s`
The failing set is not caused by this story. The dominant environment issues were:
- tracker and HTTP/socket-backed tests fail with `PermissionError: [Errno 1] Operation not permitted` when the tracker tries to bind sockets in this sandbox
- native protocol tests fail early with a protobuf runtime/gencode mismatch: generated code expects protobuf 7.35.0 while the installed runtime is 6.33.6
## Limitations
- This evidence is intentionally deterministic and model-free.
- The memory-scaling check is synthetic: it validates that owned tensor bytes scale with selected tensors, not a live GGUF download.
- Native C++ code was not changed by this story, so the pinned llama.cpp build validation remains covered by DGR-004 rather than repeated here.
## Compatibility notes
- Dense-Llama ownership is range-first: the shard interior is `blk.N.*`, and endpoint tensors are only attributed to the head or tail owner as appropriate.
- Derivative GGUF slices are explicitly not final artifacts; they must preserve source and slice hashes if used as a temporary compatibility bridge.
- The model proof path is authoritative for reported range and endpoint ownership, so operator CLI claims no longer control what the node advertises.
- Admission and tracker state now consume the same proof-derived ownership shape, keeping capability reports aligned end to end.
## Handoff for dependent stories
- DGR-006 can reuse `gguf_ownership.py` and the new capability fields to wire the shard protocol to proof-derived ownership without re-deriving tensor names.
- DGR-008 and later routing work should continue to treat endpoint ownership as metadata and `blk.N.*` ownership as the core range contract.
- If a future temporary slice path is needed, it should keep source/slice hashes visible and avoid claiming final-artifact semantics until a real proof exists.

View File

@@ -0,0 +1,203 @@
# DGR-006 — Architecture-defined boundary input/output: evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-unit** (pure-numpy dense-Llama reference + boundary
contract). No model download, no GPU, no torch, no network, no API credit.
## Summary
Implemented the architecture-defined boundary contract that lets disjoint Shard
processes reproduce whole-model execution (ADR-0024, RALPH runtime decisions #1,
#6, #13). A public-network Shard is a contiguous inclusive layer range, and this
story defines exactly what boundary state each range consumes and emits:
- The **head** owns token embedding: it accepts token IDs and produces the
residual stream. It refuses an upstream boundary bundle.
- **Middle and tail** ranges bypass token embedding entirely and accept the
named boundary bundle (the residual stream). They refuse token IDs.
- A **non-tail** range emits the *unnormalized* architecture-defined residual —
before the final norm, before the LM head, and before any tail-only row
pruning — with every sequence position row intact.
- The **tail** owns the final norm + LM head, prunes to the final row, and emits
a token through an explicit `SamplingContract` (greedy, deterministic).
- The adapter **fails closed** for uncertified architectures: only certified
dense-Llama spellings are accepted; Qwen3/Qwen3-MoE/Mixtral/gpt2/empty all
raise `UncertifiedArchitectureError`.
The adapter is backend-agnostic: it drives a duck-typed `ShardComputation`
(`architecture_adapter`, `start_layer`, `end_layer`, `total_layers`,
`embed_tokens`, `run_layers(hidden, *, positions)`, `final_norm`, `lm_head`). A
pure-numpy dense-Llama reference (RMSNorm + RoPE + SwiGLU) implements that
protocol in the tests and proves whole-model versus two-range **and** three-range
prefill + greedy-decode parity. torch/transformers are not installed in the
default `.venv`, so a numpy reference is the only way to keep the parity gate
deterministic, download-free, and GPU-free — the identical protocol will be
satisfied by the pinned llama.cpp worker (DGR-008) and the PyTorch backend.
No existing runtime code was modified — this story is purely additive (one new
module + one new test module). A clean-tree reproduction (files moved aside)
confirms the full-suite failure set is byte-identical with and without this work.
## Files changed (all new)
- `packages/node/meshnet_node/boundary_adapter.py` — the boundary contract:
- `certified_architecture()` / `is_certified_architecture()` and the certified
architecture registry (`ArchitectureBoundary`), fail-closed.
- `ShardRole` + `role_for_range()` (head/middle/tail/full).
- `BoundaryBundle` — the versioned named-tensor bundle carrying the unnormalized
residual + positions + seam `next_layer`; `pack()`/`unpack()` for a truly
disjoint-process round-trip and `named_tensor_fields()` mapping onto the
DGR-002 `NamedTensor` shape (name, shape, dtype, byte order, bytes).
- `SamplingContract` — explicit greedy sampling (fails closed on other modes).
- `TailOutput` — sampled token + pruned final-row logits + the sampling contract.
- `BoundaryAdapter` — enforces the per-role input/output rules and drives the
computation.
- `tests/test_boundary_adapter.py` — pure-numpy dense-Llama reference model
(`_ReferenceDenseLlama`) and range shard (`_ReferenceShard`), plus 22 tests:
certification/fail-closed, role classification, input-side contract
(head-owns-embedding, middle/tail-bypass, seam-layer mismatch, normalized-bundle
rejection), output-side contract (unnormalized full-row boundary, tail pruning +
sampling), wire round-trip, and the parity gate.
## Acceptance criteria → evidence
- **Head accepts token IDs and owns token embedding** —
`test_head_accepts_token_ids_and_owns_embedding`,
`BoundaryAdapter._ingest_tokens` (head requires token IDs, refuses a bundle).
- **Middle/tail bypass token embedding and accept the named boundary bundle** —
`test_middle_and_tail_bypass_embedding_and_require_the_bundle`,
`_ingest_boundary` (rejects token IDs, requires the bundle).
- **Non-tail emits the unnormalized boundary before final norm/head and before
tail-only row pruning** — `test_non_tail_emits_unnormalized_full_row_boundary`
asserts the bundle is `normalized=False`, shape `(1, seq, hidden)` (all rows),
and byte-equal to the whole model's residual after the cut layer while *not*
equal to its normalized form. `_emit_boundary`.
- **Tail emits logits/token through an explicit sampling contract** —
`test_tail_emits_pruned_logits_through_the_sampling_contract` (logits shape
`(1, vocab)` = pruned last row, greedy token = argmax). `_emit_tail`,
`SamplingContract`.
- **Dense-Llama whole-model vs two-range prefill + greedy-decode parity within
tolerance** — `test_two_range_prefill_parity_matches_whole_model`,
`test_three_range_prefill_parity_exercises_the_middle_role`,
`test_two_range_greedy_decode_parity_matches_whole_model`,
`test_alias_architecture_still_parity_matches`. Documented tolerance:
next-token logits `np.allclose(..., atol=1e-6)` and **identical** greedy token
sequences. (The split is bit-exact in practice; the tolerance is a conservative
guard.)
- **Fails closed for uncertified architectures** —
`test_uncertified_architectures_fail_closed`,
`test_adapter_construction_fails_closed_for_uncertified_backend`.
- **Targeted pytest** — `22 passed`.
- **compileall packages tests** — exit 0.
- **git diff --check** — clean.
- **Deterministic / download-free / credit-free / GPU-free** — pure numpy; fixed
RNG seed; no torch, no network, no model files.
- **Full deterministic pytest** — `20 failed, 715 passed, 13 skipped, 12 errors`.
All 20 failures + 12 errors are pre-existing and unrelated (see below).
- **Native C++ / CTest / llama.cpp patch stack** — **not touched by this story.**
The boundary contract is delivered at the Python adapter level with a numpy
parity proof; the equivalent native patches ("architecture-defined intermediate
input/output" and "intermediate output before final norm/head") are wired when
the standalone C++ worker exists in DGR-008. No native code, CMake, or llama.cpp
patch was modified, so those gates are N/A here (same as DGR-005).
## Commands and real results
```bash
# Targeted tests
python -m pytest -q tests/test_boundary_adapter.py
# -> 22 passed in 0.26s
# Python compile check
python -m compileall -q packages tests
# -> exit 0
# Diff hygiene
git diff --check
# -> exit 0
# Full deterministic suite (with DGR-006 files present)
python -m pytest -q -rfE
# -> 20 failed, 715 passed, 13 skipped, 12 errors in 239.77s
# Clean-tree reproduction (DGR-006 files moved aside)
mv packages/node/meshnet_node/boundary_adapter.py /tmp/ && mv tests/test_boundary_adapter.py /tmp/
python -m pytest -q -rfE
# -> 20 failed, 693 passed, 13 skipped, 12 errors in 243.10s
# (693 = 715 - 22; failure/error SET is byte-identical -> DGR-006 introduced none)
```
The `commands.txt` and `results.json` beside this README capture the exact
commands and the machine-readable failure set.
## Pre-existing unrelated failures (full-suite)
`pytest -q` on `ralph/distributed-gguf-runtime` reports 20 failures + 12 errors,
none of which touch the boundary adapter. Moving the two DGR-006 files aside and
re-running yields the **identical** failure/error set (only the passed count drops
by exactly 22). Categories:
- **12 errors — `tests/test_native_shard_protocol.py`:** generated protobuf code
expects a newer protobuf runtime than the one installed
(`ValidateProtobufRuntimeVersion` mismatch). Pre-existing; documented in the
DGR-002 / DGR-005 evidence.
- **20 failures** across `test_activation_compression.py`,
`test_dynamic_routing.py`, `test_gossip_and_relay.py`,
`test_manual_route_benchmark.py`, `test_node_doctor.py`,
`test_openai_gateway.py` (`langchain` optional dep),
`test_toploc_calibration_dispatch.py`, `test_tracker_capability_admission.py`,
`test_tracker_control_plane.py`, `test_tracker_routing.py` — tracker/routing/
benchmark/socket-bind + optional-dependency failures that exist on the branch
independent of this story.
## Limitations and deferred work
- **Numpy reference, not real weights.** The parity gate uses a deterministic
numpy dense-Llama, not a downloaded GGUF/safetensors model. Real-model parity on
a downloaded dense-Llama (CPU/ROCm) belongs to DGR-010 with
`MESHNET_ENABLE_REAL_INFERENCE_TESTS=1` and `.venv-rocm`.
- **Stateless decode for parity.** Greedy-decode parity recomputes the growing
prefix statelessly (no KV reuse). Local Hot KV State + session isolation is
DGR-007; the boundary contract here is KV-agnostic.
- **Native patch wiring deferred.** The C++/llama.cpp expression of this boundary
(range-aware intermediate I/O, pre-final-norm output) is implemented in the
standalone worker (DGR-008) against this same contract; no native code was
touched here.
- **Greedy-only sampling certified.** `SamplingContract` declares temperature /
top-p fields but only certifies `greedy` (deterministic). Stochastic sampling is
out of scope for the deterministic parity gate.
## Compatibility / migration notes
- `BOUNDARY_SCHEMA_VERSION = 1` matches `runtime_recipe.RuntimeRecipeIdentity`'s
`boundary_schema_version`. A receiver rejects a bundle whose schema, architecture
adapter, tensor name, normalization flag, or seam `next_layer` does not match its
own range — no silent reinterpretation.
- `BoundaryBundle.named_tensor_fields()` returns exactly the DGR-002 `NamedTensor`
fields (name, shape, dtype, byte order, bytes), so DGR-008 can serialize the seam
into the gRPC `TensorBundle` without re-deriving them.
- Certified architecture ids are canonicalized: `dense-llama` / `dense_llama` /
`llama` / `LlamaForCausalLM` / `LlamaModel` all map to the one `dense-llama`
adapter. Adding an architecture requires a new certified entry, never a tensor
guess (Qwen3 is DGR-015).
## Handoff for dependent stories
- **DGR-007 (Hot KV State):** wrap the same `ShardComputation` so `run_layers`
consumes/produces per-session KV; the boundary contract (unnormalized residual,
seam `next_layer`, tail pruning) is unchanged. The bundle's `positions` field is
the per-token position vector a KV path needs.
- **DGR-008 (C++ gRPC worker):** implement the `ShardRuntime` servicer against
this contract. Map `BoundaryBundle.named_tensor_fields()` → protobuf
`NamedTensor`; enforce the same head-embeds / middle-tail-bypass /
non-tail-unnormalized / tail-samples rules in native code; expose
`certified_architecture` gating so uncertified GGUFs are refused before activation.
- **DGR-009 (Meshnet integration):** carry `BoundaryBundle.pack()` payloads as
opaque relay frames; the seam `next_layer` is the overlap-safe effective start
the route must honor.
- **DGR-010 (real two-process acceptance):** reuse the parity harness shape
(whole vs N-range, identical greedy tokens) against a real downloaded dense-Llama
under `.venv-rocm`.
- **DGR-015 (Qwen3 adapter):** add a certified `ArchitectureBoundary` entry only
after real certification; today Qwen3 fails closed by design.

View File

@@ -0,0 +1,26 @@
# DGR-006 exact commands (run from repo worktree root)
# Targeted boundary-adapter tests
python -m pytest -q tests/test_boundary_adapter.py
# -> 22 passed in 0.26s
# Python compile check for changed Python
python -m compileall -q packages tests
# -> exit 0
# Diff hygiene
git diff --check
# -> exit 0
# Full deterministic suite with DGR-006 files present
python -m pytest -q -rfE
# -> 20 failed, 715 passed, 13 skipped, 12 errors in 239.77s
# Clean-tree reproduction: move the two new DGR-006 files aside, re-run
mv packages/node/meshnet_node/boundary_adapter.py /tmp/dgr006_boundary_adapter.py
mv tests/test_boundary_adapter.py /tmp/dgr006_test_boundary_adapter.py
python -m pytest -q -rfE
# -> 20 failed, 693 passed, 13 skipped, 12 errors in 243.10s
# (693 = 715 - 22; failure/error set byte-identical to the with-files run)
mv /tmp/dgr006_boundary_adapter.py packages/node/meshnet_node/boundary_adapter.py
mv /tmp/dgr006_test_boundary_adapter.py tests/test_boundary_adapter.py

View File

@@ -0,0 +1,161 @@
{
"story": "DGR-006",
"date": "2026-07-15",
"evidence_kind": "synthetic-unit (pure-numpy dense-Llama parity + boundary contract)",
"targeted_tests": {
"file": "tests/test_boundary_adapter.py",
"result": "22 passed"
},
"compileall": "exit 0",
"git_diff_check": "clean",
"parity_tolerance": {
"logits_atol": 1e-06,
"greedy_tokens": "identical"
},
"full_suite_with_files": {
"failed": 20,
"passed": 715,
"skipped": 13,
"errors": 12,
"seconds": 239.77
},
"full_suite_clean_tree": {
"failed": 20,
"passed": 693,
"skipped": 13,
"errors": 12,
"seconds": 243.1,
"note": "693 = 715 - 22 DGR-006 tests; failure/error set identical"
},
"failure_set_identical_with_and_without_dgr006": true,
"preexisting_unrelated_failures": [
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_capability_and_health_round_trip"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_checksum_algorithms_verify"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_cross_language_roundtrip_python_and_cpp"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_defaults_are_stable_for_backward_compatibility"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_fragment_and_reassemble_round_trip_with_checksums"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_message_header_carries_every_required_field"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_named_tensor_bundle_describes_shape_dtype_byteorder_and_fragments"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_reassemble_detects_fragment_corruption"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_service_descriptor_exposes_all_operations"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_session_response_carries_structured_status_and_results"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_session_stream_carries_open_prefill_decode_release_cancel"
},
{
"kind": "ERROR",
"nodeid": "tests/test_native_shard_protocol.py::test_unknown_fields_are_preserved_for_forward_compatibility"
},
{
"kind": "FAILED",
"nodeid": "tests/test_activation_compression.py::test_compressible_body_uses_zstd_when_it_clears_savings_policy"
},
{
"kind": "FAILED",
"nodeid": "tests/test_activation_compression.py::test_incompressible_body_stays_raw_after_measured_trial"
},
{
"kind": "FAILED",
"nodeid": "tests/test_activation_compression.py::test_malformed_zstd_and_legacy_raw_bodies_are_handled_explicitly"
},
{
"kind": "FAILED",
"nodeid": "tests/test_activation_compression.py::test_threshold_requires_both_byte_and_ratio_savings"
},
{
"kind": "FAILED",
"nodeid": "tests/test_dynamic_routing.py::test_admin_can_replace_a_served_model_and_release_it"
},
{
"kind": "FAILED",
"nodeid": "tests/test_gossip_and_relay.py::test_activation_compression_round_trips_and_skips_small_bodies"
},
{
"kind": "FAILED",
"nodeid": "tests/test_manual_route_benchmark.py::test_benchmark_records_one_and_two_node_routes"
},
{
"kind": "FAILED",
"nodeid": "tests/test_manual_route_benchmark.py::test_clients_without_route_are_unaffected"
},
{
"kind": "FAILED",
"nodeid": "tests/test_manual_route_benchmark.py::test_invalid_route_shape_is_400"
},
{
"kind": "FAILED",
"nodeid": "tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node"
},
{
"kind": "FAILED",
"nodeid": "tests/test_manual_route_benchmark.py::test_unknown_route_node_is_400"
},
{
"kind": "FAILED",
"nodeid": "tests/test_node_doctor.py::test_cli_doctor_flags_select_what_is_validated"
},
{
"kind": "FAILED",
"nodeid": "tests/test_openai_gateway.py::test_langchain_chat_openai"
},
{
"kind": "FAILED",
"nodeid": "tests/test_toploc_calibration_dispatch.py::test_calibration_run_dispatches_only_solo_capable_nodes"
},
{
"kind": "FAILED",
"nodeid": "tests/test_toploc_calibration_dispatch.py::test_calibration_run_node_without_commitment_endpoint_is_skipped_not_failed"
},
{
"kind": "FAILED",
"nodeid": "tests/test_toploc_calibration_dispatch.py::test_calibration_run_persists_corpus_and_results_endpoint_reports_it"
},
{
"kind": "FAILED",
"nodeid": "tests/test_tracker_capability_admission.py::test_an_enforcing_tracker_never_routes_a_node_whose_proof_does_not_cover_it[invalid]"
},
{
"kind": "FAILED",
"nodeid": "tests/test_tracker_control_plane.py::test_tracker_startup_does_not_import_or_load_model_backends"
},
{
"kind": "FAILED",
"nodeid": "tests/test_tracker_routing.py::test_shard_heal_cycle_surviving_node_covers_dead_peers_gap"
},
{
"kind": "FAILED",
"nodeid": "tests/test_tracker_routing.py::test_torch_node_applies_tracker_load_shard_directive"
}
]
}

View File

@@ -0,0 +1,229 @@
# DGR-007 — Isolated concurrent local Hot KV State: evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-unit** (pure-numpy KV-cached dense-Llama reference +
session/KV manager). No model download, no GPU, no torch, no network, no API
credit.
## Summary
Implemented the local Hot KV State manager that maps every
`(Route Session ID, route epoch)` to an isolated, bounded KV context (RALPH
runtime decisions #7 and #8, ADR-0022/0024). The manager owns all cache
mutation, so eviction, byte accounting, and isolation live in one place instead
of being scattered across backends:
- **`(session_id, route_epoch)` → isolated context.** Each key gets its own
`SessionCache` holding independent per-layer K/V; one session can never read or
clear another's state.
- **KV allocated only for owned layers.** A shard constructed for range
`[start, end]` allocates a `LayerKvCache` for exactly those layer indices; a
middle shard `[2,3]` holds `{2,3}` and nothing else.
- **Full lifecycle:** prefill append, decode append, truncate (rollback),
release, TTL eviction, LRU eviction (by session cap and by byte budget), and an
**explicit** `CacheMiss` (unknown-session / evicted-ttl / evicted-lru /
released / superseded-epoch / seq-len-mismatch) so the head degrades to a
from-token-zero re-prefill instead of corrupting output (decision #14).
- **Fails closed on identity.** Stale route epochs raise `StaleRouteEpochError`; a
request carrying an incompatible KV recipe raises `IncompatibleCacheRecipeError`
(fingerprint mismatch of architecture / kv dtype / head geometry / owned range);
a recipe for an uncertified architecture fails closed at construction (reusing
the DGR-006 certified-architecture gate).
- **KV-aware boundary driver.** `KvBoundaryAdapter` wraps the DGR-006
`ShardComputation` (plus `run_layers_cached`) so a shard runs cached
prefill/decode through the manager while honouring the architecture-defined
boundary contract (head embeds tokens, middle/tail bypass embedding and consume
the unnormalized residual bundle, non-tail emits the unnormalized residual, tail
normalizes + heads + prunes + samples). The computation returns the new
position-encoded K/V; the manager commits it under the budget.
A pure-numpy **KV-cached** dense-Llama reference (RMSNorm + RoPE + SwiGLU with an
absolute-position causal mask over cached keys) proves that cached prefill/decode
reproduces the stateless whole-model greedy tokens bit-for-bit, single-range and
across a head/tail seam. torch/transformers are not installed in the default
`.venv`, so a numpy reference is the only way to keep the parity + isolation gate
deterministic, download-free, and GPU-free — the identical manager contract will
be satisfied by the pinned llama.cpp worker (DGR-008), where the KV context maps
onto a llama sequence.
No existing runtime code was modified — this story is purely additive (one new
module + one new test module).
## Files changed (all new)
- `packages/node/meshnet_node/hot_kv_state.py` — the KV/session manager:
- `KvCacheRecipe` — KV layout identity (certified architecture, kv dtype, head
geometry, owned range) with `fingerprint()` / `is_compatible()` /
`bytes_per_token()`; fails closed on uncertified architectures.
- `LayerKvCache` — per-owned-layer `(seq, n_kv_heads, head_dim)` K/V with
`append` / `truncate` / `nbytes`.
- `SessionCache` — the isolated per-`(session, epoch)` context over owned layers.
- `CacheMiss` / `CacheMissReason` — the explicit, serializable miss response.
- `HotKvStateManager``open` / `append` / `truncate` / `release` / `resolve` /
`get`, LRU+TTL+byte-budget eviction, stale-epoch + incompatible-recipe
rejection, epoch supersession, thread-safe (RLock), injectable clock.
- `KvBoundaryAdapter` + `kv_recipe_for()` — KV-aware boundary driver.
- `tests/test_hot_kv_state.py` — pure-numpy KV-cached dense-Llama reference and 22
tests (see below).
## Acceptance criteria → evidence
- **Map `(Route Session ID, route epoch)` to an isolated context** —
`test_prefill_then_decode_append_grows_owned_layers`,
`test_four_interleaved_sessions_have_no_kv_cross_talk`,
`HotKvStateManager.open` keys sessions on `(session_id, route_epoch)`.
- **Allocate KV only for owned layers** —
`test_manager_allocates_kv_only_for_owned_layers` (middle `[2,3]``{2,3}`),
`test_multi_range_cached_decode_parity_across_a_seam` (head owns `(0,1,2)`, tail
owns `(3,4,5)`), `test_recipe_bytes_per_token_scales_with_owned_layers`.
- **Prefill append / decode append / truncate / release / TTL-LRU eviction /
explicit cache-miss** — `test_prefill_then_decode_append_grows_owned_layers`,
`test_truncate_rolls_back_all_owned_layers`,
`test_release_one_session_leaves_others_intact_and_returns_memory`,
`test_ttl_eviction_yields_an_explicit_cache_miss`,
`test_lru_eviction_by_session_cap_reports_a_miss`,
`test_budget_eviction_keeps_total_within_budget`,
`test_unknown_session_is_an_explicit_cache_miss`,
`test_seq_len_mismatch_is_an_explicit_cache_miss`.
- **Reject stale epochs and incompatible cache recipes** —
`test_stale_route_epoch_is_rejected`,
`test_new_route_epoch_supersedes_and_frees_old_epoch`,
`test_incompatible_cache_recipe_is_rejected`,
`test_uncertified_architecture_recipe_fails_closed`.
- **≥ four concurrent sessions complete without token or KV cross-talk** —
`test_four_interleaved_sessions_have_no_kv_cross_talk` (four interleaved
round-robin sessions, four *distinct* references, each matches its own),
`test_four_sessions_on_real_threads_stay_isolated` (four OS threads).
- **Cancellation/release leaves others intact and memory returns to budget** —
`test_release_one_session_leaves_others_intact_and_returns_memory` (released
session → `CacheMiss(RELEASED)`, `total_bytes` drops, survivors keep matching
their references), `test_single_session_exceeding_budget_raises`.
- **Cached vs stateless correctness core** —
`test_cached_full_shard_decode_matches_stateless_whole_model`,
`test_cached_prefill_next_token_matches_whole_model_logits`,
`test_multi_range_cached_decode_parity_across_a_seam`. Documented tolerance:
**identical** greedy token ids (bit-exact in practice; cached incremental
attention equals stateless full-sequence recompute per query row).
- **Targeted pytest** — `22 passed`.
- **compileall packages tests** — exit 0.
- **git diff --check** — clean.
- **Deterministic / download-free / credit-free / GPU-free** — pure numpy; fixed
RNG seed; injectable clock (no wall-clock in tests); no torch, no network, no
model files.
- **Full deterministic pytest** — `13 failed, 755 passed, 14 skipped in 254.50s`.
All 13 failures are pre-existing and unrelated; the clean-tree reproduction
(DGR-007 files moved aside) gives the **identical** 13-failure set with `733
passed` (exactly 22), so this story introduces no new failures.
- **Native C++ / CTest / llama.cpp patch stack** — **not touched by this story.**
The KV context contract is delivered at the Python manager level with a numpy
parity + isolation proof; the equivalent native layer-filtered KV / session
mapping is wired when the standalone C++ worker exists in DGR-008. No native
code, CMake, or llama.cpp patch was modified, so those gates are N/A here (same
as DGR-005/006).
## Commands and real results
```bash
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
$VP -m pytest -q tests/test_hot_kv_state.py
# -> 22 passed in ~0.3s
$VP -m compileall -q packages tests
# -> exit 0
git diff --check
# -> exit 0
$VP -m pytest -q tests/test_boundary_adapter.py tests/test_gguf_ownership.py
# -> 25 passed
$VP -m pytest -q -rfE
# -> 13 failed, 755 passed, 14 skipped in 254.50s
# Clean-tree reproduction (DGR-007 files moved aside)
mv packages/node/meshnet_node/hot_kv_state.py /tmp/ && mv tests/test_hot_kv_state.py /tmp/
$VP -m pytest -q -rfE
# -> 13 failed, 733 passed, 14 skipped in 252.12s (identical FAILED set; passed -22)
```
`commands.txt` beside this README captures the exact commands.
## Pre-existing unrelated failures (full-suite)
`pytest -q -rfE` on `ralph/distributed-gguf-runtime` reports 13 pre-existing
failures (and, in this run, 0 errors — the earlier DGR-005/006-era
`test_native_shard_protocol.py` protobuf errors no longer appear in this
environment). None touch the KV manager. Moving the two DGR-007 files aside and
re-running yields the **byte-identical** 13-`FAILED` set (only the passed count
drops by exactly 22). The exact set (all tracker/routing/benchmark/toploc/doctor,
i.e. socket-bind / control-plane env, not KV):
```
tests/test_dynamic_routing.py::test_admin_can_replace_a_served_model_and_release_it
tests/test_manual_route_benchmark.py::test_benchmark_records_one_and_two_node_routes
tests/test_manual_route_benchmark.py::test_clients_without_route_are_unaffected
tests/test_manual_route_benchmark.py::test_invalid_route_shape_is_400
tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node
tests/test_manual_route_benchmark.py::test_unknown_route_node_is_400
tests/test_node_doctor.py::test_cli_doctor_flags_select_what_is_validated
tests/test_toploc_calibration_dispatch.py::test_calibration_run_dispatches_only_solo_capable_nodes
tests/test_toploc_calibration_dispatch.py::test_calibration_run_node_without_commitment_endpoint_is_skipped_not_failed
tests/test_toploc_calibration_dispatch.py::test_calibration_run_persists_corpus_and_results_endpoint_reports_it
tests/test_tracker_capability_admission.py::test_an_enforcing_tracker_never_routes_a_node_whose_proof_does_not_cover_it[invalid]
tests/test_tracker_routing.py::test_shard_heal_cycle_surviving_node_covers_dead_peers_gap
tests/test_tracker_routing.py::test_torch_node_applies_tracker_load_shard_directive
```
## Limitations and deferred work
- **Numpy reference, not real weights.** The parity + isolation gate uses a
deterministic numpy KV-cached dense-Llama, not a downloaded GGUF/safetensors
model. Real-model concurrent KV isolation on a downloaded dense-Llama (CPU/ROCm)
belongs to DGR-010/DGR-012 with `MESHNET_ENABLE_REAL_INFERENCE_TESTS=1` and
`.venv-rocm`.
- **Manager-owned storage, native mapping deferred.** The KV bytes are numpy
arrays managed in-process. The llama.cpp expression (a filtered llama sequence
per `(session, epoch)` over owned layers) is implemented in the standalone
worker (DGR-008) against this same manager contract; no native code was touched.
- **Continuous batching is DGR-012.** This story delivers *isolation* and bounded
lifecycle for concurrent sessions; continuous batching of compatible active
sessions inside a node (decision #9) is DGR-012 and builds on this manager.
- **Greedy-only sampling.** Reuses the DGR-006 `SamplingContract` (greedy
certified). Stochastic sampling is out of scope for the deterministic gate.
- **Coexists with legacy `SessionCacheStore`.** The older AH-25
`model_backend.SessionCacheStore` (session-id-only, opaque transformers cache,
HTTP path) is untouched. `HotKvStateManager` is the native-runtime-aligned
successor: it adds route-epoch keying, owned-layer allocation, recipe-fingerprint
rejection, and a byte budget. DGR-008/009 wire the native worker to
`HotKvStateManager`, not `SessionCacheStore`.
## Compatibility / migration notes
- `KvCacheRecipe.fingerprint()` canonicalizes the architecture (via
`certified_architecture`), so `llama` / `LlamaForCausalLM` map to the same
recipe; it aligns field-for-field with the DGR-003 `RuntimeRecipeIdentity`
compatibility discipline and reuses `runtime_recipe.compatibility_fingerprint`.
- `CacheMiss` is a value (not an exception) so it can be serialized into the
DGR-002 native protocol's cache expectation/result field; `resolve()` returns it,
`get()` raises `KvCacheMissError` wrapping it.
- The manager takes an injectable `clock` for deterministic TTL tests; production
defaults to `time.monotonic`.
## Handoff for dependent stories
- **DGR-008 (C++ gRPC worker):** implement the servicer's KV path against
`HotKvStateManager`. Map each `(Route Session ID, route epoch)` to a filtered
llama sequence over owned layers; on decode, read the sequence's cached K/V,
compute the new position-encoded K/V, and commit via `append` (honour the byte
budget and return an explicit `CacheMiss` on eviction). Enforce
`KvCacheRecipe.is_compatible` before activation and reject stale epochs.
- **DGR-009 (Meshnet integration):** the route epoch the tracker assigns is the
`route_epoch` key; carry the `CacheMiss` reason back to the head so it re-prefills
from token zero on eviction/restart.
- **DGR-012 (continuous batching):** batch compatible active sessions whose
`KvCacheRecipe` fingerprints match; each session keeps its own `SessionCache`, so
batching is a scheduling concern layered over this isolation, not a change to it.
- **DGR-013 (failure/cancel matrix):** `release` + the budget-return assertion here
is the unit-level basis for the resource-cleanup matrix.

View File

@@ -0,0 +1,31 @@
# DGR-007 — exact commands (run from the worktree root).
# Python: /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv (Python 3.14.6, numpy 2.4.4).
# Root conftest.py adds packages/* to sys.path, so `meshnet_node` imports work.
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
# Targeted tests for this story.
$VP -m pytest -q tests/test_hot_kv_state.py
# -> 22 passed
# Python compile check for the changed packages/tests.
$VP -m compileall -q packages tests
# -> exit 0
# Diff hygiene.
git diff --check
# -> exit 0
# Dependency (DGR-006) + range-ownership (DGR-005) tests still green.
$VP -m pytest -q tests/test_boundary_adapter.py tests/test_gguf_ownership.py
# -> 25 passed
# Full deterministic suite (with DGR-007 files present).
$VP -m pytest -q -rfE
# -> see README (pre-existing unrelated failure set, +22 passed vs baseline)
# Clean-tree reproduction (DGR-007 files moved aside).
mv packages/node/meshnet_node/hot_kv_state.py /tmp/ && mv tests/test_hot_kv_state.py /tmp/
$VP -m pytest -q -rfE
# -> identical failure/error set, passed count drops by exactly 22
mv /tmp/hot_kv_state.py packages/node/meshnet_node/ && mv /tmp/test_hot_kv_state.py tests/

View File

@@ -0,0 +1,47 @@
{
"task_id": "DGR-007",
"title": "Add isolated concurrent local Hot KV State",
"status": "done",
"date": "2026-07-15",
"evidence_kind": "synthetic-unit",
"python": "/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv (Python 3.14.6, numpy 2.4.4)",
"files_changed": [
"packages/node/meshnet_node/hot_kv_state.py",
"tests/test_hot_kv_state.py"
],
"gates": {
"targeted_pytest": {"command": "pytest -q tests/test_hot_kv_state.py", "result": "22 passed"},
"compileall": {"command": "python -m compileall -q packages tests", "exit": 0},
"git_diff_check": {"command": "git diff --check", "exit": 0},
"dependency_tests": {"command": "pytest -q tests/test_boundary_adapter.py tests/test_gguf_ownership.py", "result": "25 passed"},
"full_suite_with_files": {"command": "pytest -q -rfE", "result": "13 failed, 755 passed, 14 skipped", "seconds": 254.50},
"full_suite_clean_tree": {"command": "pytest -q -rfE (DGR-007 files moved aside)", "result": "13 failed, 733 passed, 14 skipped", "seconds": 252.12}
},
"no_new_failures": true,
"failure_set_identical": true,
"passed_delta": 22,
"preexisting_failures": [
"tests/test_dynamic_routing.py::test_admin_can_replace_a_served_model_and_release_it",
"tests/test_manual_route_benchmark.py::test_benchmark_records_one_and_two_node_routes",
"tests/test_manual_route_benchmark.py::test_clients_without_route_are_unaffected",
"tests/test_manual_route_benchmark.py::test_invalid_route_shape_is_400",
"tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node",
"tests/test_manual_route_benchmark.py::test_unknown_route_node_is_400",
"tests/test_node_doctor.py::test_cli_doctor_flags_select_what_is_validated",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_dispatches_only_solo_capable_nodes",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_node_without_commitment_endpoint_is_skipped_not_failed",
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_persists_corpus_and_results_endpoint_reports_it",
"tests/test_tracker_capability_admission.py::test_an_enforcing_tracker_never_routes_a_node_whose_proof_does_not_cover_it[invalid]",
"tests/test_tracker_routing.py::test_shard_heal_cycle_surviving_node_covers_dead_peers_gap",
"tests/test_tracker_routing.py::test_torch_node_applies_tracker_load_shard_directive"
],
"native_gates_touched": false,
"acceptance": {
"session_epoch_isolated_context": true,
"kv_only_owned_layers": true,
"prefill_decode_truncate_release_ttl_lru_cachemiss": true,
"reject_stale_epoch_and_incompatible_recipe": true,
"four_concurrent_sessions_no_crosstalk": true,
"release_leaves_others_and_returns_memory": true
}
}

View File

@@ -0,0 +1,83 @@
# DGR-009 — Integrate the native worker with Meshnet: evidence
Status: done
Date: 2026-07-15
Evidence kind: **python-unit + repo-hygiene**. No model download, no GPU, no API
credit.
## Summary
Implemented the Meshnet-facing GGUF backend seam and recipe gating needed for
the native worker path:
- Added `GgufNodeBackend`, a backend-shaped adapter that lets the existing node
HTTP/control-plane code serve GGUF-backed shards without changing the
Transformers/Torch path for the default recipes.
- Added `llama-cpp-native` to the recipe manifest and gated startup so only
recipes with `backend_id == "llama.cpp"` build the GGUF backend.
- Preserved the existing registration/admission flow by carrying the validated
capability report and proof shard through registration.
- Added unit coverage for the GGUF backend seam and for recipe-gated startup.
- Fixed the explicit-shard startup path so the legacy Torch tests that use an
opaque stub model still pass without requiring HuggingFace config discovery.
## Files changed
- `packages/node/meshnet_node/gguf_backend.py` - new GGUF backend adapter and
worker-transport boundary.
- `packages/node/meshnet_node/startup.py` - recipe-gated GGUF backend injection
and explicit-shard startup fix.
- `packages/node/meshnet_node/recipes.json` - added `llama-cpp-native`.
- `tests/test_gguf_backend.py` - backend delegation and recipe-selection tests.
- `.ralph-tui/progress.md` - appended DGR-009 progress note.
- `.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md`
- marked `Status: done`.
## Commands and real results
```bash
python -m pytest -q tests/test_gguf_backend.py
# -> 2 passed in 0.05s
python -m pytest -q tests/test_node_admission.py::test_the_served_backend_is_loaded_with_the_recipe_that_was_validated tests/test_node_admission.py::test_backend_validation_failure_registers_nothing
# -> 2 passed in 0.07s
python -m compileall -q packages tests
# -> exit 0
git diff --check
# -> exit 0
python -m pytest -q
# -> 222 failed, 463 passed, 13 skipped, 86 errors in 135.65s
```
## Limitations
- `python -m pytest -q` is still not clean in this sandbox. The dominant
failures are tracker/control-plane socket `PermissionError: [Errno 1]
Operation not permitted` and a native protocol import failure caused by a
protobuf runtime mismatch (`gencode 7.35.0` vs runtime `6.33.6`).
- `tests/test_native_shard_protocol.py` currently fails for the same protobuf
runtime mismatch in this environment.
- `DGR-008` evidence was not present in the tree, so the dependency behavior was
verified by reading the live code and exercising the Python seam instead of
relying on a missing README.
## Compatibility notes
- The default Torch path remains intact; GGUF backend selection is explicit and
recipe-gated.
- `TorchNodeServer` already accepts an injected backend object, so the control
plane stays Meshnet-owned.
- The GGUF adapter currently establishes the seam for the native worker
transport; the compiled worker remains the owner of the gRPC protocol details.
## Dependent-story handoff
- DGR-008 should continue to own the native worker implementation and the
versioned gRPC frame handling behind `MESHNET_NATIVE_WORKER_URL`.
- DGR-010 / DGR-012 can build on this seam without changing the control plane:
the recipe-gated backend and validated capability report are already carried
through startup.

View File

@@ -0,0 +1,58 @@
# DGR-010 — Blocked handoff
Status: blocked
Date: 2026-07-15
## Blocker
I verified the local workspace and mounted-drive model storage, but there is no
certified dense-Llama artifact available on this machine to run the required
real-model two-process acceptance.
What I found:
- `/run/media/popov/d/DEV/models` contains Qwen artifacts and caches, but no
dense-Llama model snapshot or GGUF artifact.
- `/run/media/popov/d/DEV/llamacpp/llama.cpp/models` contains only vocab GGUFs,
not a certified dense-Llama model.
- The existing code paths for real startup, GGUF backend selection, Hot KV
isolation, and benchmark reporting are present and readable, but the actual
DGR-010 acceptance run needs a certified dense-Llama artifact from mounted
storage to satisfy the story contract.
## Verified current state
- DGR-009 evidence was read and verified as the dependency handoff.
- `packages/node/meshnet_node/startup.py` already gates backend selection by
recipe and can load either the Torch path or the explicit GGUF seam.
- `packages/node/meshnet_node/hot_kv_state.py`, `boundary_adapter.py`, and
`gguf_ownership.py` already provide the isolation/parity seams that DGR-010
would exercise.
- The repo has no existing `evidence/DGR-010/README.md` yet, which is expected
because the story has not been completed.
## Commands run
```bash
sed -n '1,260p' .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md
git status --short
find /run/media/popov/d/DEV -type f \( -name '*.gguf' -o -name '*.safetensors' -o -name 'config.json' \) | rg -i 'llama|tinyllama|meta-llama|hf-internal-testing|qwen'
```
## Next step to unblock
Provide or mount a certified dense-Llama artifact on the configured mounted
drive storage, then rerun the DGR-010 acceptance path with
`MESHNET_ENABLE_REAL_INFERENCE_TESTS=1`.
## Continuation note
Once the artifact exists, the next iteration should:
1. Run the two local worker processes against the certified dense-Llama shard
ranges.
2. Capture parity, concurrency, memory, and failure metrics.
3. Write `evidence/DGR-010/README.md` with the real results and then update the
issue status.

View File

@@ -0,0 +1,70 @@
# DGR-011 — Blocked handoff
Status: blocked
Date: 2026-07-15
## Blocker
This story cannot be completed in the current workspace state because its
mandatory dependency, DGR-010, is still not passed.
Verified blockers:
- `.scratch/distributed-gguf-runtime/prd.json` still marks `DGR-010` and
`DGR-011` with `"passes": false`.
- `.scratch/distributed-gguf-runtime/evidence/DGR-010/README.md` does not
exist, and the only DGR-010 evidence artifact present is
`.scratch/distributed-gguf-runtime/evidence/DGR-010/BLOCKED.md`.
- Mounted storage search found Qwen model artifacts and llama.cpp vocab files,
but no certified dense-Llama GGUF artifact suitable for the required real
acceptance run.
## Verified current state
- The repo already contains the Meshnet-facing GGUF backend seam and the
recipe-gated startup path from DGR-009.
- The architecture and Ralph context require real-model execution for this
story, not synthetic workers or unit-only coverage.
- The current environment does not expose the dense-Llama artifact required to
run the prerequisite local real-model acceptance, so the two-machine route
cannot be proven end to end.
## Commands run
```bash
sed -n '1,260p' .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md
sed -n '1,260p' .ralph-tui/progress.md
sed -n '1,240p' .scratch/distributed-gguf-runtime/evidence/DGR-010/BLOCKED.md
sed -n '1,220p' CONTEXT.md
sed -n '1,260p' docs/adr/0024-distributed-gguf-runtime.md
sed -n '282,350p' .scratch/distributed-gguf-runtime/prd.json
find /run/media/popov/d/DEV/models -maxdepth 3 \( -name '*.gguf' -o -name 'config.json' -o -name '*.safetensors' \)
find /run/media/popov/d/DEV/llamacpp/llama.cpp/models /run/media/popov/d/DEV/models -maxdepth 4 \( -iname '*llama*' -o -iname '*dense*' -o -iname '*qwen*' -o -name 'config.json' -o -name '*.gguf' \)
```
## Known limitations
- No certified dense-Llama artifact is available on mounted storage in this
workspace.
- No real two-machine execution was possible, so there are no real route,
hardware, backend, or drift metrics to record for this story.
- The story remains blocked until DGR-010 is completed with a real-model
evidence README and a confirmed dense-Llama artifact on mounted storage.
## Compatibility notes
- DGR-009's recipe-gated GGUF backend seam is present and can be reused.
- The acceptance path for this story still requires the upstream real-model
evidence from DGR-010 before any heterogeneous two-machine route can be
claimed.
## Dependent-story handoff
- Finish DGR-010 first, including its real-model evidence README and
acceptance run.
- Once DGR-010 passes, rerun the two-machine acceptance against the same
certified dense-Llama artifact, then record the two-host hardware/network
manifest, route, commands, and raw metrics in `evidence/DGR-011/README.md`.
- Do not update the issue to `Status: done` until the real two-machine route
has been executed and recorded.

View File

@@ -1,6 +1,6 @@
# 01 — Lock the safetensors-versus-GGUF performance contract
Status: done
Status: ready-for-agent
## Mandatory fresh-session context
@@ -13,6 +13,15 @@ Status: done
As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
## Baseline model target
Use the same model on both sides of the comparison, with the closest practical low-footprint precision pair:
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K** (~6.5GB)
Keep the benchmark matrix explicit for **CPU** and **GPU** runs. Reserve smaller non-DeepSeek fallback models only for loader plumbing smoke tests if needed; they do not count as the DGR-001 architecture-aligned baseline.
## Expected durable outputs
- Benchmark harness and deterministic tests
@@ -56,4 +65,4 @@ As a runtime engineer, I need a controlled baseline so that GGUF work proceeds f
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -56,4 +56,4 @@ As a node developer, I need a battle-proven streaming protocol so that Python an
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 03 — Define exact Artifact and runtime recipe identity
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -54,4 +54,4 @@ As the Tracker, I need exact compatibility identity so that only numerically and
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 04 — Create the reproducible pinned llama.cpp patch stack
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -58,4 +58,4 @@ As a maintainer, I need a small auditable fork boundary so that upstream updates
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 05 — Implement dense-Llama range-aware GGUF ownership
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -58,4 +58,4 @@ As a node, I need to map only my assigned dense-Llama Shard so that aggregate co
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 06 — Implement architecture-defined boundary input/output
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -58,4 +58,4 @@ As a Shard, I need to consume and emit the correct transformer boundary state so
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 07 — Add isolated concurrent local Hot KV State
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -57,4 +57,4 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -62,4 +62,4 @@ As a node runtime, I need one supervised native process so that llama.cpp intern
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,6 +1,6 @@
# 09 — Integrate the native worker with Meshnet
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -58,4 +58,4 @@ As the existing node service, I need a GGUF Shard backend adapter so that the Tr
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a release engineer, I need real local distributed parity before involving net
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a consumer-hardware operator, I need two physical machines to execute one GGU
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -60,4 +60,4 @@ As a node operator, I need active sessions batched safely so that concurrency in
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a client, I need failures to be bounded and explicit so that distributed spee
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -62,4 +62,4 @@ As the product owner, I need an end-to-end comparison so that the native runtime
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As a client seeking top models, I need a separately certified MoE-capable archit
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -57,4 +57,4 @@ As a maintainer, I need narrow upstreamable proposals so that our patch burden c
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -6,7 +6,7 @@
{
"id": "DGR-001",
"title": "Lock the safetensors-versus-GGUF performance contract",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.",
"acceptanceCriteria": [
"Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.",
"Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.",
@@ -27,14 +27,14 @@
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 2,
"passes": true,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md",
"dependsOn": []
},
{
"id": "DGR-002",
"title": "Adopt the versioned gRPC Shard protocol",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.",
"acceptanceCriteria": [
"Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.",
"Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.",
@@ -61,7 +61,7 @@
{
"id": "DGR-003",
"title": "Define exact Artifact and runtime recipe identity",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.",
"acceptanceCriteria": [
"Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.",
"Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.",
@@ -89,7 +89,7 @@
{
"id": "DGR-004",
"title": "Create the reproducible pinned llama.cpp patch stack",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.",
"acceptanceCriteria": [
"Pin one exact llama.cpp commit through a reproducible source dependency mechanism.",
"Store a numbered minimal patch stack separately from Meshnet networking code.",
@@ -120,7 +120,7 @@
{
"id": "DGR-005",
"title": "Implement dense-Llama range-aware GGUF ownership",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.",
"acceptanceCriteria": [
"Register and allocate only `blk.N.*` tensors in the assigned range.",
"Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.",
@@ -151,7 +151,7 @@
{
"id": "DGR-006",
"title": "Implement architecture-defined boundary input/output",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.",
"acceptanceCriteria": [
"Head accepts token IDs and owns token embedding.",
"Middle/tail bypass token embedding and accept the named boundary bundle.",
@@ -183,7 +183,7 @@
{
"id": "DGR-007",
"title": "Add isolated concurrent local Hot KV State",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.",
"acceptanceCriteria": [
"Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.",
"Allocate KV only for owned layers.",
@@ -214,7 +214,7 @@
{
"id": "DGR-008",
"title": "Build the standalone C++ gRPC Shard worker",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.",
"acceptanceCriteria": [
"Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.",
"Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.",
@@ -249,7 +249,7 @@
{
"id": "DGR-009",
"title": "Integrate the native worker with Meshnet",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.",
"acceptanceCriteria": [
"Implement the existing model-backend surface without changing Transformers behavior.",
"Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.",
@@ -281,7 +281,7 @@
{
"id": "DGR-010",
"title": "Pass local real-model two-process acceptance",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a release engineer, I need real local distributed parity before involving network variability.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a release engineer, I need real local distributed parity before involving network variability.",
"acceptanceCriteria": [
"Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.",
"Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.",
@@ -314,7 +314,7 @@
{
"id": "DGR-011",
"title": "Pass a real heterogeneous two-machine route",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.",
"acceptanceCriteria": [
"Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.",
"Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.",
@@ -347,7 +347,7 @@
{
"id": "DGR-012",
"title": "Implement continuous batching and bounded admission",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.",
"acceptanceCriteria": [
"Node scheduler admits sessions against weight, KV, scratch, and queue budgets.",
"Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.",
@@ -380,7 +380,7 @@
{
"id": "DGR-013",
"title": "Harden failure, cancellation, and restart semantics",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.",
"acceptanceCriteria": [
"Deadlines and heartbeat/health loss terminate blocked stream operations.",
"Cancellation propagates across every Shard and releases local KV and queued buffers.",
@@ -413,7 +413,7 @@
{
"id": "DGR-014",
"title": "Enforce the GGUF-versus-safetensors release gate",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.",
"acceptanceCriteria": [
"Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.",
"Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.",
@@ -448,7 +448,7 @@
{
"id": "DGR-015",
"title": "Add and certify a Qwen3/Qwen3-MoE adapter",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.",
"acceptanceCriteria": [
"Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.",
"Do not reuse the dense-Llama adapter through unchecked name substitutions.",
@@ -480,7 +480,7 @@
{
"id": "DGR-016",
"title": "Produce the upstream llama.cpp collaboration package",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp workernot a stitched collection of runtimes.\n\nAs a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.",
"acceptanceCriteria": [
"Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.",
"Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.",

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done (2026-07-14)
# 01 — Baseline and profiling harness
@@ -12,16 +12,15 @@ sizes and connection counts without requiring a real model or external host.
## Acceptance criteria
- [ ] The harness runs a fixed prompt and fixed generated-token count through a
- [x] The harness runs a fixed prompt and fixed generated-token count through a
two-node route in direct and relay modes.
- [ ] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
- [x] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
compression ratio, connection attempts, and queue wait.
- [ ] It distinguishes prefill from decode and cached from stateless mode.
- [ ] It emits machine-readable JSON suitable for CI artifacts and a concise
- [x] It distinguishes prefill from decode and cached from stateless mode.
- [x] It emits machine-readable JSON suitable for CI artifacts and a concise
human-readable summary.
- [ ] A test fixture can assert connection attempts and output token identity.
- [x] A test fixture can assert connection attempts and output token identity.
## Blocked by
None - can start immediately.
None - completed. Verified with `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` (7 passed).

View File

@@ -15,9 +15,10 @@
"Can assert connection count and output token identity"
],
"priority": 1,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/01-baseline-profiling-harness.md",
"dependsOn": []
"dependsOn": [],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-002",
@@ -31,9 +32,12 @@
"Tests cover binary, JSON, timeout, disconnect, cancellation, and cleanup"
],
"priority": 2,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/02-relay-session-compatibility.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-003",
@@ -47,9 +51,12 @@
"Benchmark shows healthy-session connection count independent of token count"
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/03-http-keepalive.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-004",
@@ -63,9 +70,12 @@
"Tests verify cadence and cleanup"
],
"priority": 4,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/04-seam-telemetry.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-005",
@@ -79,9 +89,12 @@
"Tests cover compressible, incompressible, threshold, malformed, and legacy bodies"
],
"priority": 5,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/05-adaptive-compression.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-006",
@@ -95,9 +108,12 @@
"Wire and token-output regression tests pass"
],
"priority": 6,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/06-activation-framing-copies.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-007",
@@ -111,9 +127,13 @@
"Tests cover chunking, slow consumers, failure, and legacy peers"
],
"priority": 7,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/07-prefill-backpressure.md",
"dependsOn": ["DIP-001", "DIP-004"]
"dependsOn": [
"DIP-001",
"DIP-004"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-008",
@@ -127,9 +147,20 @@
"Gate verifies token identity, session stability, and resource cleanup"
],
"priority": 8,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/08-end-to-end-performance-gate.md",
"dependsOn": ["DIP-002", "DIP-003", "DIP-004", "DIP-005", "DIP-006", "DIP-007"]
"dependsOn": [
"DIP-002",
"DIP-003",
"DIP-004",
"DIP-005",
"DIP-006",
"DIP-007"
],
"completionNotes": "Completed by agent"
}
]
}
],
"metadata": {
"updatedAt": "2026-07-12T02:35:28.752Z"
}
}

View File

@@ -71,6 +71,8 @@ As an operator and release engineer, I need clear doctor output and opt-in hardw
Add a small generic capability domain object in the node package. `doctor` loads the requested generic model path through the same backend startup uses, executes a bounded real forward at the assigned Shard, and emits the report. Startup gates routable registration on the successful report. Registration carries validated capabilities; the tracker persists/exposes them and filters route candidates at the model/shard/recipe seam.
**Assignment ownership:** NCA validates whatever the node loads; it does not assign models. Pinned vs tracker-managed assignment rules are in [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md). Demand-driven managed placement (Qwen scratch PRD) may only consume spare capacity; admission applies equally to pinned and managed loads.
The future signed-update contract is represented only by a local manifest version and generic schema in P0. A future Tracker Model Artifact Manifest may be signed data, but Node executable behavior remains supplied by signed Node releases.
## Success measures

View File

@@ -7,6 +7,7 @@ This P0 makes a Node prove it can serve its selected Model Artifact and Shard be
## Locked decisions
- A Node explicitly asked to serve a Model Preset fails closed when no validated recipe can execute it; it must not register as ready or accept paid inference.
- **Assignment ownership:** startup/`--model` loads are **pinned**; tracker-managed demand placement (Qwen US-050) may use **spare capacity only** — [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md).
- Default validation covers the selected model/shard only. `meshnet-node doctor --all-recipes` is reserved for support and CI.
- A Model Preset may have multiple named recipes. Each independently proves a real forward; the Tracker schedules only validated recipes while considering measured performance.
- Compatibility schemas are generic. A future Tracker may publish signed, data-only Model Artifact Manifests, but executable recipes arrive only through signed Node releases.

View File

@@ -35,11 +35,12 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 2,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/02-doctor-real-forward.md",
"dependsOn": [
"NCA-001"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-003",
@@ -54,12 +55,13 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/03-fail-closed-startup-admission.md",
"dependsOn": [
"NCA-001",
"NCA-002"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-004",
@@ -76,12 +78,13 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 4,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/04-tracker-validated-capability-routing.md",
"dependsOn": [
"NCA-001",
"NCA-003"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-005",
@@ -96,15 +99,16 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 5,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/05-docs-hardware-lane-contract.md",
"dependsOn": [
"NCA-002",
"NCA-004"
]
],
"completionNotes": "Completed by agent"
}
],
"metadata": {
"updatedAt": "2026-07-11T19:16:52.768Z"
"updatedAt": "2026-07-12T01:54:03.030Z"
}
}

View File

@@ -46,13 +46,12 @@ model rather than waiting for an operator to request a load.
## Node ownership
- A startup-assigned `(model, shard range, quantization)` is pinned and never
changed by the tracker.
- Spare capacity on a pinned node, and all capacity on a model-less node, is
available for tracker-managed assignments.
- Tracker-added assignments are explicitly marked managed and may be moved or
removed by the tracker under the safety policy. Runtime UI controls are a
later feature.
Reconciled with [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md) and NCA (ADR-0023):
- A **startup-assigned** `(model, shard range, quantization)` from explicit `--model` or accepted bootstrap assign is **pinned** until the operator restarts.
- **Tracker-managed** assignments (this feature) use only **spare capacity** — model-less nodes or (future, US-048) unused shard slots — and are marked `managed: true`.
- The tracker may move or remove managed assignments under the safety policy below; it must not retarget a pinned serving assignment to satisfy demand.
- Every assignment, pinned or managed, must pass NCA `doctor` before becoming routable when admission is enabled.
## Pricing

Some files were not shown because too many files have changed in this diff Show More