Compare commits
50 Commits
worktree-r
...
ralph/dist
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
989b55970b | ||
|
|
9e70b94417 | ||
|
|
9db036f91a | ||
|
|
369b2072cc | ||
|
|
81b1fa6074 | ||
|
|
02b3709311 | ||
|
|
737bade989 | ||
|
|
254627629b | ||
|
|
1fe31ef38d | ||
|
|
47b243cd98 | ||
|
|
2852b1f80b | ||
|
|
eaf00f6add | ||
|
|
22f28bd69a | ||
|
|
97e2784b37 | ||
|
|
c035bad5b7 | ||
|
|
a508768e8a | ||
|
|
e6f6782995 | ||
|
|
ba7c656364 | ||
|
|
b661590ac7 | ||
|
|
5b33bf8b99 | ||
|
|
c7554ef7d8 | ||
|
|
21e6c86147 | ||
|
|
def47f1a42 | ||
|
|
8cb00e951f | ||
|
|
7b3399760e | ||
|
|
22467f145c | ||
|
|
35af1e21de | ||
|
|
905ea16ce0 | ||
|
|
348b003d6e | ||
|
|
1e64a5b2b9 | ||
|
|
e2f3ae32b8 | ||
|
|
29351d6217 | ||
|
|
5c9a2f6c97 | ||
|
|
13d82f8032 | ||
|
|
d1a1400db9 | ||
|
|
5d87e81bc9 | ||
|
|
a6bcc69288 | ||
|
|
c938d38031 | ||
|
|
95245be512 | ||
|
|
180a7674e6 | ||
|
|
f420dc1092 | ||
|
|
efec84efef | ||
|
|
09af5c47f8 | ||
|
|
4cae4a6c5c | ||
|
|
e8ef2fd222 | ||
|
|
caa55b74bf | ||
|
|
b5fa7245df | ||
|
|
377346c301 | ||
|
|
9a1b15c020 | ||
|
|
95d79a0a16 |
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: ask-matt
|
||||
description: Ask which skill or flow fits your situation. A router over the user-invoked skills in this repo.
|
||||
description: Ask which skill or flow fits your situation. A router over the skills in this repo.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
@@ -8,26 +8,28 @@ disable-model-invocation: true
|
||||
|
||||
You don't remember every skill, so ask.
|
||||
|
||||
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone.
|
||||
A **flow** is a path through the skills. Most paths run along one **main flow**, and two **on-ramps** merge onto it. Everything else is standalone, or a vocabulary layer that runs underneath.
|
||||
|
||||
## The main flow: idea → ship
|
||||
|
||||
The route most work travels. You have an idea and want it built.
|
||||
|
||||
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone.)
|
||||
1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone. Both run the same `/grilling` primitive; `grill-with-docs` is the one that leaves a paper trail.)
|
||||
2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (see Crossing sessions):
|
||||
- **`/handoff`** out, then open a fresh session against that file,
|
||||
- **`/prototype`** to answer the question with throwaway code,
|
||||
- **`/handoff`** back what you learned, and reference it from the original idea thread.
|
||||
3. **Branch — is this a multi-session build?**
|
||||
- **Yes** → **`/to-prd`** (turn the thread into a PRD) → **`/to-issues`** (split the PRD into independently-grabbable issues). Because the issues are independent, **clear context between each one**: start a fresh session per issue and kick off **`/implement`** by passing it the PRD and the single issue to work on.
|
||||
- **Yes** → **`/to-spec`** (turn the thread into a spec), then **`/to-tickets`** to split it into tracer-bullet tickets, each declaring its **blocking edges**. On a local tracker that's one file per ticket under `.scratch/<feature>/issues/`, worked blockers-first by hand; on a real tracker the edges become native blocking links, so any ticket whose blockers are done can be grabbed — kick off **`/implement`** per ticket, **clearing context between each one**.
|
||||
- **No** → **`/implement`** right here, in the same context window.
|
||||
|
||||
Either way, **`/implement`** builds each issue by driving **`/tdd`** internally — one red-green slice at a time — then closes out by running **`/code-review`**, a two-axis review (Standards + Spec) of the diff, before committing. Reach for **`/tdd`** on its own when you just want to build a concrete behaviour test-first without a full spec, and **`/code-review`** on its own whenever you want to review a branch or PR against a fixed point.
|
||||
|
||||
### Context hygiene
|
||||
|
||||
Keep steps 1–3 in **one unbroken context window** — don't compact or clear until after `/to-issues` — so the grilling, PRD, and issues all build on the same thinking. Each `/implement` then starts fresh, working from the issue.
|
||||
Keep steps 1–3 in **one unbroken context window** — don't compact or clear until after `/to-tickets` — so the grilling, spec, and tickets all build on the same thinking. Each `/implement` then starts fresh, working from the ticket.
|
||||
|
||||
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-issues`, don't push on degraded — `/handoff` and continue in a fresh thread.
|
||||
The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-tickets`, don't push on degraded — `/handoff` and continue in a fresh thread.
|
||||
|
||||
## On-ramps
|
||||
|
||||
@@ -35,13 +37,26 @@ A starting situation that generates work, then merges onto the main flow.
|
||||
|
||||
- **Bugs and requests piling up** → **`/triage`**. It moves issues through triage roles and produces agent-ready issues, which **`/implement`** later picks up.
|
||||
|
||||
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Issues that `/to-issues` produced are already agent-ready, so **don't triage them**.
|
||||
Triage is only for issues **you didn't create** — bug reports, incoming feature requests, anything that arrives raw. Tickets that `/to-tickets` produced are already agent-ready, so **don't triage them**.
|
||||
|
||||
- **Something's broken** → **`/diagnosing-bugs`**. For the hard ones: the bug that resists a first glance, the intermittent flake, the regression that crept in between two known-good states. It refuses to theorise until it has a **tight feedback loop** — one command that already goes red on *this* bug — then fixes with a regression test. Its post-mortem hands off to **`/improve-codebase-architecture`** when the real finding is that there's no good seam to lock the bug down.
|
||||
|
||||
- **A huge, foggy effort — a greenfield project or a huge feature build, too big for one session** → **`/wayfinder`**, the most cognitively demanding flow here. When the way from here to the destination isn't visible yet, it charts a **shared map** of **decision tickets** on the issue tracker and resolves them one at a time — producing **decisions, not deliverables** — until the fog is pushed back and the way is clear. Where **`/grill-with-docs`** sharpens an idea you can hold in one session, wayfinder is for the idea you can't — and it's slower and denser, so save it for exactly that, never a well-scoped feature.
|
||||
|
||||
When the map clears, **it hands off, it doesn't build**: merge onto the main flow at **`/to-spec`**, which collapses the map's linked decisions into a buildable plan, then `/to-tickets` and `/implement` as usual. Looping the map straight into `/implement` skips that collapse and throws the linked detail away — go straight to `/implement` only when the effort turned out genuinely small.
|
||||
|
||||
## Codebase health
|
||||
|
||||
Not feature work — upkeep.
|
||||
|
||||
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces deepening opportunities; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`.
|
||||
- **`/improve-codebase-architecture`** — run whenever you have a spare moment to keep the codebase good for agents to operate in. It surfaces **deepening opportunities**; picking one _generates an idea_ you can take into the main flow at `/grill-with-docs`. It's the survey that finds the candidates; **`/codebase-design`** (below) is the bench you design the chosen one on.
|
||||
|
||||
## Vocabulary underneath
|
||||
|
||||
Two model-invoked references that run *beneath* the other skills — each the single source of truth for its vocabulary. Reach for them directly when the **words**, not the process, are the problem; or let the skills above pull them in.
|
||||
|
||||
- **`/domain-modeling`** — sharpen the project's *domain* language: challenge a fuzzy term, resolve an overloaded word ("account" doing three jobs), record a hard-to-reverse decision as an ADR. It's the active discipline `/grill-with-docs` drives to keep `CONTEXT.md` a clean glossary.
|
||||
- **`/codebase-design`** — the deep-module vocabulary (module, interface, depth, seam, adapter, leverage, locality) for designing a module's *shape*: a lot of behaviour behind a small interface at a clean seam. `/tdd` and `/improve-codebase-architecture` both speak it.
|
||||
|
||||
## Crossing sessions
|
||||
|
||||
@@ -53,6 +68,8 @@ Not feature work — upkeep.
|
||||
Off the main flow entirely.
|
||||
|
||||
- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but for when you have **no codebase**. Stateless: it saves nothing locally, builds no `CONTEXT.md`. Reach for it to sharpen any plan or design that doesn't live in a repo.
|
||||
- **`/prototype`** — a small, throwaway program that answers one design question: does this state model feel right, or what should this UI look like. Throwaway from day one — keep the answer, delete the code. It's the detour in step 2 of the main flow, but reach for it any time a design question is hard to settle on paper.
|
||||
- **`/research`** — delegate reading legwork to a **background agent**: it investigates a question against **primary sources**, then leaves a cited Markdown file in the repo. Keep working while it reads. The file it produces is something to take *into* the main flow at `/grill-with-docs` — research feeds the thinking, it doesn't replace it.
|
||||
- **`/teach`** — learn a concept over multiple sessions, using the current directory as a stateful workspace.
|
||||
- **`/writing-great-skills`** — reference for writing and editing skills well.
|
||||
|
||||
|
||||
5
.agents/skills/ask-matt/agents/openai.yaml
Normal file
5
.agents/skills/ask-matt/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Ask Matt"
|
||||
short_description: "Find the right skill or workflow"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -1,10 +1,12 @@
|
||||
---
|
||||
name: grilling
|
||||
description: Interview the user relentlessly about a plan or design. Use when the user wants to stress-test a plan before building, or uses any 'grill' trigger phrases.
|
||||
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
|
||||
---
|
||||
|
||||
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
|
||||
Interview me relentlessly about every aspect of this until we reach a shared understanding. Walk down each branch of the decision tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
|
||||
|
||||
Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering.
|
||||
|
||||
If a question can be answered by exploring the codebase, explore the codebase instead.
|
||||
If a *fact* can be found by exploring the environment (filesystem, tools, etc.), look it up rather than asking me. The *decisions*, though, are mine — put each one to me and wait for my answer.
|
||||
|
||||
Do not act on it until I confirm we have reached a shared understanding.
|
||||
|
||||
3
.agents/skills/grilling/agents/openai.yaml
Normal file
3
.agents/skills/grilling/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "Grilling"
|
||||
short_description: "Stress-test thinking one question at a time"
|
||||
@@ -9,7 +9,7 @@ Write a handoff document summarising the current conversation so a fresh agent c
|
||||
|
||||
Include a "suggested skills" section in the document, which suggests skills that the agent should invoke.
|
||||
|
||||
Do not duplicate content already captured in other artifacts (PRDs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
||||
Do not duplicate content already captured in other artifacts (specs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
||||
|
||||
Redact any sensitive information, such as API keys, passwords, or personally identifiable information.
|
||||
|
||||
|
||||
5
.agents/skills/handoff/agents/openai.yaml
Normal file
5
.agents/skills/handoff/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Handoff"
|
||||
short_description: "Compact a conversation into a handoff"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -1,15 +1,15 @@
|
||||
---
|
||||
name: implement
|
||||
description: "Implement a piece of work based on a PRD or set of issues."
|
||||
description: "Implement a piece of work based on a spec or set of tickets."
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Implement the work described by the user in the PRD or issues.
|
||||
Implement the work described by the user in the spec or tickets.
|
||||
|
||||
Use /tdd where possible, at pre-agreed seams.
|
||||
|
||||
Run typechecking regularly, single test files regularly, and the full test suite once at the end.
|
||||
|
||||
Once done, use /review to review the work.
|
||||
Once done, use /code-review to review the work.
|
||||
|
||||
Commit your work to the current branch.
|
||||
|
||||
5
.agents/skills/implement/agents/openai.yaml
Normal file
5
.agents/skills/implement/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Implement"
|
||||
short_description: "Build work from a spec or tickets"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -17,6 +17,11 @@ This command is _informed_ by the project's domain model and built on a shared d
|
||||
|
||||
### 1. Explore
|
||||
|
||||
**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look:
|
||||
|
||||
- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below.
|
||||
- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net.
|
||||
|
||||
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
|
||||
|
||||
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
|
||||
@@ -56,7 +61,7 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o
|
||||
|
||||
### 3. Grilling loop
|
||||
|
||||
Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
|
||||
Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
|
||||
|
||||
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
|
||||
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Improve Codebase Architecture"
|
||||
short_description: "Find and grill architecture improvements"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -36,7 +36,7 @@ The right shape depends on the question:
|
||||
|
||||
Pick whichever shape best fits the question being asked, *not* whichever is easiest to wire to a TUI. Keep it pure: no I/O, no terminal code, no `console.log` for control flow. The TUI imports it and calls into it; nothing flows the other direction.
|
||||
|
||||
This is what makes the prototype useful past its own lifetime. When the question's been answered, the validated reducer / machine / function set can be lifted into the real module — the TUI shell gets deleted.
|
||||
This is what makes the prototype useful past its own lifetime: when the question's been answered, the validated reducer / machine / function set can be lifted into the real module on its own.
|
||||
|
||||
### 4. Build the smallest TUI that exposes the state
|
||||
|
||||
@@ -66,9 +66,9 @@ If the host project has no task runner, just put the command at the top of the p
|
||||
|
||||
Give the user the run command. They'll drive it themselves; the interesting moments are when they say "wait, that shouldn't be possible" or "huh, I assumed X would be different" — those are the bugs in the _idea_, which is the whole point. If they want new actions added, add them. Prototypes evolve.
|
||||
|
||||
### 7. Capture the answer
|
||||
### 7. Capture the answer and the prototype
|
||||
|
||||
When the prototype has done its job, the answer to the question is the only thing worth keeping. If the user is around, ask what it taught them. If not, leave a `NOTES.md` next to the prototype so the answer can be filled in (or filled in by you, if you've watched the session) before the prototype gets deleted.
|
||||
Once the prototype has answered its question, capture the answer, then capture the prototype the way the [SKILL](SKILL.md) describes. The logic-specific mapping: the validated reducer / machine / function set lifts into the real module (the decision, absorbed); the TUI shell rides along to the throwaway branch that keeps the prototype as a primary source.
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
name: prototype
|
||||
description: Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
|
||||
disable-model-invocation: true
|
||||
description: Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
|
||||
---
|
||||
|
||||
# Prototype
|
||||
@@ -22,10 +21,6 @@ The two branches produce very different artifacts — getting this wrong wastes
|
||||
1. **Throwaway from day one, and clearly marked as such.** Locate the prototype code close to where it will actually be used (next to the module or page it's prototyping for) so context is obvious — but name it so a casual reader can see it's a prototype, not production. For throwaway UI routes, obey whatever routing convention the project already uses; don't invent a new top-level structure.
|
||||
2. **One command to run.** Whatever the project's existing task runner supports — `pnpm <name>`, `python <path>`, `bun <path>`, etc. The user must be able to start it without thinking.
|
||||
3. **No persistence by default.** State lives in memory. Persistence is the thing the prototype is _checking_, not something it should depend on. If the question explicitly involves a database, hit a scratch DB or a local file with a clear "PROTOTYPE — wipe me" name.
|
||||
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast and then delete it.
|
||||
4. **Skip the polish.** No tests, no error handling beyond what makes the prototype _runnable_, no abstractions. The point is to learn something fast.
|
||||
5. **Surface the state.** After every action (logic) or on every variant switch (UI), print or render the full relevant state so the user can see what changed.
|
||||
6. **Delete or absorb when done.** When the prototype has answered its question, either delete it or fold the validated decision into the real code — don't leave it rotting in the repo.
|
||||
|
||||
## When done
|
||||
|
||||
The _answer_ is the only thing worth keeping from a prototype. Capture it somewhere durable (commit message, ADR, issue, or a `NOTES.md` next to the prototype) along with the question it was answering. If the user is around, that capture is a quick conversation; if not, leave the placeholder so they (or you, on the next pass) can fill in the verdict before deleting the prototype.
|
||||
6. **Capture it when done.** Fold any validated decision into the real code, then capture the prototype itself as a **primary source**: commit it to a throwaway branch, out of main, and leave a context pointer to that branch on the implementation issue. Capture the answer too — the verdict and the question it settled — in the issue or a commit. The main branch keeps only the validated decision.
|
||||
|
||||
@@ -97,12 +97,12 @@ Surface the URL (and the `?variant=` keys). The user will flip through whenever
|
||||
|
||||
### 6. Capture the answer and clean up
|
||||
|
||||
Once a variant has won, write down which one and why (commit message, ADR, issue, or a `NOTES.md` next to the prototype if running AFK and the user hasn't responded yet). Then:
|
||||
Once a variant has won, capture the answer — which variant and why — then capture the prototype the way the [SKILL](SKILL.md) describes. Fold the winner into the real code and move the rest onto the throwaway branch, not into main:
|
||||
|
||||
- **Sub-shape A** — delete the losing variants and the switcher; fold the winner into the existing page.
|
||||
- **Sub-shape B** — promote the winning variant to a real route, delete the throwaway route and the switcher.
|
||||
- **Sub-shape A** — fold the winner into the existing page; drop the losing variants and the switcher from main.
|
||||
- **Sub-shape B** — promote the winning variant to a real route; drop the throwaway route and the switcher from main.
|
||||
|
||||
Don't leave variant components or the switcher lying around. They rot fast and confuse the next reader.
|
||||
The full set of variants is the primary source, so it lands on the throwaway branch, not the bin — variant components and the switcher left in the main branch rot fast and confuse the next reader.
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
|
||||
3
.agents/skills/prototype/agents/openai.yaml
Normal file
3
.agents/skills/prototype/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "Prototype"
|
||||
short_description: "Prototype to answer a design question"
|
||||
@@ -26,16 +26,18 @@ Look at the current repo to understand its starting state. Read whatever exists;
|
||||
- `docs/adr/` and any `src/*/docs/adr/` directories
|
||||
- `docs/agents/` — does this skill's prior output already exist?
|
||||
- `.scratch/` — sign that a local-markdown issue tracker convention is already in use
|
||||
- Is the `triage` skill installed? (a `triage` skill folder alongside this one, or `triage` in your available skills.) This decides whether Section B runs at all.
|
||||
- Monorepo signals — a `pnpm-workspace.yaml`, a `workspaces` field in `package.json`, or a populated `packages/*` with its own `src/`. Present only in a genuinely large multi-package repo; their absence means single-context, which is almost every repo.
|
||||
|
||||
### 2. Present findings and ask
|
||||
|
||||
Summarise what's present and what's missing. Then walk the user through the three decisions **one at a time** — present a section, get the user's answer, then move to the next. Don't dump all three at once.
|
||||
Summarise what's present and what's missing. Then take the sections in order — one section, one answer, then the next.
|
||||
|
||||
Assume the user does not know what these terms mean. Each section starts with a short explainer (what it is, why these skills need it, what changes if they pick differently). Then show the choices and the default.
|
||||
Lead each section with the recommended answer so the user can accept it in a word. Give a one-line explainer only when the choice genuinely branches; skip the section entirely when exploration already settled it (Section B when `triage` isn't installed, Section C when there's no monorepo).
|
||||
|
||||
**Section A — Issue tracker.**
|
||||
|
||||
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-issues`, `triage`, `to-prd`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
|
||||
> Explainer: The "issue tracker" is where issues live for this repo. Skills like `to-tickets`, `triage`, `to-spec`, and `qa` read from and write to it — they need to know whether to call `gh issue create`, write a markdown file under `.scratch/`, or follow some other workflow you describe. Pick the place you actually track work for this repo.
|
||||
|
||||
Default posture: these skills were designed for GitHub. If a `git remote` points at GitHub, propose that. If a `git remote` points at GitLab (`gitlab.com` or a self-hosted host), propose GitLab. Otherwise (or if the user prefers), offer:
|
||||
|
||||
@@ -44,41 +46,26 @@ Default posture: these skills were designed for GitHub. If a `git remote` points
|
||||
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
|
||||
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
|
||||
|
||||
If — and only if — the user picked **GitHub** or **GitLab**, ask one follow-up:
|
||||
Record the choice in `docs/agents/issue-tracker.md`. The GitHub and GitLab templates carry a "PRs as a request surface" flag, defaulted **off** — leave it off and don't raise it; a user who wants external PRs in the triage queue can flip the flag in the file later.
|
||||
|
||||
> Explainer: Open-source repos often receive feature requests as pull requests, not just issues — a PR is an issue with attached code. If you turn this on, `/triage` pulls *external* PRs into the same queue and runs them through the same labels and states as issues (collaborators' in-flight PRs are left alone). Leave it off if PRs aren't a request surface for you.
|
||||
**Section B — Triage label vocabulary.** Skip this section entirely if the `triage` skill isn't installed (exploration told you) — an uninstalled skill needs no labels.
|
||||
|
||||
- **PRs as a request surface** — yes / no (default: no). Record the answer in `docs/agents/issue-tracker.md`. For local-markdown and other trackers, skip this question — there are no PRs.
|
||||
If it is installed, ask exactly one question:
|
||||
|
||||
**Section B — Triage label vocabulary.**
|
||||
> Do you want to keep the default triage labels? (recommended: **yes**)
|
||||
|
||||
> Explainer: When the `triage` skill processes an incoming issue, it moves it through a state machine — needs evaluation, waiting on reporter, ready for an AFK agent to pick up, ready for a human, or won't fix. To do that, it needs to apply labels (or the equivalent in your issue tracker) that match strings *you've actually configured*. If your repo already uses different label names (e.g. `bug:triage` instead of `needs-triage`), map them here so the skill applies the right ones instead of creating duplicates.
|
||||
The defaults are the five canonical roles, each label string equal to its name: `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`. On **yes**, write them as-is. Only if the user says no — usually because their tracker already uses other names (e.g. `bug:triage` for `needs-triage`) — collect the overrides so `triage` applies existing labels instead of creating duplicates.
|
||||
|
||||
The five canonical roles:
|
||||
**Section C — Domain docs.** Default to **single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. This fits almost every repo; write it without asking.
|
||||
|
||||
- `needs-triage` — maintainer needs to evaluate
|
||||
- `needs-info` — waiting on reporter
|
||||
- `ready-for-agent` — fully specified, AFK-ready (an agent can pick it up with no human context)
|
||||
- `ready-for-human` — needs human implementation
|
||||
- `wontfix` — will not be actioned
|
||||
|
||||
Default: each role's string equals its name. Ask the user if they want to override any. If their issue tracker has no existing labels, the defaults are fine.
|
||||
|
||||
**Section C — Domain docs.**
|
||||
|
||||
> Explainer: Some skills (`improve-codebase-architecture`, `diagnosing-bugs`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
|
||||
|
||||
Confirm the layout:
|
||||
|
||||
- **Single-context** — one `CONTEXT.md` + `docs/adr/` at the repo root. Most repos are this.
|
||||
- **Multi-context** — `CONTEXT-MAP.md` at the root pointing to per-context `CONTEXT.md` files (typically a monorepo).
|
||||
Offer **multi-context** — a root `CONTEXT-MAP.md` pointing to per-context `CONTEXT.md` files — only when exploration found monorepo signals. Then confirm which layout they want.
|
||||
|
||||
### 3. Confirm and edit
|
||||
|
||||
Show the user a draft of:
|
||||
|
||||
- The `## Agent skills` block to add to whichever of `CLAUDE.md` / `AGENTS.md` is being edited (see step 4 for selection rules)
|
||||
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/triage-labels.md`, `docs/agents/domain.md`
|
||||
- The contents of `docs/agents/issue-tracker.md`, `docs/agents/domain.md`, and `docs/agents/triage-labels.md` (the last only when `triage` is installed)
|
||||
|
||||
Let them edit before writing.
|
||||
|
||||
@@ -101,7 +88,7 @@ The block:
|
||||
|
||||
### Issue tracker
|
||||
|
||||
[one-line summary of where issues are tracked, plus whether external PRs are a triage surface]. See `docs/agents/issue-tracker.md`.
|
||||
[one-line summary of where issues are tracked]. See `docs/agents/issue-tracker.md`.
|
||||
|
||||
### Triage labels
|
||||
|
||||
@@ -112,12 +99,14 @@ The block:
|
||||
[one-line summary of layout — "single-context" or "multi-context"]. See `docs/agents/domain.md`.
|
||||
```
|
||||
|
||||
Then write the three docs files using the seed templates in this skill folder as a starting point:
|
||||
Include the `### Triage labels` sub-block, and write `docs/agents/triage-labels.md`, only when `triage` is installed and Section B ran. When it isn't, both are omitted.
|
||||
|
||||
Then write the docs files using the seed templates in this skill folder as a starting point:
|
||||
|
||||
- [issue-tracker-github.md](./issue-tracker-github.md) — GitHub issue tracker
|
||||
- [issue-tracker-gitlab.md](./issue-tracker-gitlab.md) — GitLab issue tracker
|
||||
- [issue-tracker-local.md](./issue-tracker-local.md) — local-markdown issue tracker
|
||||
- [triage-labels.md](./triage-labels.md) — label mapping
|
||||
- [triage-labels.md](./triage-labels.md) — label mapping (only if `triage` is installed)
|
||||
- [domain.md](./domain.md) — domain doc consumer rules + layout
|
||||
|
||||
For "other" issue trackers, write `docs/agents/issue-tracker.md` from scratch using the user's description.
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Setup Matt Pocock Skills"
|
||||
short_description: "Configure a repo for the skills"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -32,3 +32,14 @@ Create a GitHub issue.
|
||||
## When a skill says "fetch the relevant ticket"
|
||||
|
||||
Run `gh issue view <number> --comments`.
|
||||
|
||||
## Wayfinding operations
|
||||
|
||||
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
|
||||
|
||||
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `gh issue create --label wayfinder:map`.
|
||||
- **Child ticket**: an issue linked to the map as a GitHub sub-issue (`gh api` on the sub-issues endpoint). Where sub-issues aren't enabled, add the child to a task list in the map body and put `Part of #<map>` at the top of the child body. Labels: `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
|
||||
- **Blocking**: GitHub's **native issue dependencies** — the canonical, UI-visible representation. Add an edge with `gh api --method POST repos/<owner>/<repo>/issues/<child>/dependencies/blocked_by -F issue_id=<blocker-db-id>`, where `<blocker-db-id>` is the blocker's numeric **database id** (`gh api repos/<owner>/<repo>/issues/<n> --jq .id`, _not_ the `#number` or `node_id`). GitHub reports `issue_dependencies_summary.blocked_by` (open blockers only — the live gate). Where dependencies aren't available, fall back to a `Blocked by: #<n>, #<n>` line at the top of the child body. A ticket is unblocked when every blocker is closed.
|
||||
- **Frontier query**: list the map's open children (`gh issue list --state open`, scoped to the map's sub-issues / task list), drop any with an open blocker (`issue_dependencies_summary.blocked_by > 0`, or an open issue in the `Blocked by` line) or an assignee; first in map order wins.
|
||||
- **Claim**: `gh issue edit <n> --add-assignee @me` — the session's first write.
|
||||
- **Resolve**: `gh issue comment <n> --body "<answer>"`, then `gh issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.
|
||||
|
||||
@@ -33,3 +33,14 @@ Create a GitLab issue.
|
||||
## When a skill says "fetch the relevant ticket"
|
||||
|
||||
Run `glab issue view <number> --comments`.
|
||||
|
||||
## Wayfinding operations
|
||||
|
||||
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
|
||||
|
||||
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `glab issue create --label wayfinder:map`. (On GitLab tiers with native epics, an epic may hold the map instead; a labelled issue works everywhere.)
|
||||
- **Child ticket**: an issue carrying `Part of #<map>` at the top of its description and labels `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
|
||||
- **Blocking**: GitLab's **native blocking link** — the canonical, UI-visible representation. Add it with the `/blocked_by #<n>` quick action, posted as a note (`glab issue note <child> --message "/blocked_by #<blocker>"`). Native blocking links are a Premium/Ultimate feature; on the free tier (or where unavailable) fall back to a `Blocked by: #<n>, #<n>` line at the top of the description. A ticket is unblocked when every blocker is closed.
|
||||
- **Frontier query**: `glab issue list -F json` scoped to the map's children, drop any with an open blocker — a native `blocked_by` link to an open issue (`glab api projects/:id/issues/:iid/links`), or an open issue in the `Blocked by` line — or an assignee; first in map order wins.
|
||||
- **Claim**: `glab issue update <n> --assignee @me` — the session's first write.
|
||||
- **Resolve**: `glab issue note <n> --message "<answer>"`, then `glab issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Issue tracker: Local Markdown
|
||||
|
||||
Issues and PRDs for this repo live as markdown files in `.scratch/`.
|
||||
Issues and specs (you may know a spec as a PRD) for this repo live as markdown files in `.scratch/`.
|
||||
|
||||
## Conventions
|
||||
|
||||
- One feature per directory: `.scratch/<feature-slug>/`
|
||||
- The PRD is `.scratch/<feature-slug>/PRD.md`
|
||||
- Implementation issues are `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01`
|
||||
- The spec is `.scratch/<feature-slug>/spec.md`
|
||||
- Implementation issues are one file per ticket at `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` — never a single combined tickets file
|
||||
- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings)
|
||||
- Comments and conversation history append to the bottom of the file under a `## Comments` heading
|
||||
|
||||
@@ -17,3 +17,14 @@ Create a new file under `.scratch/<feature-slug>/` (creating the directory if ne
|
||||
## When a skill says "fetch the relevant ticket"
|
||||
|
||||
Read the file at the referenced path. The user will normally pass the path or the issue number directly.
|
||||
|
||||
## Wayfinding operations
|
||||
|
||||
Used by `/wayfinder`. The **map** is a file with one **child** file per ticket.
|
||||
|
||||
- **Map**: `.scratch/<effort>/map.md` — the Notes / Decisions-so-far / Fog body.
|
||||
- **Child ticket**: `.scratch/<effort>/issues/NN-<slug>.md`, numbered from `01`, with the question in the body. A `Type:` line records the ticket type (`research`/`prototype`/`grilling`/`task`); a `Status:` line records `claimed`/`resolved`.
|
||||
- **Blocking**: a `Blocked by: NN, NN` line near the top. A ticket is unblocked when every file it lists is `resolved`.
|
||||
- **Frontier**: scan `.scratch/<effort>/issues/` for files that are open, unblocked, and unclaimed; first by number wins.
|
||||
- **Claim**: set `Status: claimed` and save before any work.
|
||||
- **Resolve**: append the answer under an `## Answer` heading, set `Status: resolved`, then append a context pointer (gist + link) to the map's Decisions-so-far in `map.md`.
|
||||
|
||||
102
.agents/skills/setup-ts-deep-modules/SKILL.md
Normal file
102
.agents/skills/setup-ts-deep-modules/SKILL.md
Normal file
@@ -0,0 +1,102 @@
|
||||
---
|
||||
name: setup-ts-deep-modules
|
||||
description: Wire dependency-cruiser into a TypeScript repo so each package is a deep module — implementation hidden in subfolders, reachable only through its entry-point files. User-invoked.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
# Setup TS Deep Modules
|
||||
|
||||
Make every package in this repo a **deep module**: a lot of behaviour behind a small interface. A package's public surface is its **entry points** — the files at the package root — and everything in its subfolders is hidden. This skill installs [dependency-cruiser](https://github.com/sverweij/dependency-cruiser) and the rules that make the entry points the only way in, then proves the rules bite.
|
||||
|
||||
For the vocabulary (deep module, interface, seam, depth), run the `/codebase-design` skill — use its language throughout.
|
||||
|
||||
## The shape this enforces
|
||||
|
||||
```
|
||||
src/packages/
|
||||
<name>/
|
||||
index.ts ← an entry point (public). Import this from outside.
|
||||
client.ts ← another entry point. Packages may expose SEVERAL.
|
||||
lib/ ← implementation: hidden from outside, free to import each other.
|
||||
tests/ ← co-located tests + fixtures (a subfolder, so private).
|
||||
```
|
||||
|
||||
The public surface is the package's **root files** — not one designated `index.ts`. By convention implementation lives in `lib/` and tests in `tests/`, giving every package the same two-folder shape. The rule itself is general, though: *anything* in *any* subfolder is private, so you never extend the config to add a folder.
|
||||
|
||||
Four rules, all `error`:
|
||||
|
||||
1. **Entry-point boundary** — code outside a package (app code or another package) may import only that package's entry points (its root files), never anything in its subfolders.
|
||||
2. **Intra-package freedom** — a package's own files import each other freely.
|
||||
3. **Tests through the entry points** — files under `<pkg>/tests/` may import any package's entry points and their own `tests/` fixtures, but never any package's subfolder internals (not even their own). Integration tests across packages are fine; deep imports are not.
|
||||
4. **No cycles** — no dependency cycles.
|
||||
|
||||
**Entry points, not a barrel.** Because the public surface is *every* root file, a package can expose several small entry points (`index.ts`, `client.ts`, `server.ts`) instead of funnelling everything through one giant `index.ts`. Barrel files that re-export a whole subtree are discouraged — keep entry points small and hide implementation in subfolders.
|
||||
|
||||
Layering (which packages may depend on which) is a *different* concern and is left as a commented stub in the config for this repo to fill in.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Detect the environment
|
||||
|
||||
- **Package manager** — `pnpm-lock.yaml` → pnpm, `yarn.lock` → yarn, `bun.lockb` → bun, else npm. Use it for every command below (`pnpm`/`yarn`/`npm run`/`bunx`).
|
||||
- **Packages root** — if `src/` exists use `src/packages`, else `packages`. Confirm the choice with the user if the repo already has a different obvious convention.
|
||||
- **Existing config** — check for a `.dependency-cruiser.*` file. If one exists, do **not** overwrite it: merge the four rules and the options in, and tell the user what you added.
|
||||
|
||||
**Done when:** package manager, packages root, and existing-config status are all known.
|
||||
|
||||
### 2. Install dependency-cruiser
|
||||
|
||||
Install `dependency-cruiser` as a devDependency with the detected package manager.
|
||||
|
||||
**Done when:** `dependency-cruiser` is in `devDependencies`.
|
||||
|
||||
### 3. Write the config
|
||||
|
||||
Copy [`dependency-cruiser.config.cjs`](./dependency-cruiser.config.cjs) to the repo root as `.dependency-cruiser.cjs`. Set `PACKAGES_ROOT` to the root detected in step 1. The rules are path-depth based and extension-agnostic, so nothing else needs adapting.
|
||||
|
||||
**Done when:** `.dependency-cruiser.cjs` exists with the correct `PACKAGES_ROOT`, and the four forbidden rules are present.
|
||||
|
||||
### 4. Wire it into the checks
|
||||
|
||||
- Add a `lint:boundaries` script: `depcruise <packages-root>` (or `depcruise src`).
|
||||
- Fold it into the repo's umbrella check command — the one that already runs typecheck (e.g. a `check` / `ci` / `validate` script). Do **not** touch `tsconfig` or add path aliases.
|
||||
- If there is no umbrella script, add `lint:boundaries` and tell the user to include it in CI.
|
||||
|
||||
**Done when:** `lint:boundaries` exists and runs as part of the same command as typecheck.
|
||||
|
||||
### 5. Scaffold the example package
|
||||
|
||||
Create a committed `<packages-root>/example/` as a copy-me template:
|
||||
|
||||
- `index.ts` — an entry point. Export one function that delegates to an internal file (so the package is visibly *deep*, not a pass-through).
|
||||
- `lib/impl.ts` — an internal file in a **subfolder**, imported by `index.ts`, not reachable from outside.
|
||||
- `tests/example.test.ts` — imports **only** `../index` (an entry point), and asserts against the public function.
|
||||
|
||||
Tell the user this is a starter template to copy or delete.
|
||||
|
||||
**Done when:** the example package exists, exposes its behaviour through a root entry point, and hides `impl` in a subfolder.
|
||||
|
||||
### 6. Prove the rules bite
|
||||
|
||||
This is the completion criterion for the whole skill — a config that doesn't fail on a violation is worthless.
|
||||
|
||||
1. Run `lint:boundaries`. It must **pass** on the clean example.
|
||||
2. Temporarily add a deep import to `tests/example.test.ts` (e.g. `import { thing } from "../lib/impl"`). Run `lint:boundaries` again — it must **fail** with `tests-through-entrypoints`.
|
||||
3. Revert the deep import. Run once more — it must **pass**.
|
||||
|
||||
**Done when:** you have observed a pass, then a fail on the deep import, then a pass again. If step 2 does not fail, the rules are not wired correctly — fix before finishing.
|
||||
|
||||
### 7. Document the convention
|
||||
|
||||
Write a `README.md` **in the packages folder** (`<packages-root>/README.md`) — next to the packages it governs — covering: the `src/packages/<name>/` layout (entry points at the root, `lib/` for implementation, `tests/` for tests), "import only through a package's entry points (its root files)", and how to run `lint:boundaries`. **Discourage barrel files** explicitly — expose several small entry points instead of re-exporting a whole subtree through one index. Keep it to the copy-me snippet plus the four rules in one paragraph each.
|
||||
|
||||
Then add a **context pointer** to it from the repo's agent-instructions file — `CLAUDE.md` if present, else `AGENTS.md` (create `AGENTS.md` if neither exists). One line is enough, e.g. `Packages are deep modules — see [src/packages/README.md](./src/packages/README.md) before adding or importing one.` This is what makes an agent discover the boundary rule instead of tripping over it.
|
||||
|
||||
**Done when:** `<packages-root>/README.md` exists and discourages barrels, and the repo's `CLAUDE.md`/`AGENTS.md` links to it.
|
||||
|
||||
## Notes
|
||||
|
||||
- The config's `$1` back-references (dependency-cruiser's group matching) are what let a package reach its own internals while outsiders can't — don't flatten them into separate per-package rules.
|
||||
- Public vs private is decided by **depth**: a package's root files are entry points; anything in a subfolder is private. The conventional subfolders are `lib/` (implementation) and `tests/`, but the rule doesn't hardcode them — any subfolder is private, so a new folder never needs a config change. Adding an entry point is just adding a root file — no barrel.
|
||||
- Packages are **flat**: one tier of immediate children under the root. A package's internals may nest as deep as you like; a package may not contain another package.
|
||||
- Use `.cjs` (not `.js`) so the config's `module.exports` works even in `"type": "module"` repos.
|
||||
5
.agents/skills/setup-ts-deep-modules/agents/openai.yaml
Normal file
5
.agents/skills/setup-ts-deep-modules/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Setup TS Deep Modules"
|
||||
short_description: "Enforce deep TypeScript modules"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -0,0 +1,95 @@
|
||||
// @ts-check
|
||||
// Deep-module enforcement for dependency-cruiser.
|
||||
//
|
||||
// Each package under the packages root is a DEEP MODULE: a lot of behaviour
|
||||
// behind a small interface. A package's PUBLIC SURFACE is its ENTRY POINTS —
|
||||
// the files at the package root. Implementation lives in SUBFOLDERS and is
|
||||
// private — by convention `lib/` for implementation and `tests/` for tests,
|
||||
// though any subfolder is private. A package may expose several small entry
|
||||
// points (index.ts, client.ts, server.ts, …) — prefer that over one giant
|
||||
// barrel index.
|
||||
//
|
||||
// The only thing you should ever need to edit here is PACKAGES_ROOT.
|
||||
|
||||
/** Where packages live. One immediate child dir per package (flat, no nesting). */
|
||||
const PACKAGES_ROOT = "src/packages";
|
||||
|
||||
// --- derived patterns (no need to edit) -------------------------------------
|
||||
const R = PACKAGES_ROOT;
|
||||
/**
|
||||
* A package's private internals: anything nested inside a package subfolder.
|
||||
* The package's root files are its entry points and are NOT matched here —
|
||||
* they stay importable from outside.
|
||||
*/
|
||||
const PACKAGE_INTERNALS = `^${R}/[^/]+/[^/]+/`;
|
||||
|
||||
/** @type {import('dependency-cruiser').IConfiguration} */
|
||||
module.exports = {
|
||||
forbidden: [
|
||||
{
|
||||
name: "entrypoint-boundary-from-app",
|
||||
comment:
|
||||
"App/root code may import a package's entry points (its root files), but nothing inside its subfolders.",
|
||||
severity: "error",
|
||||
from: { pathNot: `^${R}/` }, // importer is NOT inside any package
|
||||
to: { path: PACKAGE_INTERNALS },
|
||||
},
|
||||
{
|
||||
name: "entrypoint-boundary-across-packages",
|
||||
comment:
|
||||
"A package's own files import each other freely, but may reach OTHER packages only through their entry points — never their internals.",
|
||||
severity: "error",
|
||||
// importer is inside a package ($1), but is not a test file
|
||||
from: { path: `^${R}/([^/]+)/`, pathNot: `^${R}/[^/]+/tests/` },
|
||||
to: {
|
||||
path: PACKAGE_INTERNALS,
|
||||
pathNot: `^${R}/$1/`, // same package → intra-package freedom
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "tests-through-entrypoints",
|
||||
comment:
|
||||
"A package's tests exercise it through its entry points like everyone else: they may import any package's entry points and their own tests/ fixtures, but never any package's internals — not even their own.",
|
||||
severity: "error",
|
||||
from: { path: `^${R}/([^/]+)/tests/` }, // a test file, in package $1
|
||||
to: {
|
||||
path: PACKAGE_INTERNALS,
|
||||
pathNot: `^${R}/$1/tests/`, // own tests/ fixtures → allowed
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "tests-folder-is-private",
|
||||
comment:
|
||||
"A package's tests/ folder is reachable only from tests — nothing else may import fixtures.",
|
||||
severity: "error",
|
||||
from: { pathNot: `^${R}/[^/]+/tests/` }, // importer is not itself a test
|
||||
to: { path: `^${R}/[^/]+/tests/` },
|
||||
},
|
||||
{
|
||||
name: "no-circular",
|
||||
comment: "No dependency cycles. Scope to `^${R}/` if you want to allow cycles outside packages.",
|
||||
severity: "error",
|
||||
from: {},
|
||||
to: { circular: true },
|
||||
},
|
||||
|
||||
// --- Layering (optional, off by default) ----------------------------------
|
||||
// Interface-hiding controls HOW you import (through the entry points).
|
||||
// Layering controls WHICH packages may depend on which. Add your own rules
|
||||
// here, e.g.:
|
||||
//
|
||||
// {
|
||||
// name: "ui-may-not-depend-on-billing",
|
||||
// severity: "error",
|
||||
// from: { path: `^${R}/ui/` },
|
||||
// to: { path: `^${R}/billing/` },
|
||||
// },
|
||||
],
|
||||
options: {
|
||||
doNotFollow: { path: "node_modules" },
|
||||
tsConfig: { fileName: "tsconfig.json" },
|
||||
enhancedResolveOptions: {
|
||||
extensions: [".ts", ".tsx", ".js", ".jsx", ".json"],
|
||||
},
|
||||
},
|
||||
};
|
||||
@@ -5,104 +5,32 @@ description: Test-driven development. Use when the user wants to build features
|
||||
|
||||
# Test-Driven Development
|
||||
|
||||
## Philosophy
|
||||
TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle — consult them before and during the loop, not after.
|
||||
|
||||
**Core principle**: Tests should verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't.
|
||||
When exploring the codebase, read `CONTEXT.md` (if it exists) so test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
|
||||
|
||||
**Good tests** are integration-style: they exercise real code paths through public APIs. They describe _what_ the system does, not _how_ it does it. A good test reads like a specification - "user can checkout with valid cart" tells you exactly what capability exists. These tests survive refactors because they don't care about internal structure.
|
||||
## What a good test is
|
||||
|
||||
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
|
||||
Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification — "user can checkout with valid cart" tells you exactly what capability exists — and survives refactors because it doesn't care about internal structure.
|
||||
|
||||
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
|
||||
|
||||
## Anti-Pattern: Horizontal Slices
|
||||
## Seams — where tests go
|
||||
|
||||
**DO NOT write all tests first, then all implementation.** This is "horizontal slicing" - treating RED as "write all tests" and GREEN as "write all code."
|
||||
A **seam** is the public boundary you test at: the interface where you observe behavior without reaching inside. Tests live at seams, never against internals.
|
||||
|
||||
This produces **crap tests**:
|
||||
**Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. No test is written at an unconfirmed seam. You can't test everything — agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case.
|
||||
|
||||
- Tests written in bulk test _imagined_ behavior, not _actual_ behavior
|
||||
- You end up testing the _shape_ of things (data structures, function signatures) rather than user-facing behavior
|
||||
- Tests become insensitive to real changes - they pass when behavior breaks, fail when behavior is fine
|
||||
- You outrun your headlights, committing to test structure before understanding the implementation
|
||||
Ask: "What's the public interface, and which seams should we test?"
|
||||
|
||||
**Correct approach**: Vertical slices via tracer bullets. One test → one implementation → repeat. Each test responds to what you learned from the previous cycle. Because you just wrote the code, you know exactly what behavior matters and how to verify it.
|
||||
## Anti-patterns
|
||||
|
||||
```
|
||||
WRONG (horizontal):
|
||||
RED: test1, test2, test3, test4, test5
|
||||
GREEN: impl1, impl2, impl3, impl4, impl5
|
||||
- **Implementation-coupled** — mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
|
||||
- **Tautological** — the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth — a known-good literal, a worked example, the spec.
|
||||
- **Horizontal slicing** — writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead — one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.
|
||||
|
||||
RIGHT (vertical):
|
||||
RED→GREEN: test1→impl1
|
||||
RED→GREEN: test2→impl2
|
||||
RED→GREEN: test3→impl3
|
||||
...
|
||||
```
|
||||
## Rules of the loop
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Planning
|
||||
|
||||
When exploring the codebase, read `CONTEXT.md` (if it exists) so that test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
|
||||
|
||||
Before writing any code:
|
||||
|
||||
- [ ] Confirm with user what interface changes are needed
|
||||
- [ ] Confirm with user which behaviors to test (prioritize)
|
||||
- [ ] Identify opportunities for deep modules (small interface, deep implementation) — run the `/codebase-design` skill for the vocabulary and the testability checks
|
||||
- [ ] List the behaviors to test (not implementation steps)
|
||||
- [ ] Get user approval on the plan
|
||||
|
||||
Ask: "What should the public interface look like? Which behaviors are most important to test?"
|
||||
|
||||
**You can't test everything.** Confirm with the user exactly which behaviors matter most. Focus testing effort on critical paths and complex logic, not every possible edge case.
|
||||
|
||||
### 2. Tracer Bullet
|
||||
|
||||
Write ONE test that confirms ONE thing about the system:
|
||||
|
||||
```
|
||||
RED: Write test for first behavior → test fails
|
||||
GREEN: Write minimal code to pass → test passes
|
||||
```
|
||||
|
||||
This is your tracer bullet - proves the path works end-to-end.
|
||||
|
||||
### 3. Incremental Loop
|
||||
|
||||
For each remaining behavior:
|
||||
|
||||
```
|
||||
RED: Write next test → fails
|
||||
GREEN: Minimal code to pass → passes
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- One test at a time
|
||||
- Only enough code to pass current test
|
||||
- Don't anticipate future tests
|
||||
- Keep tests focused on observable behavior
|
||||
|
||||
### 4. Refactor
|
||||
|
||||
After all tests pass, look for [refactor candidates](refactoring.md):
|
||||
|
||||
- [ ] Extract duplication
|
||||
- [ ] Deepen modules (move complexity behind simple interfaces)
|
||||
- [ ] Apply SOLID principles where natural
|
||||
- [ ] Consider what new code reveals about existing code
|
||||
- [ ] Run tests after each refactor step
|
||||
|
||||
**Never refactor while RED.** Get to GREEN first.
|
||||
|
||||
## Checklist Per Cycle
|
||||
|
||||
```
|
||||
[ ] Test describes behavior, not implementation
|
||||
[ ] Test uses public interface only
|
||||
[ ] Test would survive internal refactor
|
||||
[ ] Code is minimal for this test
|
||||
[ ] No speculative features added
|
||||
```
|
||||
- **Red before green.** Write the failing test first, then only enough code to pass it. Don't anticipate future tests or add speculative features.
|
||||
- **One slice at a time.** One seam, one test, one minimal implementation per cycle.
|
||||
- **Refactoring is not part of the loop.** It belongs to the review stage (see the `code-review` skill), not the red → green implementation cycle.
|
||||
|
||||
3
.agents/skills/tdd/agents/openai.yaml
Normal file
3
.agents/skills/tdd/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "TDD"
|
||||
short_description: "Test-driven red-green-refactor"
|
||||
@@ -59,3 +59,19 @@ test("createUser makes user retrievable", async () => {
|
||||
expect(retrieved.name).toBe("Alice");
|
||||
});
|
||||
```
|
||||
|
||||
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
|
||||
|
||||
```typescript
|
||||
// BAD: Expected value is recomputed the way the code computes it
|
||||
test("calculateTotal sums line items", () => {
|
||||
const items = [{ price: 10 }, { price: 5 }];
|
||||
const expected = items.reduce((sum, i) => sum + i.price, 0);
|
||||
expect(calculateTotal(items)).toBe(expected);
|
||||
});
|
||||
|
||||
// GOOD: Expected value is an independent, known literal
|
||||
test("calculateTotal sums line items", () => {
|
||||
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
|
||||
});
|
||||
```
|
||||
|
||||
75
.agents/skills/to-spec/SKILL.md
Normal file
75
.agents/skills/to-spec/SKILL.md
Normal file
@@ -0,0 +1,75 @@
|
||||
---
|
||||
name: to-spec
|
||||
description: Turn the current conversation into a spec and publish it to the project issue tracker — no interview, just synthesis of what you've already discussed.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
This skill takes the current conversation context and codebase understanding and produces a spec (you may know this document as a PRD). Do NOT interview the user — just synthesize what you already know.
|
||||
|
||||
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
|
||||
|
||||
## Process
|
||||
|
||||
1. Explore the repo to understand the current state of the codebase, if you haven't already. Use the project's domain glossary vocabulary throughout the spec, and respect any ADRs in the area you're touching.
|
||||
|
||||
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can. The fewer seams across the codebase, the better - the ideal number is one.
|
||||
|
||||
Check with the user that these seams match their expectations.
|
||||
|
||||
3. Write the spec using the template below, then publish it to the project issue tracker. Apply the `ready-for-agent` triage label - no need for additional triage.
|
||||
|
||||
<spec-template>
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The problem that the user is facing, from the user's perspective.
|
||||
|
||||
## Solution
|
||||
|
||||
The solution to the problem, from the user's perspective.
|
||||
|
||||
## User Stories
|
||||
|
||||
A LONG, numbered list of user stories. Each user story should be in the format of:
|
||||
|
||||
1. As an <actor>, I want a <feature>, so that <benefit>
|
||||
|
||||
<user-story-example>
|
||||
1. As a mobile bank customer, I want to see balance on my accounts, so that I can make better informed decisions about my spending
|
||||
</user-story-example>
|
||||
|
||||
This list of user stories should be extremely extensive and cover all aspects of the feature.
|
||||
|
||||
## Implementation Decisions
|
||||
|
||||
A list of implementation decisions that were made. This can include:
|
||||
|
||||
- The modules that will be built/modified
|
||||
- The interfaces of those modules that will be modified
|
||||
- Technical clarifications from the developer
|
||||
- Architectural decisions
|
||||
- Schema changes
|
||||
- API contracts
|
||||
- Specific interactions
|
||||
|
||||
Do NOT include specific file paths or code snippets. They may end up being outdated very quickly.
|
||||
|
||||
Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it within the relevant decision and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
|
||||
|
||||
## Testing Decisions
|
||||
|
||||
A list of testing decisions that were made. Include:
|
||||
|
||||
- A description of what makes a good test (only test external behavior, not implementation details)
|
||||
- Which modules will be tested
|
||||
- Prior art for the tests (i.e. similar types of tests in the codebase)
|
||||
|
||||
## Out of Scope
|
||||
|
||||
A description of the things that are out of scope for this spec.
|
||||
|
||||
## Further Notes
|
||||
|
||||
Any further notes about the feature.
|
||||
|
||||
</spec-template>
|
||||
5
.agents/skills/to-spec/agents/openai.yaml
Normal file
5
.agents/skills/to-spec/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "To Spec"
|
||||
short_description: "Turn a conversation into a spec"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
107
.agents/skills/to-tickets/SKILL.md
Normal file
107
.agents/skills/to-tickets/SKILL.md
Normal file
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: to-tickets
|
||||
description: Break a plan, spec, or the current conversation into a set of tracer-bullet tickets, each declaring its blocking edges, published to the configured tracker — edges as text in one file per ticket locally, or native blocking links on a real tracker.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
# To Tickets
|
||||
|
||||
Break a plan, spec, or conversation into a set of **tickets** — tracer-bullet vertical slices, each declaring the tickets that **block** it.
|
||||
|
||||
The issue tracker and triage label vocabulary should have been provided to you — run `/setup-matt-pocock-skills` if not.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Gather context
|
||||
|
||||
Work from whatever is already in the conversation context. If the user passes a reference (a spec path, an issue number or URL) as an argument, fetch it and read its full body and comments.
|
||||
|
||||
### 2. Explore the codebase (optional)
|
||||
|
||||
If you have not already explored the codebase, do so to understand the current state of the code. Ticket titles and descriptions should use the project's domain glossary vocabulary, and respect ADRs in the area you're touching.
|
||||
|
||||
Look for opportunities to prefactor the code to make the implementation easier. "Make the change easy, then make the easy change."
|
||||
|
||||
### 3. Draft vertical slices
|
||||
|
||||
Break the work into **tracer bullet** tickets.
|
||||
|
||||
<vertical-slice-rules>
|
||||
|
||||
- Each slice cuts a narrow but COMPLETE path through every layer (schema, API, UI, tests) — vertical, NOT a horizontal slice of one layer
|
||||
- A completed slice is demoable or verifiable on its own
|
||||
- Each slice is sized to fit in a single fresh context window
|
||||
- Any prefactoring should be done first
|
||||
|
||||
</vertical-slice-rules>
|
||||
|
||||
Give each ticket its **blocking edges** — the other tickets that must complete before it can start. A ticket with no blockers can start immediately.
|
||||
|
||||
**Wide refactors are the exception to vertical slicing.** A **wide refactor** is one mechanical change — rename a column, retype a shared symbol — whose **blast radius** fans across the whole codebase, so a single edit breaks thousands of call sites at once and no vertical slice can land green. Don't force it into a tracer bullet; sequence it as **expand–contract**. First expand: add the new form beside the old so nothing breaks. Then migrate the call sites over in batches sized by blast radius (per package, per directory), each batch its own ticket blocked by the expand, keeping CI green batch to batch because the old form still exists. Finally contract: delete the old form once no caller remains, in a ticket blocked by every migrate batch. When even the batches can't stay green alone, keep the sequence but let them share an integration branch that all block a final integrate-and-verify ticket — green is promised only there.
|
||||
|
||||
### 4. Quiz the user
|
||||
|
||||
Present the proposed breakdown as a numbered list. For each ticket, show:
|
||||
|
||||
- **Title**: short descriptive name
|
||||
- **Blocked by**: which other tickets (if any) must complete first
|
||||
- **What it delivers**: the end-to-end behaviour this ticket makes work
|
||||
|
||||
Ask the user:
|
||||
|
||||
- Does the granularity feel right? (too coarse / too fine)
|
||||
- Are the blocking edges correct — does each ticket only depend on tickets that genuinely gate it?
|
||||
- Should any tickets be merged or split further?
|
||||
|
||||
Iterate until the user approves the breakdown.
|
||||
|
||||
### 5. Publish the tickets to the configured tracker
|
||||
|
||||
Publish the approved tickets. **How** depends on the tracker `/setup-matt-pocock-skills` configured — the tickets are the same either way, only the shape of the blocking edges changes:
|
||||
|
||||
- **Local files** → write one file per ticket under `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01` in dependency order (blockers first). Each file's "Blocked by" lists the numbers/titles it depends on. Use the per-ticket file template below — one ticket per file, never a single combined file.
|
||||
- **A real issue tracker (GitHub, Linear, …)** → publish one issue per ticket in dependency order (blockers first) so each ticket's blocking edges can reference real identifiers. Use the platform's native blocking / sub-issue relationship where it has one; otherwise set each ticket's "Blocked by" to the blocking issues. Apply the `ready-for-agent` triage label unless instructed otherwise — the tickets are agent-grabbable by construction.
|
||||
|
||||
Work the **frontier**: any ticket whose blockers are all done. For a purely linear chain that means top to bottom.
|
||||
|
||||
Do NOT close or modify any parent issue.
|
||||
|
||||
<local-ticket-template>
|
||||
|
||||
# <NN> — <Ticket title>
|
||||
|
||||
**What to build:** the end-to-end behaviour this ticket makes work, from the user's perspective — not a layer-by-layer implementation list.
|
||||
|
||||
**Blocked by:** the numbers/titles of the tickets that gate this one, or "None — can start immediately".
|
||||
|
||||
**Status:** ready-for-agent
|
||||
|
||||
- [ ] Acceptance criterion 1
|
||||
- [ ] Acceptance criterion 2
|
||||
|
||||
</local-ticket-template>
|
||||
|
||||
<issue-template>
|
||||
|
||||
## Parent
|
||||
|
||||
A reference to the parent issue on the tracker (if the source was an existing issue, otherwise omit this section).
|
||||
|
||||
## What to build
|
||||
|
||||
The end-to-end behaviour this ticket makes work, from the user's perspective — not layer-by-layer implementation.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Criterion 1
|
||||
- [ ] Criterion 2
|
||||
|
||||
## Blocked by
|
||||
|
||||
- A reference to each blocking ticket, or "None — can start immediately".
|
||||
|
||||
</issue-template>
|
||||
|
||||
In either form, avoid specific file paths or code snippets — they go stale fast. Exception: if a prototype produced a snippet that encodes a decision more precisely than prose can (state machine, reducer, schema, type shape), inline it and note briefly that it came from a prototype. Trim to the decision-rich parts — not a working demo, just the important bits.
|
||||
|
||||
Work the frontier one ticket at a time with `/implement`, clearing context between tickets.
|
||||
5
.agents/skills/to-tickets/agents/openai.yaml
Normal file
5
.agents/skills/to-tickets/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "To Tickets"
|
||||
short_description: "Split a plan into tracer-bullet tickets"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -1,9 +1,16 @@
|
||||
---
|
||||
name: wayfinder
|
||||
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of investigation tickets on your issue tracker, and resolve them one at a time until the way to the goal is clear.
|
||||
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of decision tickets on your issue tracker, and resolve them one at a time until the way to the destination is clear.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
A loose idea has arrived — too big for one agent session, and wrapped in fog: the route from here to a plan isn't visible yet. This skill charts it as a **shared map** on the repo's issue tracker, then works its tickets one at a time. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
|
||||
A loose idea has arrived — too big for one agent session, and wrapped in fog: the way from here to the **destination** isn't visible yet. Wayfinding is about finding that way, not charging at the destination. This skill charts the way as a **shared map** on the repo's issue tracker, then works its **decision tickets** — questions whose resolution is a decision, not slices of a build to execute — one at a time until the route is clear.
|
||||
|
||||
The destination varies per effort, and naming it is the first act of charting — it shapes every ticket. It might be a spec to hand off and iterate on, a decision to lock before planning starts, or a change made in place like a data-structure migration. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
|
||||
|
||||
## Plan, don't do
|
||||
|
||||
Wayfinder is **planning** by default: each ticket resolves a decision, and the map is done when the way is clear — nothing left to decide before someone goes and does the thing. The pull to just do the work is usually the signal you've reached the edge of the map and it's time to hand off. An effort can override this in its **Notes** — carrying execution into the map itself — but absent that, produce decisions, not deliverables.
|
||||
|
||||
## Refer by name
|
||||
|
||||
@@ -15,13 +22,17 @@ The map is a single issue on this repo's issue tracker, labelled `wayfinder:map`
|
||||
|
||||
The map is an **index**, not a store. It lists the decisions made and points at the tickets that hold their detail; a decision lives in exactly one place — its ticket — so the map never restates it, only gists it and links.
|
||||
|
||||
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** Consult `docs/agents/issue-tracker.md` (the "Wayfinding operations" section) for how _this_ repo expresses them. If that doc is absent, default to the local-markdown tracker.
|
||||
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** The issue tracker should have been provided to you — run `/setup-matt-pocock-skills` if not. Consult the tracker doc's "Wayfinding operations" section for how _this_ repo expresses them. If no tracker has been provided, default to the local-markdown tracker.
|
||||
|
||||
### The map body
|
||||
|
||||
The whole map at low resolution, loaded once per session. Open tickets are **not** listed — they are open child issues, found by query.
|
||||
|
||||
```markdown
|
||||
## Destination
|
||||
|
||||
<what reaching the end of this map looks like — the spec, decision, or change this effort is finding its way to. One or two lines; every session orients to it before choosing a ticket.>
|
||||
|
||||
## Notes
|
||||
|
||||
<domain; skills every session should consult; standing preferences for this effort>
|
||||
@@ -32,9 +43,13 @@ The whole map at low resolution, loaded once per session. Open tickets are **not
|
||||
|
||||
- [<closed ticket title>](link) — <one-line gist of the answer>
|
||||
|
||||
## Fog
|
||||
## Not yet specified
|
||||
|
||||
<!-- see "Fog of war" for what belongs here -->
|
||||
<!-- see "Fog of war": in-scope fog you can't ticket yet; graduates as the frontier advances -->
|
||||
|
||||
## Out of scope
|
||||
|
||||
<!-- see "Out of scope": work ruled beyond the destination; closed, never graduates -->
|
||||
```
|
||||
|
||||
### Tickets
|
||||
@@ -57,36 +72,48 @@ The answer isn't part of the body — it's recorded on resolution (see [Work thr
|
||||
|
||||
## Ticket Types
|
||||
|
||||
- **Research**: Reading documentation, third-party APIs, or local resources like knowledge bases. Creates a markdown summary as a linked asset. Use when knowledge outside the current working directory is required.
|
||||
- **Prototype**: Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
|
||||
- **Grilling**: Conversation with the agent. Uses the /grilling and /domain-modeling skills. Asks one question at a time. The default case.
|
||||
- **Task**: Literal manual work that must be done before the discussion can move forward — nothing to decide, prototype, or research. Moving data, signing up for a service, provisioning access. The agent automates it where it can; otherwise it hands the human a precise checklist. Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
|
||||
Every ticket is either **HITL** — human in the loop, worked *with* a human who speaks for themselves — or **AFK**, driven by the agent alone. A HITL ticket only resolves through that live exchange; the agent never stands in for the human's side of it (a grilling agent that answers its own questions has broken this).
|
||||
|
||||
- **Research** (AFK): Reading documentation, third-party APIs, or local resources like knowledge bases to surface a fact a decision waits on. Resolved by a `/research` **subagent**. Use when knowledge outside the current working directory is required.
|
||||
- **Prototype** (HITL): Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
|
||||
- **Grilling** (HITL): Conversation via the /grilling and /domain-modeling skills, one question at a time. The default case.
|
||||
- **Task** (HITL or AFK): Manual work that must happen before a *decision* can be made — nothing to decide, prototype, or research, but the discussion is blocked until it's done. Signing up for a service so its API can be judged, provisioning access, moving data so its shape can be seen. This is the one type that *does* rather than decides — and it earns its place by unblocking a decision, not by delivering the destination. The agent drives it alone where it can (AFK); otherwise it hands the human a precise checklist (HITL). Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
|
||||
|
||||
## Fog of war
|
||||
|
||||
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the tickets lies fog — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the goal is clear and no tickets remain.
|
||||
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the live tickets lies the **fog of war** — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the destination is clear and no tickets remain.
|
||||
|
||||
The map's **Fog** section is where that dim view is written down: the suspected question, the area to revisit later, the risk you're deferring. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
|
||||
The map's **Not yet specified** section is where that dim view is written down: the suspected question, the area to revisit later. It's the undiscovered frontier _toward_ the destination — everything here is in scope, just not sharp enough to ticket. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
|
||||
|
||||
**Fog or ticket?** The test is whether you can state the question precisely now — _not_ whether you can answer it now.
|
||||
|
||||
- **Ticket when** the question is already sharp — even if it's blocked and you can't act on it yet.
|
||||
- **Fog when** you can't yet phrase it that sharply. Don't pre-slice fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
|
||||
- **Not yet specified when** you can't yet phrase it that sharply. Don't pre-slice the fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
|
||||
|
||||
Fog excludes only what's already decided (that's Decisions so far) and what's already a ticket.
|
||||
**Not yet specified** excludes what's already decided (Decisions so far), what's already a live ticket, and what's out of scope (the next section).
|
||||
|
||||
## Out of scope
|
||||
|
||||
Fog only ever gathers _toward_ the destination. The destination fixes the scope, so work beyond it is **out of scope** — it isn't fog, and it doesn't belong in **Not yet specified**. It gets its own **Out of scope** section on the map: work you've consciously ruled out of _this_ effort. Scope, not sharpness, lands it here.
|
||||
|
||||
Out-of-scope work never graduates — the frontier stops at the destination — so it returns only if the destination is redrawn, and then as a fresh effort, not a resumption.
|
||||
|
||||
Ruling something out of scope is a scoping act, not a step on the route. When a ticket that already exists turns out to sit past the destination — mis-scoped in while charting, or exposed by a resolution — **close it** (a closed ticket is unambiguously off the frontier) and leave one line in the **Out of scope** section: the gist plus why it's out of scope, linking the closed ticket. It stays out of **Decisions so far**, which records the route actually walked — a scope boundary isn't a step on it.
|
||||
|
||||
## Invocation
|
||||
|
||||
Two modes. Either way, **never resolve more than one ticket per session.**
|
||||
Two modes. Either way, **never resolve more than one ticket per session** — with the exception of research tickets.
|
||||
|
||||
### Chart the map
|
||||
|
||||
User invokes with a loose idea.
|
||||
|
||||
1. Run a `/grilling` and `/domain-modeling` session to surface the open decisions.
|
||||
2. **Create the map** (label `wayfinder:map`): Notes filled in, Decisions-so-far empty, Fog sketched.
|
||||
3. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the Fog.
|
||||
4. Stop — charting the map is one session's work; do not also resolve tickets.
|
||||
1. **Name the destination.** Run a `/grilling` and `/domain-modeling` session to pin down what this map is finding its way to — the spec, decision, or change. The destination fixes the scope, so it's settled first.
|
||||
2. **Map the frontier.** Grill again, **breadth-first** this time: fan out across the whole space rather than deep on any one thread, surfacing the open decisions and the first steps takeable now. **If this surfaces no fog** — the way to the destination is already clear, the whole journey small enough for one session — you don't need a map. Stop and ask the user how they'd like to proceed.
|
||||
3. **Create the map** (label `wayfinder:map`): Destination and Notes filled in, Decisions-so-far empty, the fog sketched into **Not yet specified**.
|
||||
4. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the fog — the **Not yet specified** section.
|
||||
5. **Fire the research subagents.** For each `research` ticket you just created, spin up a `/research` subagent to resolve it in parallel, capturing its findings on a throwaway `research/<name>` branch with a context pointer from the ticket.
|
||||
6. Stop — charting is one session's work; it hand-resolves nothing.
|
||||
|
||||
### Work through the map
|
||||
|
||||
@@ -96,6 +123,6 @@ User invokes with a map (URL or number). A ticket is **optional** — without on
|
||||
2. Choose the ticket. If the user named one, use it. Otherwise take the first frontier ticket in order. **Claim it**: assign it to yourself before any work.
|
||||
3. Resolve it — **zoom as needed**: fetch the full body of any related or closed ticket on demand; invoke the skills the `## Notes` block names. If in doubt, use `/grilling` and `/domain-modeling`.
|
||||
4. Record the resolution: post the answer as a **resolution comment**, **close** the issue, and **append a context pointer** to the map's Decisions-so-far.
|
||||
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from the Fog so it lives only as its new ticket. If the decision invalidates other parts of the map, update or delete those tickets.
|
||||
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from **Not yet specified** so it lives only as its new ticket. If the answer reveals a ticket — this one or another — sits beyond the destination, **rule it out of scope** rather than resolving it on the route. If the decision invalidates other parts of the map, update or delete those tickets.
|
||||
|
||||
The user may run unblocked tickets in parallel, so expect other sessions to be editing the tracker concurrently.
|
||||
|
||||
5
.agents/skills/wayfinder/agents/openai.yaml
Normal file
5
.agents/skills/wayfinder/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Wayfinder"
|
||||
short_description: "Map a large effort as decision tickets"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -158,6 +158,12 @@ _Failure mode._ Ending the current step before it is genuinely done, because the
|
||||
|
||||
_Avoid_: premature closure, the rush, rushing, shortcutting
|
||||
|
||||
### Negation
|
||||
|
||||
_Failure mode._ Steering by prohibition — telling the agent what _not_ to do — which drags the forbidden behaviour into context and makes it _more_ available, not less. _Don't think of an elephant_, and the elephant is all there is; _never write verbose comments_, and verbosity is the pattern the agent has just read. The negation is a weak modifier the strongly-activated concept overruns, so the ban half-reads as an instruction to do the thing. Its **leading word** is the _elephant_: whatever a prohibition names into the frame. Cure: prompt the **positive** — describe the target behaviour ("write one-line comments") so the banned one is never spoken. A prohibition earns its place only as a hard guardrail on a behaviour you cannot phrase positively; even then, pair it with the positive target so attention lands on what to do.
|
||||
|
||||
_Avoid_: ironic rebound, don't-prompting, the pink elephant
|
||||
|
||||
## Pruning
|
||||
|
||||
Keeping a skill lean — each remedy paired with the failure it cures.
|
||||
|
||||
@@ -80,3 +80,4 @@ Use these to diagnose issues the user may be having with the skill.
|
||||
- **Sediment** — stale layers that settle because adding feels safe and removing feels risky. The default fate of any skill without a pruning discipline.
|
||||
- **Sprawl** — a skill simply too long, even when every line is live and unique. Hurts readability and maintainability and wastes tokens. The cure is the ladder: disclose **reference** behind pointers, and split by **branch** or sequence so each path carries only what it needs.
|
||||
- **No-op** — a line the model already obeys by default, so you pay load to say nothing. The test: does it change behaviour versus the default? A weak leading word (_be thorough_ when the agent is already thorough-ish) is a no-op; the fix is a stronger word (_relentless_), not a different technique.
|
||||
- **Negation** — steering by prohibition backfires: _don't think of an elephant_ names the elephant and makes it more available, not less. Prompt the **positive** — state the target behaviour so the banned one is never spoken; keep a prohibition only as a hard guardrail you can't phrase positively, and even then pair it with what to do instead.
|
||||
|
||||
5
.agents/skills/writing-great-skills/agents/openai.yaml
Normal file
5
.agents/skills/writing-great-skills/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Writing Great Skills"
|
||||
short_description: "Principles for predictable skills"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
@@ -2,8 +2,9 @@
|
||||
|
||||
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
||||
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
||||
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
|
||||
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../.scratch/alpha-hardening/README.md), [handoff](../.scratch/alpha-hardening/handoff.md))
|
||||
- [Project status](project-status.md) — US-001…US-035 done; US-036…US-050 in docs/prd.json; alpha hardening + scratch features next
|
||||
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
|
||||
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
|
||||
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../.scratch/node-capability-admission/PRD.md), [README](../.scratch/node-capability-admission/README.md), ADR-0023)
|
||||
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan; [ADR-0023](../../docs/adr/0023-model-agnostic-node-capability-admission.md), [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md))
|
||||
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
||||
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
||||
|
||||
@@ -20,13 +20,13 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path.
|
||||
|
||||
**Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above.
|
||||
|
||||
**Two new issues from this session, both `ready-for-agent`:**
|
||||
- **21 — Honest-noise calibration corpus** (`.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`) rescoped from "prod gate" to a **hard alpha-release blocker**. Confirmed by code read: `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns bool only, no raw divergence value; fleet-dispatch exists but wrong shape (`server.py:2998-3104`, pinned routes + latency, not full-fleet + TOPLOC divergence); storage wrong shape (`registry_events` has no divergence/hardware columns). Three-part build: (1) surface raw TOPLOC distance from audit.py, (2) extend dispatch to hit every registered node with fixed prompt/seed, (3) new SQLite table keyed by node+GPU+dtype. Small-fleet exception granted (N = actual hired-VPS fleet size). Hired VPS hosts stay stake-free until this closes.
|
||||
- **23 — Dynamic HF-benchmarked pricing** (`.scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md`), high priority but not a release blocker. Pricing today is 100% static (`DEFAULT_PRICE_PER_1K_TOKENS = 0.02`, `billing.py:21`; `model_presets.json` has no per-model price). Target: 80% of cheapest comparable provider on `https://huggingface.co/inference/models` (per-provider-per-model marketplace, `?search=` query param works, no confirmed JSON API — plain scrape attempted first, escalate to headless browser only if the table isn't in raw HTML). Human-verified `hf_aliases` + `hf_verified_match_note` (params/quantization) per model, not auto-discovered matching. Reuses the `_settlement_loop` daemon-thread pattern for a daily refresh; falls back silently to the static default on any failure.
|
||||
**Two new issues from this session:**
|
||||
- **21 — Honest-noise calibration corpus** — `Status: ready-for-human` (engineering done 2026-07-06; blocked on human fleet calibration run before mainnet launch).
|
||||
- **23 — Dynamic HF-benchmarked pricing** — `Status: done` (see `23-dynamic-hf-pricing_completed.md`).
|
||||
|
||||
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
|
||||
|
||||
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (35/35 done) and does NOT see `.scratch/alpha-hardening/issues/`. No ralph loop is running and no `.ralph-tui/` state exists. `.scratch/alpha-hardening/prd.json` now has 23 stories (AH-001…AH-023); point Ralph at that file for the alpha-hardening branch. Do NOT use `ralph auto --parallel` on server.py-touching issues — 21 and 23 both touch `server.py`/`billing.py`/`audit.py`; if run in the same Ralph pass, run them serially, not in parallel (merge-conflict risk, same lesson as 03/04 previously).
|
||||
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (US-001…US-047; base 35/35 done, friends-test arc 36–47 open/in-progress). Alpha hardening uses `.scratch/alpha-hardening/prd.json` (AH-001…AH-023). Point Ralph at the prd.json for the branch you're running.
|
||||
|
||||
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
|
||||
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].
|
||||
|
||||
@@ -6,7 +6,18 @@ metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
# Project Status (2026-07-02)
|
||||
# Project Status (2026-07-13)
|
||||
|
||||
## Selected-node model placement (2026-07-14)
|
||||
|
||||
- Admin Model placement now opens a node selector for load and release; the control-plane accepts optional `node_id` and targets only that registry assignment. Multi-model serving remains supported through `ADD_SHARD` and `max_loaded_shards`.
|
||||
- Total node pool resource values are rendered from `/v1/network/map`'s `node.capacity` contract. Route selection remains assignment/capability/throughput/queue based; capacity is used for placement and falls back to tracker defaults only if a node truly omits it.
|
||||
|
||||
## Distributed inference performance (2026-07-14)
|
||||
|
||||
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
|
||||
|
||||
> Doc reconciliation 2026-07-13: `docs/prd.json` tracks US-001…US-050 (048 memory budget, 049 mainnet pilot, 050 Qwen demand placement). ADRs 0025–0026 added (TAI phase B/C, assignment ownership).
|
||||
|
||||
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:
|
||||
|
||||
|
||||
15
.codex/hooks.json
Normal file
15
.codex/hooks.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"hooks": {
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Write|Edit",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bash -c 'SRC=\"/mnt/d/DEV/workspace/REPOS/git.d-popov.com/neuron-tai/.claude/memory\" && DST=\"/home/dev/.claude/projects/-mnt-d-DEV-workspace-REPOS-git-d-popov-com-neuron-tai/memory\" && mkdir -p \"$DST\" && rsync -a \"$SRC/\" \"$DST/\" 2>/dev/null; true'"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -20,6 +20,7 @@ dist/
|
||||
!.env.testnet
|
||||
.rocm-local/*
|
||||
.pytest-tmp/*
|
||||
.cache/
|
||||
|
||||
# Local tracker/node sqlite databases (never commit runtime state)
|
||||
*.sqlite
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
|
||||
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
|
||||
|
||||
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet SOL/USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers, stake-free). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
||||
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers; no upfront stake, probation only). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
||||
|
||||
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch.
|
||||
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
||||
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch. Runbook: [04-toploc-calibration-run](./runbooks/04-toploc-calibration-run.md).
|
||||
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
||||
|
||||
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
||||
|
||||
@@ -41,22 +41,22 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
|
||||
|
||||
| Order | Issue | ID | Depends on |
|
||||
|---|---|---|---|
|
||||
| 1 | [Unified auth boundary](./issues/02-a2-unified-auth-boundary.md) + [Validator service token](./issues/20-validator-service-token.md) | A2, — | — |
|
||||
| 2 | [C1 hive gossip auth enforcement](./issues/01-c1-gossip-auth.md) | C1 | 02 |
|
||||
| 3 | [Persist strike/ban/reputation](./issues/05-a1-a5-persist-strike-ban-reputation.md) | A1/A5 | 02 |
|
||||
| 4 | [Starting credit 0 + spend cap](./issues/03-c5-starting-credit-zero.md) | C5, M1 | 02 |
|
||||
| 5 | [Tracker-authoritative accounting](./issues/04-h2-tracker-authoritative-accounting.md) | H2 | 02 |
|
||||
| 6 | [Wallet binding proof](./issues/11-c6-wallet-binding-proof.md) | C6 | 02, 03 |
|
||||
| 1 | [Unified auth boundary](./issues/02-a2-unified-auth-boundary_completed.md) + [Validator service token](./issues/20-validator-service-token_completed.md) | A2, — | — |
|
||||
| 2 | [C1 hive gossip auth enforcement](./issues/01-c1-gossip-auth_completed.md) | C1 | 02 |
|
||||
| 3 | [Persist strike/ban/reputation](./issues/05-a1-a5-persist-strike-ban-reputation_completed.md) | A1/A5 | 02 |
|
||||
| 4 | [Starting credit 0 + spend cap](./issues/03-c5-starting-credit-zero_completed.md) | C5, M1 | 02 |
|
||||
| 5 | [Tracker-authoritative accounting](./issues/04-h2-tracker-authoritative-accounting_completed.md) | H2 | 02 |
|
||||
| 6 | [Wallet binding proof](./issues/11-c6-wallet-binding-proof_completed.md) | C6 | 02, 03 |
|
||||
|
||||
### Phase 2 — Fraud arc (after Phase 1)
|
||||
|
||||
| Order | Issue | Depends on |
|
||||
|---|---|---|
|
||||
| 6 | [TOPLOC integration](./issues/06-fraud-toploc-integration.md) | 05 |
|
||||
| 7 | [Commitment + bisection blame](./issues/07-fraud-commitment-bisection-blame.md) | 06 |
|
||||
| 8 | [Reputation model](./issues/08-fraud-reputation-model-persistence.md) | 05, 07 |
|
||||
| 9 | [Routing + adaptive audit](./issues/09-fraud-reputation-routing-adaptive-audit.md) | 08 |
|
||||
| 10 | [Penalty calibration wiring](./issues/10-fraud-penalty-calibration-wiring.md) | 07, 08, 02 |
|
||||
| 6 | [TOPLOC integration](./issues/06-fraud-toploc-integration_completed.md) | 05 |
|
||||
| 7 | [Commitment + bisection blame](./issues/07-fraud-commitment-bisection-blame_completed.md) | 06 |
|
||||
| 8 | [Reputation model](./issues/08-fraud-reputation-model-persistence_completed.md) | 05, 07 |
|
||||
| 9 | [Routing + adaptive audit](./issues/09-fraud-reputation-routing-adaptive-audit_completed.md) | 08 |
|
||||
| 10 | [Penalty calibration wiring](./issues/10-fraud-penalty-calibration-wiring_completed.md) | 07, 08, 02 |
|
||||
|
||||
**Prod gate:** [21 honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) must complete before enabling production TOPLOC audit thresholds (issues 09–10 in prod). Dev/staging TOPLOC wiring (06–08) may proceed in parallel.
|
||||
|
||||
@@ -73,11 +73,12 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
|
||||
|
||||
| Issue |
|
||||
|---|
|
||||
| [16 US-006 + fraud issue reconciliation](./issues/16-doc-us006-reconciliation.md) |
|
||||
| [16 US-006 + fraud issue reconciliation](./issues/16-doc-us006-reconciliation_completed.md) |
|
||||
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
||||
| [18 Operational runbooks](./issues/18-doc-operational-runbooks.md) |
|
||||
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env.md) |
|
||||
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status.md) (done) |
|
||||
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
|
||||
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
|
||||
| [04 TOPLOC calibration run](./runbooks/04-toploc-calibration-run.md) (issue 21 ops) |
|
||||
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
|
||||
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
||||
|
||||
### Phase 5 — Distributed-inference performance (post-routing-fix)
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
## 1. Mission / where we are
|
||||
|
||||
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 0016–0019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first.
|
||||
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer. Pre-release audits found the money/trust path was not alpha-ready; **Bucket 1 alpha blockers are implemented** (see `.scratch/alpha-hardening/README.md`). Remaining launch gates: issue **21** (human calibration run), post-alpha Bucket 2 (12–15), and active scratch tracks (NCA, perf, distributed GGUF).
|
||||
|
||||
---
|
||||
|
||||
@@ -42,7 +42,7 @@ Point to artifacts — do not re-derive from this handoff.
|
||||
| Path | What it contains |
|
||||
|---|---|
|
||||
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
|
||||
| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 1–3) |
|
||||
| `.scratch/alpha-hardening/issues/` | 25 work items (Buckets 1–3 + perf follow-ups) |
|
||||
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
|
||||
| `docs/adr/0016–0019` | Alpha scope, auth, fraud, multi-tracker design |
|
||||
| `docs/agents/issue-tracker.md` | Issue file conventions |
|
||||
|
||||
@@ -38,4 +38,4 @@ Implement per ADR-0017 §3 using the auth helper/config from issue 02: shared hi
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md` — owns shared auth middleware/config. Implement in the same PR if simpler.
|
||||
- `02-a2-unified-auth-boundary_completed.md` — owns shared auth middleware/config. Implement in the same PR if simpler.
|
||||
@@ -16,7 +16,7 @@ Replace header-presence stubs with a single auth middleware that resolves API ke
|
||||
- `packages/tracker/meshnet_tracker/server.py` — `_session_account` (~2468+), `_handle_admin_accounts` (~2588–2608) — H4
|
||||
- `packages/tracker/meshnet_tracker/accounts.py` — `session_account()`, `create_session()` only (session store; not handler wiring)
|
||||
|
||||
Per ADR-0017 §4: forfeit → validator or admin; benchmark → admin; billing summary/settlements/registry wallets → admin session. Include the validator service token shape from `20-validator-service-token.md` in the same implementation if practical.
|
||||
Per ADR-0017 §4: forfeit → validator or admin; benchmark → admin; billing summary/settlements/registry wallets → admin session. Include the validator service token shape from `20-validator-service-token_completed.md` in the same implementation if practical.
|
||||
|
||||
## Test-first
|
||||
|
||||
@@ -39,8 +39,8 @@ Per ADR-0017 §4: forfeit → validator or admin; benchmark → admin; billing s
|
||||
|
||||
## Related
|
||||
|
||||
- `20-validator-service-token.md` — checklist for validator service token format, rotation, forfeit auth
|
||||
- `20-validator-service-token_completed.md` — checklist for validator service token format, rotation, forfeit auth
|
||||
|
||||
## Blocked by
|
||||
|
||||
None. This issue should land before `01-c1-gossip-auth.md`.
|
||||
None. This issue should land before `01-c1-gossip-auth_completed.md`.
|
||||
@@ -35,4 +35,4 @@ Per ADR-0017 §2 and ADR-0016 §3.
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md` (admin credit path secured)
|
||||
- `02-a2-unified-auth-boundary_completed.md` (admin credit path secured)
|
||||
@@ -35,4 +35,4 @@ Accounting fraud = inflating tokens or shard span. Per ADR-0018 §5.
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md`
|
||||
- `02-a2-unified-auth-boundary_completed.md`
|
||||
@@ -37,4 +37,4 @@ Include fields for: `strike_count`, `banned`, `completed_job_count`, graduated *
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md`
|
||||
- `02-a2-unified-auth-boundary_completed.md`
|
||||
@@ -42,6 +42,6 @@ Pin one canonical precision/quantization per model preset. Add `toploc` to valid
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `05-a1-a5-persist-strike-ban-reputation.md`
|
||||
- `05-a1-a5-persist-strike-ban-reputation_completed.md`
|
||||
|
||||
**Prod gate:** do not enable production audit thresholds until `21-honest-noise-calibration-corpus.md` completes (see README Phase 2 note).
|
||||
@@ -32,4 +32,4 @@ On audit selection, require nodes to supply TOPLOC-style fingerprints of **outpu
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `06-fraud-toploc-integration.md`
|
||||
- `06-fraud-toploc-integration_completed.md`
|
||||
@@ -35,5 +35,5 @@ Implement graduated reputation per ADR-0018 §6: score derives only from tracker
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `05-a1-a5-persist-strike-ban-reputation.md`
|
||||
- `07-fraud-commitment-bisection-blame.md` (audit outcomes feed reputation)
|
||||
- `05-a1-a5-persist-strike-ban-reputation_completed.md`
|
||||
- `07-fraud-commitment-bisection-blame_completed.md` (audit outcomes feed reputation)
|
||||
@@ -35,4 +35,4 @@ Audit selection must be unpredictable at request time (tracker RNG after commitm
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `08-fraud-reputation-model-persistence.md`
|
||||
- `08-fraud-reputation-model-persistence_completed.md`
|
||||
@@ -37,6 +37,6 @@ Per ADR-0018: **full pending forfeiture** is primary penalty; ×0.8 is routing d
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `07-fraud-commitment-bisection-blame.md`
|
||||
- `08-fraud-reputation-model-persistence.md`
|
||||
- `02-a2-unified-auth-boundary.md`
|
||||
- `07-fraud-commitment-bisection-blame_completed.md`
|
||||
- `08-fraud-reputation-model-persistence_completed.md`
|
||||
- `02-a2-unified-auth-boundary_completed.md`
|
||||
@@ -33,5 +33,5 @@ Require signed message from wallet pubkey (ed25519 via `cryptography` / solders)
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md`
|
||||
- `03-c5-starting-credit-zero.md`
|
||||
- `02-a2-unified-auth-boundary_completed.md`
|
||||
- `03-c5-starting-credit-zero_completed.md`
|
||||
@@ -9,7 +9,7 @@ Reconcile stale US-006 (Solana testnet stake contracts) with ADR-0015/0016 devne
|
||||
Also reconcile legacy fraud issues with the alpha-hardening fraud arc:
|
||||
|
||||
- `docs/issues/07-fraud-detection-slash.md` — on-chain stake slash model superseded by pending-balance forfeiture + TOPLOC (ADR-0018)
|
||||
- `docs/issues/34-forfeiture-penalty.md` — partially implemented; remaining fraud work lives in `.scratch/alpha-hardening/issues/06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`
|
||||
- `docs/issues/34-forfeiture-penalty.md` — partially implemented; remaining fraud work lives in `.scratch/alpha-hardening/issues/06-fraud-toploc-integration_completed.md` through `10-fraud-penalty-calibration-wiring_completed.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
@@ -45,8 +45,8 @@ Per [ADR-0017 §4](../../docs/adr/0017-tracker-authentication-and-authorization.
|
||||
|
||||
## Related
|
||||
|
||||
- `02-a2-unified-auth-boundary.md` — middleware + role checks
|
||||
- `02-a2-unified-auth-boundary_completed.md` — middleware + role checks
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `02-a2-unified-auth-boundary.md`
|
||||
- `02-a2-unified-auth-boundary_completed.md`
|
||||
@@ -1,6 +1,6 @@
|
||||
Status: ready-for-human
|
||||
|
||||
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money (mainnet SOL/USDT) traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
||||
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money mainnet USDT traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
||||
|
||||
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
|
||||
|
||||
@@ -14,9 +14,9 @@ Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and
|
||||
|
||||
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
||||
|
||||
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware.
|
||||
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay on probation (no upfront stake) until it's done, then move to paid USDT serving once thresholds derive from their own hardware.
|
||||
|
||||
**Current gap (confirmed 2026-07-06 by code read):** none of the three pieces below exist yet.
|
||||
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
|
||||
|
||||
1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (06–10) currently runs on a guessed threshold baked into that bool, not a calibrated one.
|
||||
2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 1–3-node *routes* and measures latency, not TOPLOC divergence across *every* registered node.
|
||||
@@ -36,7 +36,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
|
||||
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
|
||||
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
|
||||
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
|
||||
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section.
|
||||
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section; [runbook 04](../runbooks/04-toploc-calibration-run.md).
|
||||
|
||||
## ADR links
|
||||
|
||||
@@ -44,7 +44,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `06-fraud-toploc-integration.md` (TOPLOC wired; calibration uses same primitive) — done
|
||||
- `06-fraud-toploc-integration_completed.md` (TOPLOC wired; calibration uses same primitive) — done
|
||||
|
||||
## Blocks (prod gate)
|
||||
|
||||
|
||||
@@ -440,12 +440,12 @@
|
||||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||||
],
|
||||
"priority": 21,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money mainnet USDT). Operator runbook: .scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md",
|
||||
"dependsOn": [
|
||||
"AH-006"
|
||||
],
|
||||
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
|
||||
"completionNotes": "Engineering complete and unit-tested. Remaining: human runs POST /v1/calibration/toploc/run on live hired-VPS fleet, records envelope/FPR, wires thresholds — see runbook 04 and packages/validator/README.md."
|
||||
},
|
||||
{
|
||||
"id": "AH-022",
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# Runbook 04 — Honest-noise TOPLOC calibration (issue 21)
|
||||
|
||||
**Status:** engineering complete; **operator action required** before production audit thresholds.
|
||||
|
||||
**Blocks:** enabling calibrated TOPLOC thresholds on a mainnet / friends-test fleet (issue 21, ADR-0018).
|
||||
|
||||
## When to run
|
||||
|
||||
- Before first real-money traffic with audit enforcement enabled.
|
||||
- Again whenever the fleet’s **hardware mix** changes materially (new GPU generation, CPU-only nodes added, precision/recipe change per model).
|
||||
|
||||
Alpha exception: with a **small hired-VPS-only** fleet, `gate_status.ready` may mean “covers every node we operate today” (`--toploc-calibration-gate-min-hardware-profiles 1`).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Tracker running with billing + registry + `--toploc-calibration-db PATH` (or default under tracker cwd).
|
||||
- At least one **solo-capable** node per hardware profile you want in the corpus (full model coverage — partial shards are skipped).
|
||||
- Admin or validator credentials (`Authorization` header or validator service token per ADR-0017).
|
||||
- Reference validator can replay the fixed calibration prompt (same model/seed as dispatch uses).
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Register the fleet** — all nodes you intend to pay on mainnet should be up, admitted (NCA when enabled), and solo-serving the calibration model.
|
||||
|
||||
2. **Dispatch the job** (admin/validator only):
|
||||
|
||||
```bash
|
||||
curl -X POST "https://<tracker>/v1/calibration/toploc/run" \
|
||||
-H "Authorization: Bearer <admin-or-validator-token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
Partial-shard nodes appear under `skipped_partial_shard_node_ids`. Per-node failures appear under `skipped` with reasons.
|
||||
|
||||
3. **Wait for completion** — watch tracker logs and node consoles until every solo-capable node has a row in the corpus.
|
||||
|
||||
4. **Fetch results**:
|
||||
|
||||
```bash
|
||||
curl "https://<tracker>/v1/calibration/toploc/results" \
|
||||
-H "Authorization: Bearer <admin-or-validator-token>"
|
||||
```
|
||||
|
||||
Record:
|
||||
- `envelope` — p99 metrics + 20% safety margin (recommended tolerances).
|
||||
- `gate_status.ready` and `gate_status.hardware_profiles`.
|
||||
- `estimated_false_positive_rate` (in-sample sanity check only).
|
||||
|
||||
5. **Write up thresholds** — paste envelope values into operator notes / issue 21 comment. Do **not** wire into production `ToplocAuditConfig` until you have reviewed FPR on this fleet.
|
||||
|
||||
6. **Mark issue 21 done** — when corpus covers the launch fleet and thresholds are documented.
|
||||
|
||||
## Two-wallet / minimal pilot variant
|
||||
|
||||
If your “fleet” is one node machine + one client:
|
||||
|
||||
- Run calibration against the **node** profile only (one hardware row is enough for `gate_status` with min profiles = 1).
|
||||
- Client wallet is irrelevant to calibration — it never serves inference.
|
||||
|
||||
## Do not
|
||||
|
||||
- Enable stricter production audit thresholds before this completes.
|
||||
- Reuse a corpus collected on devnet/mock hardware for a different mainnet GPU mix without re-running.
|
||||
|
||||
## References
|
||||
|
||||
- Issue: `.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`
|
||||
- Code: `packages/tracker/meshnet_tracker/calibration.py`, `POST /v1/calibration/toploc/run`
|
||||
- Validator: `packages/validator/README.md` — TOPLOC audit contract
|
||||
@@ -12,4 +12,10 @@ Provide an opt-in, admin-only tracker Dashboard Testing tab that dynamically dis
|
||||
- One active run.
|
||||
- Real inference stays separately environment-gated and excluded from default suites.
|
||||
|
||||
## Operator workflow
|
||||
|
||||
See [`docs/dev/dashboard-test-runner.md`](../../docs/dev/dashboard-test-runner.md)
|
||||
for launch configuration, default safe suites vs the gated real-inference suite,
|
||||
and required environment variables.
|
||||
|
||||
See `prd.json` for executable Ralph user stories and acceptance criteria.
|
||||
|
||||
@@ -51,15 +51,16 @@
|
||||
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
|
||||
],
|
||||
"priority": 3,
|
||||
"passes": false,
|
||||
"passes": true,
|
||||
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
|
||||
"dependsOn": [
|
||||
"US-001",
|
||||
"US-002"
|
||||
]
|
||||
],
|
||||
"completionNotes": "Completed by agent"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"updatedAt": "2026-07-11T17:02:30.520Z"
|
||||
"updatedAt": "2026-07-12T01:58:06.286Z"
|
||||
}
|
||||
}
|
||||
@@ -1,177 +0,0 @@
|
||||
# ADR-0020: Distributed GGUF/llama.cpp Runtime With Per-Shard Local KV
|
||||
|
||||
Status: Proposed
|
||||
|
||||
## Context
|
||||
|
||||
The project currently uses PyTorch/Transformers for real model shards. That decision was captured in ADR-0001 because llama.cpp RPC at the time required the primary node to load the full model and distribute weights to workers, which conflicted with the desired model where nodes independently hold shards.
|
||||
|
||||
We now want to serve very large open models, including GLM-5.2 and Ornith-class MoE models, over a torrent-like inference marketplace. CPU and mixed consumer hardware matter. LM Studio and llama.cpp demonstrate much better CPU/GGUF performance than our current PyTorch CPU path. The user also has a personal relationship with Georgi Gerganov, making upstream collaboration plausible.
|
||||
|
||||
The current distributed PyTorch path is not yet production-grade: it recomputes the full growing sequence for every output token and disables KV cache inside manual layer calls. It sends hidden activations across seams, not KV, but those activations currently cover the full sequence every decode step.
|
||||
|
||||
## Decision
|
||||
|
||||
Adopt a distributed GGUF/llama.cpp runtime track while keeping PyTorch as the reference and fast-architecture backend.
|
||||
|
||||
The runtime model is:
|
||||
|
||||
- GGUF/model artifacts are distributed through torrent/content-addressed storage.
|
||||
- Nodes independently acquire and verify artifacts; no root node streams model weights to workers at session start.
|
||||
- Tracker chooses a sticky route covering all layers.
|
||||
- Each node owns hot KV/state for the layers it executes.
|
||||
- Prefill sends chunked activations through the route and builds local per-shard KV.
|
||||
- Decode sends one-step activations through the route and appends local KV at every shard.
|
||||
- Cache/CDN servers store cold artifacts and optional prefix/session snapshots, not hot per-token KV.
|
||||
- Context is capped at 128K for the first serious product path.
|
||||
|
||||
## Technical Framework
|
||||
|
||||
The design separates five planes:
|
||||
|
||||
- **Control plane**: tracker registry, coverage map, route selection, session lifecycle, telemetry, billing, and audit.
|
||||
- **Artifact plane**: Shard Swarms, GGUF/safetensors/tokenizer files, manifests, hashes, and local node storage.
|
||||
- **Execution plane**: active Inference Route, chunked prefill, one-step decode, and hidden-state movement across activation seams.
|
||||
- **Session state plane**: per-shard Hot KV State on route nodes, plus optional Prefix Snapshots outside the hot loop.
|
||||
- **Economics/trust plane**: reward accounting, validation events, slash proofs, public/private route policy.
|
||||
|
||||
Hard invariants:
|
||||
|
||||
1. Public-network Shards are contiguous layer ranges.
|
||||
2. Hot KV State is local to the node serving that Shard in that Route Session.
|
||||
3. Artifact distribution and route execution are separate systems.
|
||||
4. Decode seam payload must be `O(hidden_size)`.
|
||||
5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
|
||||
6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
|
||||
7. Model/backend-specific cache internals stay behind backend capability reports.
|
||||
8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
|
||||
9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
|
||||
|
||||
The full challenge register is in [technical-challenges.md](./technical-challenges.md). The open decision gates are in [decision-framework.md](./decision-framework.md).
|
||||
|
||||
Resolved gate:
|
||||
|
||||
- Public-network Shards are layer ranges. Tensor-parallel/ring execution belongs inside a trusted node, colocated pod, or future composite node abstraction, not as the v1 public routing primitive.
|
||||
- Hot KV State is local to each route node for the Shard it serves. Cache servers may store Prefix Snapshots, but they are not part of the per-token decode path.
|
||||
- Distributed Route Session and Hot KV State semantics will be proven in the PyTorch route before llama.cpp/GGUF is extended for layer-boundary execution.
|
||||
- Streaming responses are preferred when feasible. Realtime Generation Telemetry is required so clients can see phase, generated token count, and tokens/sec even during prefill or non-streaming fallback paths.
|
||||
- llama.cpp/GGUF work targets upstreamable `libllama`/ggml hooks. A prototype fork is acceptable for exploration, but a permanent fork is not the plan.
|
||||
- Model targeting is two-tiered: use a small llama.cpp-supported GGUF model for the first protocol smoke test, then use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. GLM-5.2 and Ornith remain later support audits.
|
||||
- Alpha fails Route Sessions on route-node loss instead of attempting automatic route repair. Repair requires compatible Prefix Snapshots and is a later capability.
|
||||
- v1 activation transfer stays on binary HTTP as defined by ADR-0008. QUIC/WebRTC/custom transport can be introduced later behind the same activation protocol.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not put remote cache servers in the per-token hot KV path.
|
||||
- Do not require every node to hold the full model.
|
||||
- Do not fork llama.cpp long-term if upstream APIs can support the needed layer-boundary hooks.
|
||||
- Do not target GLM-5.2 or Ornith first; prove the route/KV protocol on a simpler well-supported GGUF model, then target DeepSeek-V4-Flash as the first serious large model.
|
||||
|
||||
## Options Considered
|
||||
|
||||
### A. Keep PyTorch-only distributed inference
|
||||
|
||||
Pros:
|
||||
|
||||
- Easy access to new Hugging Face architectures.
|
||||
- Transformers has mature single-process KV semantics.
|
||||
- Existing code already loads shards.
|
||||
|
||||
Cons:
|
||||
|
||||
- CPU inference is much slower than llama.cpp/GGUF.
|
||||
- Current distributed path bypasses `generate()` and disables cache.
|
||||
- Quantized GGUF ecosystem and LM Studio users are outside the runtime.
|
||||
|
||||
### B. Use llama.cpp only as a full local model backend
|
||||
|
||||
Pros:
|
||||
|
||||
- Quick performance win for nodes with enough RAM/VRAM.
|
||||
- Minimal coordination with distributed protocol.
|
||||
|
||||
Cons:
|
||||
|
||||
- Does not unlock 397B/753B-class models for ordinary nodes.
|
||||
- Does not solve marketplace layer routing.
|
||||
|
||||
### C. Distributed GGUF with per-shard local KV (chosen)
|
||||
|
||||
Pros:
|
||||
|
||||
- Aligns with torrent artifact distribution.
|
||||
- Avoids root streaming weights to workers.
|
||||
- Uses llama.cpp/GGUF performance where supported.
|
||||
- Compatible with public node rewards by layer/work contribution.
|
||||
- Scales KV memory by layer range.
|
||||
|
||||
Cons:
|
||||
|
||||
- Requires new runtime APIs around layer-boundary hidden states and per-session KV.
|
||||
- Requires model-specific cache metadata for DSA/MLA/hybrid attention.
|
||||
- Harder to debug than single-process `generate()`.
|
||||
|
||||
### D. Centralized KV cache servers
|
||||
|
||||
Pros:
|
||||
|
||||
- Easier apparent session failover.
|
||||
- Central accounting of active cache.
|
||||
|
||||
Cons:
|
||||
|
||||
- Puts remote storage in the per-token hot path.
|
||||
- Adds bandwidth and latency at the worst possible point.
|
||||
- Creates consistency and privacy problems.
|
||||
|
||||
Rejected for hot decode. Accepted only for cold prefix snapshots and failover checkpoints.
|
||||
|
||||
## Consequences
|
||||
|
||||
- ADR-0001 should eventually be amended: PyTorch remains valid, but llama.cpp/GGUF becomes a first-class backend.
|
||||
- The activation protocol must split prefill and decode explicitly.
|
||||
- Session IDs must be stable across the full request. The current fresh UUID-per-hop-call behavior must change.
|
||||
- Backends must report cache budget and cache compatibility.
|
||||
- Tracker route selection must include disk, memory pressure, cache warmth, and network latency.
|
||||
- Billing can be based on layer work, prefill tokens, decode tokens, and observed route participation.
|
||||
- Client UX should stream token deltas when feasible and must include route-session progress telemetry even when token deltas are not streamed.
|
||||
|
||||
## Required Runtime Capabilities
|
||||
|
||||
PyTorch path:
|
||||
|
||||
- manual layer calls with `past_key_values` / model-specific cache object
|
||||
- per-shard session cache store
|
||||
- prefill chunk append
|
||||
- decode step append
|
||||
- stable session lifecycle endpoints
|
||||
|
||||
llama.cpp/GGUF path:
|
||||
|
||||
- full local GGUF serving
|
||||
- layer/tensor map extraction from GGUF
|
||||
- optional partial layer loading or mmap-backed selected execution
|
||||
- inbound hidden-state execution from arbitrary start layer
|
||||
- outbound hidden-state return at stop layer
|
||||
- per-session KV ownership for loaded layers
|
||||
- cache budget/compatibility introspection
|
||||
- GLM-5.2 DSA support when upstream/runtime supports it
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
1. Add full-model `LlamaCppBackend` using `llama-server` or `libllama`.
|
||||
2. Implement distributed KV in the PyTorch path to prove semantics.
|
||||
3. Add session lifecycle and prefill/decode wire protocol.
|
||||
4. Add model artifact manifest and torrent seeding metadata.
|
||||
5. Prototype localhost two-process llama.cpp layer boundary execution.
|
||||
6. Generalize to network route.
|
||||
7. Bring in GLM-5.2/Ornith once backend support and cache accounting are verified.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A two-node localhost route can prefill once and decode N tokens without recomputing the full prompt.
|
||||
- Seam payload during decode is `O(hidden_size)`, not `O(sequence_length * hidden_size)`.
|
||||
- Per-node KV memory grows with owned layer count and context length.
|
||||
- Route loss during alpha fails cleanly with explicit reason.
|
||||
- Full local GGUF backend outperforms PyTorch CPU on a supported model.
|
||||
- Artifact manifest can identify exactly which files/chunks a node must seed for its advertised layer range.
|
||||
@@ -1,83 +1,714 @@
|
||||
# PRD: Distributed GGUF Runtime
|
||||
|
||||
## Summary
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
Build a distributed inference runtime that can serve large, quality-first open models by combining torrent-style model artifact distribution with sticky multi-node Inference Routes and per-shard local Hot KV State.
|
||||
## Goal
|
||||
|
||||
The first runtime proof uses the existing PyTorch route because it exposes model internals and cache semantics more directly. GGUF/llama.cpp becomes the performance path after the route-session contract is proven.
|
||||
Deliver benchmark-gated, concurrent, distributed GGUF inference across consumer machines through existing Meshnet control-plane behavior and a standalone native worker around upstream llama.cpp. The accepted target is DeepSeek V4 Flash, not the superseded legacy target.
|
||||
|
||||
## Goals
|
||||
## Locked scope
|
||||
|
||||
- Eliminate full-prompt recompute in distributed decode.
|
||||
- Keep decode activation seams proportional to `hidden_size`, not `context_length * hidden_size`.
|
||||
- Keep Hot KV State local to the node serving the relevant Shard.
|
||||
- Stream token deltas when feasible and always expose Generation Telemetry.
|
||||
- Add a local full-model GGUF backend for immediate CPU performance wins.
|
||||
- Define Model Artifact manifests so nodes can verify, seed, and advertise artifacts without depending on Hugging Face at request time.
|
||||
- Prototype an upstreamable llama.cpp/libllama layer-boundary API.
|
||||
- Use DeepSeek-V4-Flash as the first serious large-model target after smaller protocol smoke tests.
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
## Non-Goals
|
||||
## Target identities
|
||||
|
||||
- No centralized hot KV cache in the per-token decode path.
|
||||
- No automatic route repair in alpha.
|
||||
- No permanent llama.cpp fork as the intended architecture.
|
||||
- No GLM-5.2 or Ornith first; they remain follow-up support audits.
|
||||
- No transport rewrite to QUIC/WebRTC before route/session semantics are proven.
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
## Resolved Decisions
|
||||
## Quality gates
|
||||
|
||||
- Public-network Shards are contiguous transformer layer ranges.
|
||||
- Tensor/ring parallelism belongs inside one trusted node, one colocated pod, or a future composite node abstraction.
|
||||
- Hot KV State is local to route nodes; Prefix Snapshots are optional cold recovery/reuse artifacts.
|
||||
- PyTorch distributed KV/session semantics are proven before llama.cpp distributed execution.
|
||||
- Streaming responses are preferred; Generation Telemetry is mandatory.
|
||||
- llama.cpp/GGUF work targets upstreamable `libllama`/ggml hooks.
|
||||
- Alpha fails Route Sessions on route-node loss.
|
||||
- v1 activation transfer stays on binary HTTP.
|
||||
|
||||
## Target User Experience
|
||||
|
||||
A client sends an OpenAI-compatible request. The Gateway or Tracker Node accepts the request, creates a Route Session, and streams token deltas when supported. The client receives live Generation Telemetry for route phase, prefill progress, generated token count, rolling tokens/sec, route health, and failure reason.
|
||||
|
||||
If a route node drops in alpha, the request fails clearly. A retry starts a new Route Session from scratch.
|
||||
|
||||
## Runtime Shape
|
||||
|
||||
```text
|
||||
client request
|
||||
-> Gateway / Tracker Node creates Route Session
|
||||
-> Tracker selects sticky Inference Route
|
||||
-> prefill:
|
||||
prompt chunks move through Shards
|
||||
each node appends local Hot KV State
|
||||
-> decode:
|
||||
one-step activation moves through Shards
|
||||
each node reads/appends local Hot KV State
|
||||
tail returns token/logits
|
||||
-> client receives streamed token deltas where possible
|
||||
-> Generation Telemetry continues until complete or failed
|
||||
```
|
||||
The canonical gate groups live in `prd.json`. Every story explicitly requires its applicable shared gates: deterministic targeted tests, Python compile checks where applicable, native CMake/CTest and exact patch verification where applicable, real artifact/hardware provenance for opt-in lanes, `git diff --check`, and a durable evidence handoff.
|
||||
|
||||
## Milestones
|
||||
|
||||
| Milestone | Outcome | Issues |
|
||||
|---|---|---|
|
||||
| M1 — Session protocol proof | Stub route has stable Route Sessions, prefill/decode split, telemetry, and streaming contract | 01, 02, 03 |
|
||||
| M2 — PyTorch reference route | Distributed PyTorch decode uses local per-shard cache and stops full-prompt recompute | 04 |
|
||||
| M3 — Local GGUF performance path | Single-node GGUF backend serves through the node API and reports backend metadata | 05 |
|
||||
| M4 — Artifact plane | Model Artifact manifest supports verification, layer mapping, and node advertisement | 06 |
|
||||
| M5 — llama.cpp collaboration proof | Localhost layer-boundary prototype identifies upstreamable llama.cpp/libllama API | 07 |
|
||||
| M6 — Networked GGUF route | Multi-node GGUF route uses the resolved protocol and fails cleanly on node loss | 08 |
|
||||
| M7 — First large model | DeepSeek-V4-Flash support path is audited and converted into follow-up runtime tasks | 09 |
|
||||
- **M0 — Truth and contracts (DGR-017..DGR-020):** Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
|
||||
- **M1 — Protocol and native substrate (DGR-021..DGR-033):** Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
|
||||
- **M2 — Dense vertical proof (DGR-034..DGR-043):** Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
|
||||
- **M3 — DeepSeek V4 Flash alpha (DGR-044..DGR-054):** Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
|
||||
- **M4 — Performance and beta hardening (DGR-055..DGR-067):** Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
|
||||
- **M5 — Release and maintenance (DGR-068..DGR-071):** Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.
|
||||
|
||||
## Acceptance Criteria
|
||||
## User stories
|
||||
|
||||
- A two-node route can prefill once and decode without resending full prompt activations.
|
||||
- Decode seam payload is one token/hidden-state step after prefill.
|
||||
- Route Session telemetry is visible before first token and during decode.
|
||||
- Streaming token deltas work where the backend supports them.
|
||||
- Route-node loss produces a structured alpha failure and does not attempt unsafe repair.
|
||||
- A local GGUF model can serve via the node API.
|
||||
- A Model Artifact manifest can prove which Shards a node can serve.
|
||||
- DeepSeek-V4-Flash has a written support recommendation: PyTorch, vLLM/SGLang, llama.cpp/GGUF, or blocked.
|
||||
### DGR-017: Reconcile and clean the superseded DGR backlog
|
||||
|
||||
**Milestone:** M0 · **Mode:** AFK · **State:** complete · **Depends on:** none
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
|
||||
|
||||
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-018: Define canonical Ralph and Gitea metadata schema
|
||||
|
||||
**Milestone:** M0 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
|
||||
|
||||
- [ ] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
||||
- [ ] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
||||
- [ ] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
||||
- [ ] Add deterministic model-free tests for parse, validation, and generation round trips.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-019: Lock alpha and beta performance contracts
|
||||
|
||||
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
|
||||
|
||||
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
|
||||
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
|
||||
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
|
||||
- [ ] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
|
||||
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-020: Run the controlled whole-model GGUF baseline
|
||||
|
||||
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-019
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
|
||||
|
||||
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
|
||||
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
|
||||
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
|
||||
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-021: Define the versioned named-tensor stream envelope
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md`, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.
|
||||
|
||||
- [ ] Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
|
||||
- [ ] Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
|
||||
- [ ] Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
|
||||
- [ ] Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-022: Define Shard lifecycle and structured status RPCs
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.
|
||||
|
||||
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
|
||||
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
|
||||
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
|
||||
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-023: Make Python and C++ protobuf generation reproducible
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.
|
||||
|
||||
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
|
||||
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
|
||||
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests.
|
||||
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-024: Implement in-memory fake gRPC seam transport
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.
|
||||
|
||||
- [ ] Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
|
||||
- [ ] Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
|
||||
- [ ] Verify direct and opaque-relay framing preserve identical protobuf bytes.
|
||||
- [ ] Tests require no sockets outside localhost, model downloads, or native accelerator.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-025: Define exact artifact and runtime recipe identity
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018, DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md`, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.
|
||||
|
||||
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
|
||||
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant.
|
||||
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
|
||||
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-026: Provision exact split-GGUF artifacts outside /home
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.
|
||||
|
||||
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
|
||||
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
|
||||
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
|
||||
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.
|
||||
|
||||
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
|
||||
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
|
||||
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
|
||||
- [ ] Offline reuse is supported only after the cached tree’s exact identity is verified.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-028: Implement numbered patch-stack apply and verification
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.
|
||||
|
||||
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
|
||||
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
|
||||
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
|
||||
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-029: Create the native CMake skeleton and deterministic CPU lane
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027, DGR-028
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.
|
||||
|
||||
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
|
||||
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
|
||||
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
|
||||
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-030: Add accelerator build presets and native CI matrix
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-029
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md`, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.
|
||||
|
||||
- [ ] Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
|
||||
- [ ] Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
|
||||
- [ ] Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
|
||||
- [ ] Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-031: Introduce the project-owned `ShardEngine` interface
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-025
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md`, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.
|
||||
|
||||
- [ ] Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
|
||||
- [ ] Use project-owned request/result/state types; expose no `ggml_tensor`, llama context, scheduler, or ABI-owned structure.
|
||||
- [ ] Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
|
||||
- [ ] Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-032: Implement deterministic fake `ShardEngine`
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-031
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md`, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.
|
||||
|
||||
- [ ] Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
|
||||
- [ ] Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
|
||||
- [ ] Support configurable delay, memory pressure, malformed output, and crash injection.
|
||||
- [ ] Contract tests distinguish fixture evidence from real-model certification.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-033: Build a standalone fake C++ gRPC Shard worker
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-024, DGR-032
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.
|
||||
|
||||
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
|
||||
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
|
||||
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
|
||||
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-034: Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-028, DGR-029, DGR-031
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md`, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.
|
||||
|
||||
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
|
||||
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state.
|
||||
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
|
||||
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-035: Implement dense architecture boundary input/output
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-031, DGR-034
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md`, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.
|
||||
|
||||
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
|
||||
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail returns logits or sampled-token output under an explicit contract.
|
||||
- [ ] Uncertified architectures and incompatible boundary schemas fail closed.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-036: Prove dense fixture and real-model range parity
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-035
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md`, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.
|
||||
|
||||
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
|
||||
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity.
|
||||
- [ ] At least 32 greedy decode tokens match the locked tolerance.
|
||||
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-037: Bind llama.cpp to the standalone worker
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023, DGR-031, DGR-034, DGR-035
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp `ShardEngine` implementation without changing the public worker contract.
|
||||
|
||||
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
|
||||
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
|
||||
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
|
||||
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-038: Implement isolated shard-local Hot KV State
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-037
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md`, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.
|
||||
|
||||
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
|
||||
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
|
||||
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-039: Pass local two-process dense acceptance
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-036, DGR-037, DGR-038
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md`, and evidence READMEs for dependencies (DGR-036, DGR-037, DGR-038) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the first real, independently executing distributed GGUF route on one host.
|
||||
|
||||
- [ ] Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
|
||||
- [ ] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
|
||||
- [ ] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
|
||||
- [ ] Killing one worker returns a bounded structured failure rather than hanging.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-040: Add node-side native worker supervision
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-037
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md`, and evidence READMEs for dependencies (DGR-033, DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Let the existing node service safely start, probe, stop, and restart the standalone worker.
|
||||
|
||||
- [ ] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
|
||||
- [ ] Startup verifies worker binary, artifact identity, recipe, and range before registration.
|
||||
- [ ] Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
|
||||
- [ ] Tests use the fake worker and deterministic crash injection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-041: Register native Shard capabilities without redesigning Meshnet
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025, DGR-040
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md`, and evidence READMEs for dependencies (DGR-025, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt native worker identity and capacity to existing backend-agnostic capability admission.
|
||||
|
||||
- [ ] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
|
||||
- [ ] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
|
||||
- [ ] Uncertified backend/model/recipe combinations are visible but unroutable.
|
||||
- [ ] Existing Transformers registration and route tests remain unchanged in behavior.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-042: Carry native frames through direct and existing relay seams
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-024, DGR-040
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`, and evidence READMEs for dependencies (DGR-024, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Use direct gRPC where available and the existing relay as opaque protobuf transport without redesigning relay behavior.
|
||||
|
||||
- [ ] Direct paths use the long-lived gRPC activation stream.
|
||||
- [ ] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
|
||||
- [ ] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
|
||||
- [ ] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-041
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md`, and evidence READMEs for dependencies (DGR-041) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt exact GGUF recipe/range compatibility and measured capacity, queue, seam-cost, health, and reliability data into the existing backend-agnostic routing inputs without redesigning routing or load balancing.
|
||||
|
||||
- [ ] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
|
||||
- [ ] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
|
||||
- [ ] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
|
||||
- [ ] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-044: Pin the DeepSeek V4 Flash target contract
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-019, DGR-025, DGR-026, DGR-027
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/044-pin-the-deepseek-v4-flash-target-contract.md`, and evidence READMEs for dependencies (DGR-019, DGR-025, DGR-026, DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Fix the alpha/beta model identity and upstream maturity assumptions before implementing the adapter.
|
||||
|
||||
- [ ] Pin Hugging Face target SHA `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- [ ] Record the selected GGUF provenance and exact source binding without assuming one quant.
|
||||
- [ ] Record that llama.cpp support began with PR 24162/commit `8c146a8` and remains maturing; the build manifest still pins one exact validated upstream commit.
|
||||
- [ ] Record 43 main layers plus MTP and all known architecture-specific state obligations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-045: Inventory V4 GGUF tensors and layer ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-026, DGR-044
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/045-inventory-v4-gguf-tensors-and-layer-ownership.md`, and evidence READMEs for dependencies (DGR-026, DGR-044) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Derive exact head, repeated-layer, tail, expert, auxiliary-state, and MTP ownership from the pinned artifact.
|
||||
|
||||
- [ ] Produce a machine-readable inventory tied to the target SHA and exact GGUF artifact.
|
||||
- [ ] Classify embeddings, 43 main layers, final head, MTP, mHC, attention variants, indexer/compressor, and MoE tensors.
|
||||
- [ ] Validate half-open range ownership and fail on unclassified required tensors.
|
||||
- [ ] Inventory tooling works across quants without embedding quant names in architecture logic.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-046: Define the V4 typed architecture boundary schema
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-045
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.
|
||||
|
||||
- [ ] Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.
|
||||
- [ ] Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.
|
||||
- [ ] Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.
|
||||
- [ ] Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-047: Adapt the upstream V4 mHC boundary for ranged ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-046
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.
|
||||
|
||||
- [ ] Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.
|
||||
- [ ] Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.
|
||||
- [ ] Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.
|
||||
- [ ] Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-048: Carry token-ID sideband through the first three hash-routed layers
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/048-carry-token-id-sideband-through-the-first-three-hash-routed-layers.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure any stage containing the first three hash-routed layers receives exact token identities.
|
||||
|
||||
- [ ] Head emits token IDs in the typed sideband for boundaries that require them.
|
||||
- [ ] Intermediate stages preserve or consume the sideband according to their layer range.
|
||||
- [ ] Routes splitting before, within, and after the first three layers pass deterministic partition tests.
|
||||
- [ ] Missing or mismatched token IDs fail closed rather than silently selecting incorrect experts.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-049: Keep V4 attention and auxiliary state shard-local
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/049-keep-v4-attention-and-auxiliary-state-shard-local.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Bind V4 CSA/HCA/SWA/indexer/compressor and KV state to the owning shard session/epoch rather than transferring caches over the WAN.
|
||||
|
||||
- [ ] Map CSA, HCA, SWA, indexer, compressor, and KV ownership from the pinned upstream graph to each owned layer range.
|
||||
- [ ] Key all such caches/state by route session and epoch with bounded lifecycle, memory accounting, stale-epoch rejection, release, and eviction.
|
||||
- [ ] Prove cross-stage messages carry only the typed architecture boundary and cache expectations, never per-layer cache contents.
|
||||
- [ ] On route change or state loss, use explicit cache miss and re-prefill/restart; do not migrate V4 caches across workers.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-050: Validate upstream V4 MoE and hash-routing execution under ranged ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/050-validate-upstream-v4-moe-and-hash-routing-execution-under-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Validate the pinned upstream llama.cpp V4 MoE and hash-routing graph when layers are range-owned, without reimplementing routing or expert kernels in Meshnet.
|
||||
|
||||
- [ ] Load range-owned tensors for 256 routed plus one shared expert and validate upstream execution activates six routed experts where specified.
|
||||
- [ ] Validate the first three upstream hash-routed layers receive exact token-ID sideband across splits before, within, and after them.
|
||||
- [ ] Compare upstream whole-model and ranged router/expert outputs, shared-expert contribution, and malformed sideband failures.
|
||||
- [ ] Document that llama.cpp owns V4 MoE, attention, hash routing, graph, and kernels; no expert-parallel WAN collective or Meshnet reimplementation is introduced.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-051: Assemble the DeepSeek V4 Flash `ShardEngine` adapter
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/051-assemble-the-deepseek-v4-flash-shardengine-adapter.md`, and evidence READMEs for dependencies (DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Assemble range ownership hooks and boundary/local-state adapters around upstream llama.cpp V4 execution behind the stable ShardEngine interface.
|
||||
|
||||
- [ ] Support valid head, middle, tail, and whole-model ranges over all 43 main layers using the pinned upstream llama.cpp V4 graph/kernels.
|
||||
- [ ] Integrate exact range ownership, mHC boundary, positions, required token-ID sideband, and shard-local session/epoch state adapters.
|
||||
- [ ] Report MTP reserved-but-not-executable for alpha and reject unsupported split points or cache expectations.
|
||||
- [ ] Include exact adapter/schema/runtime/artifact identity and valid split restrictions without reimplementing V4 MoE, attention, mHC, or hash routing.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-052: Pass local V4 whole-model versus ranged parity
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-051
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/052-pass-local-v4-whole-model-versus-ranged-parity.md`, and evidence READMEs for dependencies (DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4 prefill and decode correctness locally before network certification.
|
||||
|
||||
- [ ] Compare pinned whole-model llama.cpp with at least two valid ranged partitions.
|
||||
- [ ] Prefill intermediates and locked-length greedy decode pass architecture-specific tolerances.
|
||||
- [ ] Test splits around the first three hash-routed layers and at least one CSA/HCA/SWA/indexer/compressor transition.
|
||||
- [ ] Four concurrent sessions pass KV/state isolation and cleanup.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-053: Certify a real 2–4-stage V4 route
|
||||
|
||||
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-043, DGR-052
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.
|
||||
|
||||
- [ ] Run one documented 2–4-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.
|
||||
- [ ] Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.
|
||||
- [ ] Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.
|
||||
- [ ] Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.
|
||||
- [ ] Only the exact real-hardware lanes exercised become eligible for advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-054: Enforce the DeepSeek V4 Flash alpha gate
|
||||
|
||||
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/054-enforce-the-deepseek-v4-flash-alpha-gate.md`, and evidence READMEs for dependencies (DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Release alpha only when V4 is correct and clears the pre-locked useful-speed requirement.
|
||||
|
||||
- [ ] Evaluate correctness, model fit, TTFT, prefill/decode speed, seam cost, memory, and failure rate against DGF-004 without changing thresholds.
|
||||
- [ ] Separate quant/model-fit gains from runtime/transport gains.
|
||||
- [ ] Decision is `alpha`, `optimize measured bottleneck`, or `stop`; failures cannot be relabeled as certification.
|
||||
- [ ] Alpha documentation states MTP is reserved but not implemented and identifies every dark hardware lane.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-055: Implement compatible continuous decode batching
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-040, DGR-054
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/055-implement-compatible-continuous-decode-batching.md`, and evidence READMEs for dependencies (DGR-038, DGR-040, DGR-054) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Batch compatible active sessions without violating route, recipe, position, or architecture state.
|
||||
|
||||
- [ ] Batch only sessions with compatible engine, recipe, phase, boundary schema, and state layout.
|
||||
- [ ] Preserve per-session positions, sequence IDs, outputs, cancellation, and accounting.
|
||||
- [ ] Decode receives bounded service while prefill remains schedulable.
|
||||
- [ ] Deterministic 1/2/4/8 concurrency tests show no cross-session corruption.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-056: Enforce bounded admission, queues, and backpressure
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/056-enforce-bounded-admission-queues-and-backpressure.md`, and evidence READMEs for dependencies (DGR-055) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prevent weight, KV, state, scratch, and stream buffers from exceeding admitted capacity.
|
||||
|
||||
- [ ] Admission accounts separately for weights, KV/state, scratch, in-flight fragments, and queue slots.
|
||||
- [ ] Queue and stream limits produce structured retryable or terminal responses.
|
||||
- [ ] Prefill cannot starve decode and one route session cannot monopolize queue capacity.
|
||||
- [ ] Telemetry reports active sessions, queue depth/time, batch occupancy, memory pressure, and rejects.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-057: Benchmark batching saturation and fairness
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055, DGR-056
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/057-benchmark-batching-saturation-and-fairness.md`, and evidence READMEs for dependencies (DGR-055, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Identify useful concurrency and saturation points without assuming one global batch size.
|
||||
|
||||
- [ ] Run controlled concurrency 1/2/4/8 and record aggregate throughput, per-request p50/p95, TTFT, queue time, occupancy, and memory.
|
||||
- [ ] Measure dense and V4 lanes separately where hardware permits.
|
||||
- [ ] Verify bounded fairness and no prefill/decode starvation.
|
||||
- [ ] Produce machine-readable saturation recommendations consumed by capability reporting, not hardcoded constants.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-058: Harden worker and stream failure semantics
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-040, DGR-042, DGR-056
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/058-harden-worker-and-stream-failure-semantics.md`, and evidence READMEs for dependencies (DGR-040, DGR-042, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure malformed input, process death, transport reset, and cancellation terminate cleanly.
|
||||
|
||||
- [ ] Test worker death, stream reset, malformed bundle, checksum failure, deadline, cache miss, stale epoch, and partial fragment loss.
|
||||
- [ ] Cancellation propagates across all stages and releases KV/state, queue entries, and fragment buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never silently replayed.
|
||||
- [ ] Billing/work outcomes distinguish completed, cancelled, failed, and unverified work.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-059: Implement alpha-safe route recovery
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/059-implement-alpha-safe-route-recovery.md`, and evidence READMEs for dependencies (DGR-043, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Recover from route loss without importing or continuing unverified state.
|
||||
|
||||
- [ ] On route loss or local-state cache miss, restart from token zero or re-prefill on a newly compatible route according to the locked alpha contract.
|
||||
- [ ] Never import KV, CSA, HCA, SWA, indexer, compressor, recurrent, or MTP cache/state across workers.
|
||||
- [ ] Keep exact same-route duplicate delivery idempotent while cross-route continuation fails closed.
|
||||
- [ ] Test direct/relay loss, cache miss, replacement selection, client cancellation, and billing reconciliation.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-060: Certify V4 long-context state correctness
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-056, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4’s KV and auxiliary state remain correct and bounded at long contexts.
|
||||
|
||||
- [ ] Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.
|
||||
- [ ] Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.
|
||||
- [ ] Measure memory growth, fragment sizes, TTFT, decode degradation, cancellation latency, and cleanup.
|
||||
- [ ] Cache mismatch, truncation, and context-limit behavior fail deterministically.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-061: Certify existing routing with 10+ GGUF stage candidates
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-057, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/061-certify-existing-routing-with-10-gguf-stage-candidates.md`, and evidence READMEs for dependencies (DGR-043, DGR-057, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Certify that existing backend-agnostic routing consumes GGUF compatibility and measured-cost inputs for a 10+ candidate scenario without algorithm or policy redesign.
|
||||
|
||||
- [ ] Generate deterministic pools with 10+ compatible GGUF stage candidates plus slower, overloaded, unhealthy, and incompatible alternatives.
|
||||
- [ ] Run the existing routing implementation unchanged and verify complete coverage, compatibility, acyclicity, and use of measured inputs.
|
||||
- [ ] Regression-test that no stage count, quant, fixed split, architecture, or backend sequence is encoded in production routing policy.
|
||||
- [ ] Verify existing route recomputation reacts to measured load/failure while tracker, load-balancing, billing, relay, and provider semantics remain unchanged.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-062: Certify a dynamic real 10+ stage V4 scenario
|
||||
|
||||
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-053, DGR-060, DGR-061
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/062-certify-a-dynamic-real-10-stage-v4-scenario.md`, and evidence READMEs for dependencies (DGR-053, DGR-060, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Demonstrate that a large real route can be dynamically selected and usefully execute V4.
|
||||
|
||||
- [ ] Run one real 10+-stage V4 certification scenario; stage count, quant, and split remain scenario metadata only.
|
||||
- [ ] Every selected stage proves real work, exact compatible identity, local state ownership, and bounded resources.
|
||||
- [ ] Record end-to-end and per-stage TTFT, prefill/decode, seam cost, queueing, memory, failures, and cleanup.
|
||||
- [ ] Inject at least one incompatible or degraded candidate and verify dynamic exclusion/reselection.
|
||||
- [ ] The scenario must meet its pre-locked usefulness threshold or remain failed evidence.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-063: Profile and optimize the critical seam
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-057, DGR-061
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/063-profile-and-optimize-the-critical-seam.md`, and evidence READMEs for dependencies (DGR-057, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Optimize only the measured dominant compute, serialization, copy, or transport bottleneck.
|
||||
|
||||
- [ ] Profile worker compute, tensor conversion, copies, protobuf framing, compression, relay/direct transport, queueing, and synchronization.
|
||||
- [ ] Select one dominant bottleneck from evidence and lock its before/after benchmark.
|
||||
- [ ] Implement one bounded optimization without weakening correctness or protocol identity.
|
||||
- [ ] Report whether the optimization changes TTFT, prefill, decode, throughput, or memory and preserve raw data.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-064: Add negotiated activation compression
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-060, DGR-063
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/064-add-negotiated-activation-compression.md`, and evidence READMEs for dependencies (DGR-060, DGR-063) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce V4 seam cost only when a certified compression recipe improves end-to-end performance acceptably.
|
||||
|
||||
- [ ] Negotiate compression through recipe identity; unnegotiated or mismatched compression fails closed.
|
||||
- [ ] Preserve names, shapes, semantic axes, token-ID sideband, and checksums.
|
||||
- [ ] Measure quality drift, encode/decode cost, bytes, TTFT, and throughput against uncompressed boundaries.
|
||||
- [ ] Compression remains dark unless it meets pre-locked quality and speed thresholds.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-065: Define V4 MTP ownership and distributed state contract
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-054, DGR-060
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/065-define-v4-mtp-ownership-and-distributed-state-contract.md`, and evidence READMEs for dependencies (DGR-051, DGR-054, DGR-060) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Lock MTP tensor, execution, boundary, acceptance/rejection, and session-state ownership before implementation while keeping MTP disabled for alpha.
|
||||
|
||||
- [ ] Derive exact MTP tensor and execution ownership from the pinned V4 target and upstream llama.cpp graph, distinguishing main-layer, tail, and MTP responsibilities.
|
||||
- [ ] Define typed MTP inputs/outputs plus accepted-token, rejection, rollback/truncation, position, cache, and session/epoch lifecycle semantics.
|
||||
- [ ] Specify which MTP state remains shard-local and which typed boundary values cross a seam; prohibit WAN migration of KV or V4 auxiliary caches.
|
||||
- [ ] Keep every MTP recipe disabled and unroutable for alpha, with fail-closed capability negotiation and deterministic contract fixtures.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-066: Implement and benchmark V4 MTP
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-054, DGR-060, DGR-064, DGR-065
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/066-implement-and-benchmark-v4-mtp.md`, and evidence READMEs for dependencies (DGR-054, DGR-060, DGR-064, DGR-065) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Implement the locked MTP contract around upstream llama.cpp before beta, without making MTP an alpha prerequisite.
|
||||
|
||||
- [ ] Implement the exact DGR-065 MTP ownership and typed state contract against the pinned upstream V4 graph while preserving shard-local caches.
|
||||
- [ ] Pass whole-model versus ranged MTP correctness, accepted-token, rejection, rollback/truncation, and session-isolation tests.
|
||||
- [ ] Benchmark MTP-off versus MTP-on quality, accepted-token rate, TTFT, decode speed, seam bytes, and memory on the locked lane.
|
||||
- [ ] Distinguish implemented, certified, disabled, and unsupported MTP recipes in fail-closed capability advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-067: Certify the backend capability matrix
|
||||
|
||||
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-053, DGR-062, DGR-066
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/067-certify-the-backend-capability-matrix.md`, and evidence READMEs for dependencies (DGR-030, DGR-053, DGR-062, DGR-066) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Publish only backend/model/recipe lanes proven on actual CPU, CUDA, ROCm, Vulkan, or Metal hardware.
|
||||
|
||||
- [ ] Maintain independent build and certification status for CPU, CUDA, ROCm, Vulkan, and Metal.
|
||||
- [ ] Each advertised lane includes real hardware/driver/runtime/model evidence for parity, concurrency, cancellation, cleanup, and useful speed.
|
||||
- [ ] Build success alone never makes a lane routable; unavailable hardware remains explicitly dark.
|
||||
- [ ] Tracker admission consumes signed/versioned certification records rather than backend-name allowlists.
|
||||
- [ ] Mixed-backend routes require exact compatibility evidence and fail closed otherwise.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-068: Package reproducible native worker releases
|
||||
|
||||
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-030, DGR-054, DGR-067
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/068-package-reproducible-native-worker-releases.md`, and evidence READMEs for dependencies (DGR-030, DGR-054, DGR-067) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Produce installable worker packages that preserve exact source, patch, protocol, and certification identity.
|
||||
|
||||
- [ ] Package worker binaries/configuration for supported host/backend combinations without bundling an unverified llama.cpp tree.
|
||||
- [ ] Embed protocol, engine, upstream pin, patch-stack, build-toolchain, and certification fingerprints.
|
||||
- [ ] Installation/startup refuses mismatched artifacts, runtime recipes, or unsupported hardware lanes.
|
||||
- [ ] Produce checksums, SBOM/license attribution, reproducible build instructions, and smoke tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-069: Prepare narrow upstream llama.cpp collaboration patches
|
||||
|
||||
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-028, DGR-035, DGR-038, DGR-051
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/069-prepare-narrow-upstream-llama-cpp-collaboration-patches.md`, and evidence READMEs for dependencies (DGR-028, DGR-035, DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce long-term patch burden through generic upstreamable hooks without asking llama.cpp to own Meshnet.
|
||||
|
||||
- [ ] Separate generic range loading, boundary I/O, and filtered state hooks from project worker/protocol code.
|
||||
- [ ] Provide one scoped patch, focused test, and minimal reproducer per concern against the exact upstream pin.
|
||||
- [ ] Compare the proposal with prior-art evidence and current upstream V4 support/maturity.
|
||||
- [ ] Prepare human-ready design/outreach text; actual issue/PR submission remains a human action.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-070: Enforce the DeepSeek V4 Flash beta gate
|
||||
|
||||
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/070-enforce-the-deepseek-v4-flash-beta-gate.md`, and evidence READMEs for dependencies (DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Promote beta only after concurrency, resilience, long-context, MTP, scale, packaging, and speed gates pass.
|
||||
|
||||
- [ ] Evaluate all beta metrics against DGF-004 without changing thresholds after observing results.
|
||||
- [ ] Require V4 parity, batching/fairness, bounded backpressure, failure recovery, long-context correctness, MTP evidence, and dynamic 10+ stage evidence.
|
||||
- [ ] Publish the exact certified hardware/backend/recipe matrix; all other lanes remain dark.
|
||||
- [ ] Decision is `beta`, `targeted optimization`, or `stop/rollback`, with unresolved failures listed explicitly.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-071: Establish upstream pin, patch, and certification maintenance
|
||||
|
||||
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-069, DGR-070
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/071-establish-upstream-pin-patch-and-certification-maintenance.md`, and evidence READMEs for dependencies (DGR-069, DGR-070) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Keep beta reproducible and detect upstream/model/hardware drift before it reaches routable nodes.
|
||||
|
||||
- [ ] Add a documented update procedure for llama.cpp pin, numbered patches, protocol/engine versions, V4 target revision, and certification records.
|
||||
- [ ] A candidate pin update must apply/build/test in isolation and cannot replace the production pin automatically.
|
||||
- [ ] Any changed graph, tensor inventory, boundary, cache/state layout, backend behavior, or performance invalidates affected certification.
|
||||
- [ ] Add scheduled model-free checks plus opt-in real-hardware recertification instructions and rollback procedure.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Replacing or redesigning Tracker routing/load-balancing, billing, relay, or provider policy.
|
||||
- Reimplementing DeepSeek V4 architecture graphs or kernels in Meshnet.
|
||||
- Hardcoding a quant, stage count, node count, split, architecture, or backend sequence.
|
||||
- WAN migration of KV or V4 auxiliary caches.
|
||||
- Advertising a compile-only or fixture-only hardware lane.
|
||||
- Making MTP an alpha dependency.
|
||||
|
||||
57
.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
Normal file
57
.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Ralph context: Distributed GGUF Runtime
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
## Mandatory startup for every fresh story
|
||||
|
||||
1. Read this file and authoritative `prd.json` completely.
|
||||
2. Read the generated source issue named in the selected story description.
|
||||
3. Read every dependency evidence README; legacy DGR-001..016 evidence is provenance only.
|
||||
4. Read `docs/adr/0024-distributed-gguf-runtime.md`, root `CONTEXT.md`, `.claude/memory/MEMORY.md`, and relevant live source/tests.
|
||||
5. Inspect `git status`; preserve unrelated work. Never infer implementation from planning text or old pass states.
|
||||
6. If blocked or oversized, keep `passes: false` and write an honest `BLOCKED.md`/`DECOMPOSITION.md`; never weaken criteria or fabricate evidence.
|
||||
|
||||
## Locked scope
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
## Target identities
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
## Control/data-plane contract
|
||||
|
||||
Meshnet continues to own registration, coverage, existing route selection/load balancing, route epochs/sessions, direct/relay behavior, capability admission, cancellation, telemetry, billing, validation, and attribution. The GGUF adapter exposes measured inputs to those existing mechanisms. Direct seams use long-lived gRPC streams; relay seams carry byte-identical protobuf frames opaquely.
|
||||
|
||||
The project-owned `ShardEngine` hides llama.cpp internals. A worker loads one exact artifact/recipe/range identity. Default tests use fake/tiny fixtures. Real runs are opt-in, preserve raw metrics, and never download models under `/home`.
|
||||
|
||||
## Gitea issue synchronization
|
||||
|
||||
Gitea is a projection of `prd.json`, never a competing source of truth. Before and after every supervised Ralph run, invoke:
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_gitea_sync.py sync
|
||||
```
|
||||
|
||||
For a complete Ralph invocation with automatic state reconciliation, use:
|
||||
|
||||
```bash
|
||||
scripts/ralph-gitea-run.sh ralph-tui run --prd .scratch/distributed-gguf-runtime/prd.json --agent claude --model sonnet --iterations 1 --no-tui --no-setup --direct-merge --no-sandbox
|
||||
```
|
||||
|
||||
The sync creates/reconciles one Gitea issue per `DGR-*` story, creates missing labels/milestones, closes issues whose `passes` is true, marks the selected next eligible story `status:in-progress`, and marks blocked stories `status:blocked`. `gitea-issues.json` is a derived mapping only.
|
||||
|
||||
## Evidence and completion
|
||||
|
||||
Each story writes `/run/media/popov/d/DEV/repos/d-popov.com/AI/.claude/worktrees/distributed-gguf-runtime/.scratch/distributed-gguf-runtime/evidence/<DGR-ID>/README.md` with exact files, commands/results, limitations, identities, and dependent-story handoff. Only `prd.json` may record `passes`; DGR-017 and DGR-018 are complete and DGR-019 onward remain false. Generated Markdown and Gitea issues cannot override it. One scoped commit per story is expected during future execution.
|
||||
@@ -1,63 +1,32 @@
|
||||
# Distributed GGUF runtime — planning index
|
||||
# Distributed GGUF Runtime planning workspace
|
||||
|
||||
Status: draft scratch package.
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
Goal: make the node network capable of serving large, high-quality open models by distributing GGUF/model artifacts over a torrent-style swarm while executing inference over a sticky multi-node route with per-shard local KV cache.
|
||||
|
||||
This scratch supersedes the old assumption in [ADR-0001](../../docs/adr/0001-pytorch-over-llama-cpp.md) that llama.cpp is only a single-node leaf backend. That assumption was correct for the original llama.cpp RPC shape, but the target is now different: torrent-distributed GGUF artifacts plus an explicit route/KV protocol owned by this platform, ideally developed in collaboration with upstream llama.cpp.
|
||||
## Locked scope
|
||||
|
||||
## Artifacts
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| [architecture.md](./architecture.md) | Proposed runtime architecture, data flow, session state, and failure model |
|
||||
| [technical-challenges.md](./technical-challenges.md) | Detailed challenge/solution register with acceptance tests |
|
||||
| [decision-framework.md](./decision-framework.md) | Grilling framework for open decisions and recommended answers |
|
||||
| [research-prior-art.md](./research-prior-art.md) | Prior-art notes for Petals, exo, Distributed Llama, prima.cpp, llama.cpp, DeepSeek-V4-Flash, GLM-5.2, and Ornith |
|
||||
| [ADR-0020-distributed-gguf-runtime.md](./ADR-0020-distributed-gguf-runtime.md) | Draft decision record for the GGUF/llama.cpp distributed runtime |
|
||||
| [PRD.md](./PRD.md) | Product/runtime requirements and acceptance criteria |
|
||||
| [milestones.md](./milestones.md) | Dependency-ordered implementation milestones |
|
||||
| [issues/](./issues/) | Implementation-ready tracer-bullet issue briefs |
|
||||
## Target identities
|
||||
|
||||
## Decision Summary
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
Adopt a hybrid runtime:
|
||||
## Navigation
|
||||
|
||||
- **Weights and artifacts**: distributed by torrent / content-addressed storage / optional CDN.
|
||||
- **Hot KV cache**: local to the node that owns the corresponding layer range.
|
||||
- **Prefix snapshots**: optionally persisted to cache servers for reuse, retry, and failover.
|
||||
- **Active route**: sticky for one request/session.
|
||||
- **Context cap**: 128K hard product limit for large models unless explicitly revised.
|
||||
- **Backends**: keep PyTorch for fast model-architecture coverage and validation; add llama.cpp/GGUF as the performance path for supported models.
|
||||
- **Client feedback**: stream token deltas when feasible; always expose Generation Telemetry.
|
||||
- **First serious target model**: DeepSeek-V4-Flash after a smaller GGUF protocol smoke test.
|
||||
|
||||
## What We Learned
|
||||
|
||||
- Our current full-model PyTorch path uses Transformers `generate()` and gets local KV cache.
|
||||
- Our current distributed PyTorch path disables cache and recomputes the full growing sequence per token.
|
||||
- The seam today carries hidden activations, not KV cache; at 128K this becomes impossible for serious models if repeated every decode token.
|
||||
- The missing capability is not "send KV across the network"; it is **stable per-session local KV cache per shard**.
|
||||
- GGUF distribution is solved enough at the artifact layer, but GGUF/llama.cpp needs explicit layer-boundary execution APIs for our route model.
|
||||
|
||||
## Recommended Order
|
||||
|
||||
See [milestones.md](./milestones.md) for the full dependency map.
|
||||
|
||||
1. [01 — Route Session lifecycle](./issues/01-route-session-lifecycle.md)
|
||||
2. [02 — Prefill/decode binary HTTP protocol](./issues/02-prefill-decode-binary-http.md)
|
||||
3. [03 — Generation Telemetry and streaming response contract](./issues/03-generation-telemetry-and-streaming.md)
|
||||
4. [04 — PyTorch distributed KV reference route](./issues/04-pytorch-distributed-kv-reference.md)
|
||||
5. [05 — Local llama.cpp/GGUF backend](./issues/05-local-llamacpp-gguf-backend.md)
|
||||
6. [06 — Model Artifact manifest and Shard advertisement](./issues/06-model-artifact-manifest.md)
|
||||
7. [07 — llama.cpp layer-boundary prototype](./issues/07-llamacpp-layer-boundary-prototype.md)
|
||||
8. [08 — Networked distributed GGUF route](./issues/08-networked-distributed-gguf-route.md)
|
||||
9. [09 — DeepSeek-V4-Flash support audit](./issues/09-deepseek-v4-flash-support-audit.md)
|
||||
10. [10 — GLM-5.2 and Ornith follow-up support audit](./issues/10-glm52-ornith-followup-audit.md)
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Does upstream llama.cpp already expose enough internal API for arbitrary layer-range execution and hidden-state boundary I/O, or do we need an extension?
|
||||
- Can GGUF split metadata be made layer/tensor semantic enough for torrent placement and partial loading?
|
||||
- What is the minimum protocol needed for compressed KV formats such as GLM-5.2 DSA/MLA without exposing model-specific internals to the tracker?
|
||||
- How much reliability do we need in alpha: fail request on route loss, or support route repair with KV snapshots?
|
||||
- [`prd.json`](prd.json) — sole authoritative 55-story backlog, DGR-017..071.
|
||||
- [`PRD.md`](PRD.md) — human-readable projection of goals, gates, and all stories.
|
||||
- [`RALPH-CONTEXT.md`](RALPH-CONTEXT.md) — mandatory fresh-session context.
|
||||
- [`architecture.md`](architecture.md), [`implementation-strategy.md`](implementation-strategy.md), [`milestones.md`](milestones.md) — design and execution sequence.
|
||||
- [`issues/`](issues/) — generated story specs; files 01..16 are retained legacy artifacts pending DGR-017.
|
||||
- [`evidence/`](evidence/) — provenance and future per-story handoffs.
|
||||
|
||||
@@ -1,274 +1,45 @@
|
||||
# Distributed GGUF Runtime Architecture
|
||||
# Distributed GGUF Runtime architecture
|
||||
|
||||
## Product Stance
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
The platform optimizes for access to high-quality models, not lowest latency. Latency is acceptable if the user can run models that are otherwise unavailable to them. The hard context limit for the first serious distributed runtime should be **128K tokens**. Longer context usually means the product is compensating for missing task decomposition, retrieval, or workspace summarization.
|
||||
|
||||
## Current State
|
||||
## Locked scope
|
||||
|
||||
The current node has two materially different inference paths:
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
- **Full local PyTorch model**: calls Hugging Face `model.generate()`, so Transformers owns autoregressive decode and local KV cache.
|
||||
- **Distributed PyTorch route**: bypasses `model.generate()`, calls individual layers with `use_cache=False`, and recomputes the full growing sequence for every generated token.
|
||||
## Target identities
|
||||
|
||||
Current distributed data flow:
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
## Topology
|
||||
|
||||
```text
|
||||
client request
|
||||
-> head node formats prompt
|
||||
-> for each output token:
|
||||
head tokenizes full current text
|
||||
head runs early layers over all tokens
|
||||
head sends full activation [batch, sequence, hidden] to next node
|
||||
middle nodes run their layers over all tokens
|
||||
tail returns one decoded token string
|
||||
head appends token to text
|
||||
existing Meshnet Tracker/control plane
|
||||
-> existing backend-agnostic route/load-balancing decision
|
||||
-> direct gRPC or existing opaque relay
|
||||
-> project-owned standalone C++ Shard worker
|
||||
-> project-owned ShardEngine
|
||||
-> pinned upstream llama.cpp + numbered range/boundary/state hook patches
|
||||
-> GGUF mmap, upstream V4 graph/kernels, local per-shard state
|
||||
```
|
||||
|
||||
This is correct for small demos but not viable for large models. For GLM-5.2, a single 128K seam activation is roughly:
|
||||
A route is ordered contiguous half-open ranges. Head owns token embedding; tail owns final norm/head/sampling. Compatibility fingerprints bind source/split hashes, tokenizer, architecture adapter, typed boundary, runtime pin/patches, backend, quant, activation/compute/KV layout, range, and certification.
|
||||
|
||||
```text
|
||||
128K tokens * hidden_size 6144 * 2 bytes ~= 1.5 GiB per hop
|
||||
```
|
||||
## V4 boundary and state
|
||||
|
||||
Sending that every output token is the bottleneck.
|
||||
The inter-stage boundary is semantic and versioned: mHC 4×4096 residual, positions, token IDs only where the first three hash-routed layers require them, and cache/schema expectations. CSA/HCA/SWA/indexer/compressor/KV state belongs to upstream layer execution on the owning worker and is isolated by `(route_session_id, route_epoch)`. On loss, return cache miss and re-prefill/restart. Never serialize those caches into the WAN bundle.
|
||||
|
||||
## Target State
|
||||
## Concurrency, failure, and admission
|
||||
|
||||
Target distributed data flow:
|
||||
|
||||
```text
|
||||
client request
|
||||
-> tracker selects route and pins session
|
||||
-> head node creates session_id
|
||||
-> prefill:
|
||||
prompt is chunked
|
||||
each shard computes its layer range
|
||||
each shard appends local KV/state for its own layers
|
||||
activations cross only layer seams
|
||||
-> decode loop:
|
||||
head sends one new token / one-step hidden state
|
||||
each shard reads local KV/state for session_id
|
||||
each shard appends one step to local KV/state
|
||||
only one-step activation crosses seams
|
||||
tail returns logits/token
|
||||
```
|
||||
|
||||
The KV cache remains local to the node that computed it. It is not sent to the next node and not read from a remote cache server during every decode step.
|
||||
|
||||
## Client Feedback
|
||||
|
||||
Streaming responses are desirable when the backend and client transport support them. The product should stream token deltas when possible, and it must always provide realtime Generation Telemetry while the route is working.
|
||||
|
||||
The fallback behavior is a non-streaming final answer plus live telemetry. That fallback is acceptable for early route proofs or models/backends that cannot expose clean token deltas yet, but the preferred client experience is streamed output plus telemetry.
|
||||
|
||||
Minimum client-visible telemetry:
|
||||
|
||||
- route/session accepted
|
||||
- selected model and quantization
|
||||
- prefill phase started/completed
|
||||
- decode phase started
|
||||
- generated token count
|
||||
- rolling tokens per second
|
||||
- route health or retry/failure reason
|
||||
- estimated billing units when available
|
||||
|
||||
Implementation options:
|
||||
|
||||
- Server-Sent Events or WebSocket for realtime progress
|
||||
- polling endpoint for simple clients
|
||||
- OpenAI-compatible streaming for clients that require token deltas
|
||||
|
||||
This means "no token streaming" is acceptable only as a fallback. "Silent wait for minutes" is not acceptable.
|
||||
|
||||
## Artifact Plane
|
||||
|
||||
Artifact distribution is separate from execution.
|
||||
|
||||
```text
|
||||
model publisher
|
||||
-> produces model manifest
|
||||
-> creates GGUF / safetensors / tokenizer artifacts
|
||||
-> content-addresses every file/chunk
|
||||
-> publishes torrent/magnet + HTTP fallback metadata
|
||||
|
||||
node
|
||||
-> chooses model/layer range
|
||||
-> downloads needed files/chunks
|
||||
-> verifies hash
|
||||
-> advertises availability to tracker
|
||||
```
|
||||
|
||||
Required manifest fields:
|
||||
|
||||
- model id and version
|
||||
- upstream source repo and revision
|
||||
- license
|
||||
- architecture name
|
||||
- tokenizer files and hashes
|
||||
- quantization
|
||||
- tensor-to-layer map
|
||||
- file/chunk hashes
|
||||
- optional GGUF split files
|
||||
- supported runtime backends
|
||||
- context cap
|
||||
- KV/cache format descriptor
|
||||
|
||||
## Execution Plane
|
||||
|
||||
The tracker selects routes using layer coverage and observed performance:
|
||||
|
||||
```text
|
||||
route = [
|
||||
head node: embeddings + layers 0..k
|
||||
middle nodes: contiguous layer ranges
|
||||
tail node: final layers + norm + lm_head
|
||||
]
|
||||
```
|
||||
|
||||
Route selection inputs:
|
||||
|
||||
- model id/version/quantization
|
||||
- layer coverage
|
||||
- node hardware
|
||||
- measured prefill throughput
|
||||
- measured decode throughput
|
||||
- queue depth
|
||||
- latency to neighboring nodes
|
||||
- cache warmth for the requested prefix/session
|
||||
- reliability/reputation
|
||||
|
||||
The route is sticky for the request/session. A new route means either a fresh prefill or restoring compatible KV snapshots.
|
||||
|
||||
## KV Cache Ownership
|
||||
|
||||
KV/state ownership is by layer range:
|
||||
|
||||
```text
|
||||
session_id = request scoped id
|
||||
node A owns layers 0..15 KV for session_id
|
||||
node B owns layers 16..31 KV for session_id
|
||||
node C owns layers 32..77 KV for session_id
|
||||
```
|
||||
|
||||
The tracker does not own hot KV. It may know which nodes hold active KV for session accounting and failure handling.
|
||||
|
||||
Cache servers may store:
|
||||
|
||||
- prompt-prefix snapshots
|
||||
- session checkpoints for retry
|
||||
- cold reusable context blocks
|
||||
- audit samples
|
||||
|
||||
Cache servers must not be in the per-token hot loop unless colocated with the compute node.
|
||||
|
||||
## 128K KV Budget
|
||||
|
||||
GLM-5.2 compressed DSA/MLA-style estimate from config:
|
||||
|
||||
```text
|
||||
layers = 78
|
||||
kv_lora_rank = 512
|
||||
qk_rope_head_dim = 64
|
||||
dtype = bf16 = 2 bytes
|
||||
context = 128K
|
||||
|
||||
per_token ~= 78 * (512 + 64) * 2 = 89,856 bytes ~= 87.75 KiB
|
||||
128K total ~= 10.7 GiB
|
||||
per layer ~= 137 MiB
|
||||
```
|
||||
|
||||
This is feasible when sharded:
|
||||
|
||||
| Layer count | Approx active KV at 128K |
|
||||
|---:|---:|
|
||||
| 1 | 137 MiB |
|
||||
| 10 | 1.37 GiB |
|
||||
| 20 | 2.75 GiB |
|
||||
| 78 | 10.7 GiB |
|
||||
|
||||
The exact runtime value depends on implementation and cache quantization, but the order of magnitude is acceptable.
|
||||
|
||||
## Protocol Sketch
|
||||
|
||||
### Prefill
|
||||
|
||||
```http
|
||||
POST /v1/sessions/{session_id}/prefill
|
||||
Content-Type: application/octet-stream
|
||||
X-Meshnet-Model: zai-org/GLM-5.2
|
||||
X-Meshnet-Route-Id: ...
|
||||
X-Meshnet-Token-Range: 0-2047
|
||||
X-Meshnet-Shape: 1,2048,6144
|
||||
X-Meshnet-Dtype: bfloat16
|
||||
|
||||
<activation bytes>
|
||||
```
|
||||
|
||||
The receiver:
|
||||
|
||||
- validates route/session
|
||||
- runs assigned layer range for that chunk
|
||||
- appends local KV/state
|
||||
- forwards resulting activation to next hop
|
||||
|
||||
### Decode
|
||||
|
||||
```http
|
||||
POST /v1/sessions/{session_id}/decode-step
|
||||
Content-Type: application/octet-stream
|
||||
X-Meshnet-Model: zai-org/GLM-5.2
|
||||
X-Meshnet-Position: 131072
|
||||
X-Meshnet-Shape: 1,1,6144
|
||||
X-Meshnet-Dtype: bfloat16
|
||||
|
||||
<one-step activation bytes>
|
||||
```
|
||||
|
||||
The receiver:
|
||||
|
||||
- loads local KV/state by `session_id`
|
||||
- runs one decode step for assigned layers
|
||||
- appends one token position to local KV/state
|
||||
- forwards one-step activation
|
||||
|
||||
## GGUF / llama.cpp Integration
|
||||
|
||||
The target llama.cpp integration needs more than `llama-server`.
|
||||
|
||||
Required capabilities:
|
||||
|
||||
- load full GGUF locally for immediate single-node performance
|
||||
- optionally load only selected tensors/layers
|
||||
- execute a layer range against inbound hidden states
|
||||
- expose outbound hidden states at a boundary
|
||||
- own per-session KV/state for only the loaded layer range
|
||||
- support prefill chunks and decode-step calls
|
||||
- expose model-specific cache metadata for DSA/MLA without requiring the tracker to understand tensor internals
|
||||
|
||||
If llama.cpp cannot expose these as stable APIs today, the collaboration target is an upstream extension rather than a long-lived fork.
|
||||
|
||||
## Failure Model
|
||||
|
||||
Alpha behavior:
|
||||
|
||||
- Route node drops during prefill: fail request and retry from scratch.
|
||||
- Route node drops during decode: fail request unless a recent KV snapshot exists.
|
||||
- Tracker restart: active sessions may be lost; completed billing records persist.
|
||||
- Node restart: local hot KV is lost.
|
||||
|
||||
Later behavior:
|
||||
|
||||
- periodic KV snapshots for long sessions
|
||||
- prefix cache reuse across requests
|
||||
- route repair when a semantically equivalent node has the same model/layer range and compatible cache snapshot
|
||||
|
||||
## Security And Trust
|
||||
|
||||
Activation/KV data can reveal user prompts. Public volunteer routes are not private. For sensitive workloads:
|
||||
|
||||
- use private swarms
|
||||
- allow paid trusted nodes
|
||||
- encrypt transport
|
||||
- avoid storing hot KV on untrusted shared cache servers
|
||||
- sample outputs for fraud/audit as already planned in alpha hardening
|
||||
Compatible sessions may be continuously batched within a worker while retaining isolated positions/state. Admission bounds weights, local state/KV, scratch, fragments, and queues. Uncertain cross-route mutation is not replayed. Registration can show an uncertified lane, but existing admission keeps it unroutable until signed/versioned real-hardware evidence exists.
|
||||
|
||||
@@ -1,268 +1,40 @@
|
||||
# Distributed GGUF Decision Framework
|
||||
|
||||
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
|
||||
|
||||
## Core Vocabulary
|
||||
|
||||
Use the existing domain terms this way:
|
||||
|
||||
- **Shard**: contiguous transformer layer range. This is the compute, routing, cache, and reward unit.
|
||||
- **Shard Swarm**: storage/download group for artifacts needed by a shard.
|
||||
- **Inference Route**: ordered node sequence that covers all layers for one request.
|
||||
- **Route Session**: one active request bound to one inference route and stable session id.
|
||||
- **Hot KV State**: live per-shard cache held by the route node during a route session.
|
||||
- **Prefix Snapshot**: persisted route-session state used for reuse or failover, not the hot decode path.
|
||||
- **Artifact Manifest**: canonical mapping from model artifacts to semantic model parts and runtime support.
|
||||
- **Generation Telemetry**: realtime progress for a route session, including phase and tokens/sec, independent of whether token deltas are streamed.
|
||||
|
||||
## The Five Planes
|
||||
|
||||
### 1. Control Plane
|
||||
|
||||
Owner: Tracker.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- node registry
|
||||
- coverage map
|
||||
- route selection
|
||||
- rebalance directives
|
||||
- route-session creation
|
||||
- health and telemetry
|
||||
- client-visible Generation Telemetry
|
||||
- billing/audit records
|
||||
|
||||
Must not do:
|
||||
|
||||
- serve hot KV during every token
|
||||
- become the only place model artifacts can be fetched
|
||||
|
||||
### 2. Artifact Plane
|
||||
|
||||
Owner: Shard Swarms, local node storage, optional CDN/bootstrap mirrors.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- GGUF/safetensors/tokenizer download
|
||||
- content-addressed verification
|
||||
- local artifact inventory
|
||||
- artifact-to-layer mapping
|
||||
- cache eviction
|
||||
|
||||
Must not do:
|
||||
|
||||
- define execution order by file split alone
|
||||
- imply that a downloaded file chunk equals a Shard
|
||||
|
||||
### 3. Execution Plane
|
||||
|
||||
Owner: active Inference Route.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- chunked prefill
|
||||
- one-step decode
|
||||
- hidden-state transfer across activation seams
|
||||
- start-layer handling for overlapping shards
|
||||
- backpressure
|
||||
|
||||
Must not do:
|
||||
|
||||
- resend full context activations during decode
|
||||
- require cross-node tensor parallel all-reduce for public v1
|
||||
|
||||
### 4. Session State Plane
|
||||
|
||||
Owner: route nodes for hot KV; cache servers only for snapshots.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- per-shard local KV ownership
|
||||
- cache allocation and eviction
|
||||
- cache ABI compatibility
|
||||
- session close/release
|
||||
- optional prefix snapshots
|
||||
|
||||
Must not do:
|
||||
|
||||
- centralize hot KV in a remote service
|
||||
- let a replacement node continue from incompatible state
|
||||
|
||||
### 5. Economics And Trust Plane
|
||||
|
||||
Owner: tracker plus settlement/validation components.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- distinguish storage/seeding work from inference work
|
||||
- account for prefill and decode separately
|
||||
- record route participation
|
||||
- sample validation events
|
||||
- slash proven fraud
|
||||
|
||||
Must not do:
|
||||
|
||||
- pay a node for merely holding files as if it generated tokens
|
||||
- hide public-swarm privacy limits from clients
|
||||
|
||||
## Hard Invariants
|
||||
|
||||
These are the framework rules unless we deliberately write a new ADR:
|
||||
|
||||
1. Public-network Shards are contiguous layer ranges.
|
||||
2. Hot KV State is local to the node serving that Shard in that Route Session.
|
||||
3. Artifact distribution and route execution are separate systems.
|
||||
4. Decode seam payload must be `O(hidden_size)`.
|
||||
5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
|
||||
6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
|
||||
7. Model/backend-specific cache internals stay behind backend capability reports.
|
||||
8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
|
||||
9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
|
||||
|
||||
## Resolved Gates
|
||||
|
||||
### Gate 1: Public Shard Semantics
|
||||
|
||||
Decision: public-network Shards are contiguous transformer layer ranges. Tensor-parallel or ring-style execution is allowed only inside one trusted node, one colocated pod, or a future composite node abstraction.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Layer ranges match the existing `Shard`, `Coverage Map`, `Inference Route`, billing, and fraud vocabulary.
|
||||
- Public volunteer nodes should not require cross-node all-reduce or tight per-layer synchronization in v1.
|
||||
- Existing projects such as prima.cpp and Distributed Llama can still inform local-cluster/backend execution without becoming the public routing primitive.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Artifact Manifests must map files/tensors to semantic layer ranges.
|
||||
- Route selection remains ordered layer coverage.
|
||||
- Rewards can be attributed to layer-range work.
|
||||
- Hot KV State is naturally owned by the node serving that layer range for the Route Session.
|
||||
|
||||
### Gate 2: Hot KV Strategy
|
||||
|
||||
Decision: v1 rejects centralized hot KV. Hot KV State is local to the node serving the relevant Shard in the active Route Session. Cache servers may store Prefix Snapshots for reuse, retry, or failover, but they are not in the per-token decode path.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Decode is the tight loop; adding remote cache I/O there makes latency and bandwidth worse at the worst point.
|
||||
- Local KV naturally follows layer-range Shard ownership.
|
||||
- Centralized hot KV increases privacy exposure and creates consistency problems.
|
||||
- Prefix Snapshots preserve the useful part of central storage without making it mandatory for every generated token.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Route Session must be sticky.
|
||||
- Failover is limited in alpha unless a compatible Prefix Snapshot exists.
|
||||
- Cache servers are optimization infrastructure, not required runtime infrastructure.
|
||||
- Route repair requires compatible model revision, layer range, backend cache ABI, and snapshot position.
|
||||
|
||||
### Gate 3: First Runtime Proof
|
||||
|
||||
Decision: prove distributed Route Session and Hot KV State semantics in the existing PyTorch route before modifying llama.cpp/GGUF.
|
||||
|
||||
Rationale:
|
||||
|
||||
- PyTorch exposes model internals and cache objects more directly, so it is the fastest way to validate the distributed protocol.
|
||||
- The current distributed PyTorch route already has the right high-level shape but disables cache and recomputes full prompts.
|
||||
- Fixing that path gives us a reference implementation for correctness tests, telemetry, session lifecycle, and wire protocol behavior.
|
||||
- llama.cpp/GGUF should receive a clear target ABI rather than becoming both the protocol experiment and the performance backend at once.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Issue 02 precedes issue 05.
|
||||
- llama.cpp collaboration has a concrete target ABI.
|
||||
- The PyTorch route remains the architecture-coverage/reference backend even after GGUF becomes the preferred performance path.
|
||||
- The first success metric is eliminating full-prompt recompute in distributed decode.
|
||||
|
||||
### Gate 3A: Client Feedback During Latency
|
||||
|
||||
Decision: streaming responses are preferred when feasible, and realtime Generation Telemetry is required regardless of streaming support.
|
||||
|
||||
Rationale:
|
||||
|
||||
- The product optimizes for access to large capable models, so some latency is acceptable.
|
||||
- Users still need confidence that the route is alive and roughly how fast it is generating.
|
||||
- Streaming token deltas give the best user experience when the backend exposes them cleanly.
|
||||
- Tokens/sec remains useful during prefill, queueing, and any backend that cannot stream token deltas.
|
||||
|
||||
Consequences:
|
||||
|
||||
- The gateway should stream token deltas through an OpenAI-compatible response when possible.
|
||||
- The gateway must expose progress through SSE, WebSocket, or polling.
|
||||
- The final answer can be delivered after completion only as a fallback.
|
||||
- Telemetry must include route phase, generated token count, and rolling tokens/sec.
|
||||
- Non-streaming clients still need realtime telemetry.
|
||||
|
||||
### Gate 4: llama.cpp Collaboration Shape
|
||||
|
||||
Decision: target upstreamable `libllama`/ggml hooks instead of planning around a permanent fork.
|
||||
|
||||
Rationale:
|
||||
|
||||
- llama.cpp changes quickly across model support, quantization, kernels, and hardware backends.
|
||||
- A permanent fork would become expensive to maintain and would lag upstream improvements.
|
||||
- A short-lived prototype branch is acceptable if it proves the API and makes upstream collaboration concrete.
|
||||
- Keeping tracker/routing logic outside llama.cpp makes the upstream ask smaller and cleaner.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Need a minimal reproducible localhost demo before asking upstream to carry the design.
|
||||
- Need to separate "what llama.cpp should expose" from "what our tracker does".
|
||||
- Desired upstream surface is layer-range execution, hidden-state boundary I/O, partial loading/introspection, and per-session KV ownership.
|
||||
- If upstream rejects the shape, we revisit whether to carry a narrow adapter fork or keep GGUF distributed execution as experimental.
|
||||
|
||||
### Gate 5: First Model Target
|
||||
|
||||
Decision: use a two-tier model target. Use a small, boring, llama.cpp-supported GGUF model for the first protocol smoke test. Use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. Keep GLM-5.2 and Ornith as later support audits.
|
||||
|
||||
Rationale:
|
||||
|
||||
- The first protocol proof should isolate route/session/KV bugs from model-architecture bugs.
|
||||
- DeepSeek-V4-Flash is a strong first serious target because it is much smaller than 1.6T-class models while still being large enough to validate the product thesis.
|
||||
- DeepSeek-V4-Flash still has architecture-specific risks, so it should not be the first smoke test.
|
||||
- GLM-5.2 and Ornith remain valuable targets, but they add DSA/MLA/hybrid attention uncertainty.
|
||||
|
||||
Consequences:
|
||||
|
||||
- 128K cache accounting can be modeled now.
|
||||
- The first "real" target-model audit is DeepSeek-V4-Flash support in PyTorch, vLLM/SGLang, and any available GGUF/llama.cpp quantization path.
|
||||
- Production support waits for backend capability reports and exact cache ABI support.
|
||||
|
||||
### Gate 6: Failure Semantics
|
||||
|
||||
Decision: alpha fails Route Sessions on route-node loss instead of attempting automatic route repair.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Route repair requires compatible Prefix Snapshots, cache ABI checks, replacement-node selection, billing correction, and client stream/error recovery.
|
||||
- Local Hot KV State means a replacement node cannot continue unless it has compatible state at the same position.
|
||||
- Fail-fast keeps the first implementation correct while the session/KV protocol is still being proven.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Better observability and explicit errors are required.
|
||||
- Snapshotting becomes a later feature, not a blocker for first inference.
|
||||
- Generation Telemetry must report the last known phase and failure reason.
|
||||
- Client or gateway retry starts a new Route Session from scratch.
|
||||
|
||||
### Gate 7: Transport
|
||||
|
||||
Decision: keep binary HTTP for v1 activation transfer instead of jumping immediately to QUIC, WebRTC, or a custom transport.
|
||||
|
||||
Rationale:
|
||||
|
||||
- ADR-0008 already defines binary activation bodies with HTTP headers.
|
||||
- HTTP keeps the first implementation debuggable with the existing server stack and tooling.
|
||||
- The core risk is route/session/KV correctness, not transport optimization.
|
||||
- QUIC/WebRTC can be introduced later behind the same activation protocol once semantics are proven.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Focus benchmark work on payload shape, chunking, and cache behavior first.
|
||||
- QUIC/WebRTC can be introduced as an optimization behind the same activation protocol.
|
||||
- v1 implementation can reuse the current HTTP routing, relay, and observability infrastructure.
|
||||
- Transport abstraction should be kept narrow enough that HTTP can be replaced later without changing backend cache semantics.
|
||||
|
||||
## Grilling Progress
|
||||
|
||||
Gates 1, 2, 3, 3A, 4, 5, 6, and 7 are resolved. The remaining work is to convert the resolved framework into implementation-ready issue briefs and prototype milestones.
|
||||
# Distributed GGUF Runtime decision framework
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
## Decision order
|
||||
|
||||
1. DGR-019 locks comparable lanes and thresholds before results.
|
||||
2. DGR-020 runs safetensors and whole-model llama.cpp only, then returns `go`, `optimize baseline`, or `stop`.
|
||||
3. Dense and V4 work must prove parity, independent per-stage execution, local-state isolation, bounded failure, and measured resources.
|
||||
4. DGR-054 returns `alpha`, `optimize measured bottleneck`, or `stop`; MTP is explicitly off.
|
||||
5. Post-alpha optimizations must be selected from profiles, not assumptions.
|
||||
6. DGR-070 returns `beta`, `targeted optimization`, or `stop/rollback`, and requires MTP and the exact certified hardware/recipe matrix.
|
||||
|
||||
## Interpretation rules
|
||||
|
||||
- Quant/model-fit gains are separate from runtime/kernel/transport gains.
|
||||
- Fixture, real-model, real-hardware, and release evidence are never interchangeable.
|
||||
- 2–4 and 10+ stages are certification scenarios only.
|
||||
- Existing routing policy is certified, not redesigned.
|
||||
- Build success is not hardware certification; dark lanes remain unroutable.
|
||||
- Route loss uses cache miss and re-prefill/restart, never WAN cache migration.
|
||||
|
||||
## Locked scope
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
## Target identities
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
114
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
114
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# DGR-017 evidence — superseded backlog cleanup
|
||||
|
||||
**Completed:** 2026-07-16
|
||||
**Branch:** `ralph/distributed-gguf-runtime`
|
||||
**Planning checkpoint before cleanup:** `81b1fa6`
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
|
||||
## Outcome
|
||||
|
||||
The old DGR-001…016 completion claims and active artifacts were reconciled against the live branch. No old pass state transferred to the new implementation roadmap.
|
||||
|
||||
The active `packages/` and `tests/` trees were restored exactly to `origin/master`. The branch therefore no longer exposes a nominal GGUF startup path backed by unimplemented transport methods, a protobuf-only native scaffold, or isolated synthetic scheduler/cache/failure modules as if they were a working distributed GGUF runtime.
|
||||
|
||||
## Classification and disposition
|
||||
|
||||
### Retained
|
||||
|
||||
- Accepted ADRs and repository research, including `docs/research/colibri-implementation-audit.md`.
|
||||
- The authoritative 55-story roadmap `DGR-017…071` and its generated issue specifications.
|
||||
- The real public-relay smoke benchmark, moved with provenance to `legacy-public-relay-smoke-benchmark.json`.
|
||||
- Git history containing the complete superseded implementation/reference work.
|
||||
|
||||
### Removed from the active tree
|
||||
|
||||
- Legacy issue specifications DGR-001…016 and their stale/blocked/synthetic evidence directories.
|
||||
- The nonfunctional `gguf_backend` startup path whose gRPC execution methods raised not-implemented errors.
|
||||
- Synthetic/reference-only boundary, Hot KV, scheduler, failure, recipe, ownership, and native-protocol modules that were not a real llama.cpp Shard runtime.
|
||||
- The protobuf round-trip-only native scaffold, placeholder llama.cpp patch, generated bindings/build workspace, and associated tests.
|
||||
- Tracker/admission/source modifications coupled to that superseded scaffold.
|
||||
|
||||
### Confirmed absent and still required
|
||||
|
||||
- Real standalone C++ gRPC Shard worker.
|
||||
- Exact pinned llama.cpp manifest and verified patch stack.
|
||||
- Range-aware GGUF tensor ownership and real ranged execution.
|
||||
- Real Shard-local llama.cpp KV/V4 auxiliary state.
|
||||
- DeepSeek V4 boundary adapter and ranged parity.
|
||||
- Real multi-machine DeepSeek V4 alpha or beta acceptance.
|
||||
|
||||
These remain `passes: false` in DGR-018…071.
|
||||
|
||||
## Before-cleanup baseline
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
.venv-rocm/bin/python -m pytest -q \
|
||||
tests/test_performance_contract.py tests/test_native_shard_protocol.py \
|
||||
tests/test_gguf_ownership.py tests/test_boundary_adapter.py \
|
||||
tests/test_hot_kv_state.py tests/test_gguf_backend.py \
|
||||
tests/test_batch_scheduler.py tests/test_failure_semantics.py \
|
||||
tests/test_llama_worker_build.py tests/test_node_admission.py \
|
||||
tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
216 passed, 2 skipped, 1 failed, 1 warning
|
||||
```
|
||||
|
||||
The failure was a synthetic capability-test helper `KeyError: 'compatibility_fingerprint'`. The warning was a pre-existing heartbeat-thread `SystemExit` warning.
|
||||
|
||||
## Cleanup verification
|
||||
|
||||
### Source equality
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
git diff --quiet origin/master -- packages tests
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
packages_tests_match_origin_master=yes
|
||||
```
|
||||
|
||||
The staged cleanup removes approximately 15.3k obsolete source/test/evidence lines from the active branch.
|
||||
|
||||
### Cleanup-relevant regression suite
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
.venv-rocm/bin/python -m pytest -q \
|
||||
tests/test_node_admission.py tests/test_node_capability.py \
|
||||
tests/test_tracker_capability_admission.py \
|
||||
tests/test_kv_cache_distributed.py tests/test_real_distributed_inference.py
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
119 passed, 2 skipped, 1 warning in 15.90s
|
||||
```
|
||||
|
||||
The warning is the same pre-existing heartbeat-thread `SystemExit` warning.
|
||||
|
||||
### Known `origin/master` limitations
|
||||
|
||||
The wider routing run produced `210 passed, 2 skipped, 4 failed, 1 warning`. Each failure reproduced individually while `packages/` and `tests/` matched `origin/master` exactly:
|
||||
|
||||
- `test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`
|
||||
- `test_torch_node_applies_tracker_load_shard_directive`
|
||||
- `test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`
|
||||
- `test_a_node_with_an_unusable_precision_covers_no_layers`
|
||||
|
||||
They are recorded as pre-existing baseline defects and were not repaired or hidden by this cleanup story.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
DGR-018 and later stories must start from the cleaned upstream-equivalent runtime tree. Reuse concepts from superseded commits only by explicitly porting the smallest verified slice under the new story’s contracts, tests, and evidence gates. Git history is provenance, not completion evidence.
|
||||
@@ -0,0 +1,83 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"executed_at_utc": "2026-07-15T10:41:14Z",
|
||||
"test_kind": "public-relay-single-node-streaming-smoke-benchmark",
|
||||
"target": {
|
||||
"public_chat_endpoint": "https://meshnet.2.d-popov.com/v1/chat/completions",
|
||||
"relay_url": "wss://meshnet.2.d-popov.com/ws",
|
||||
"model": "qwen2.5-0.5b-instruct",
|
||||
"quantization": "bfloat16"
|
||||
},
|
||||
"recovery": {
|
||||
"problem": "The local node's capability proof had expired and its port-7000 HTTP server had wedged with CLOSE-WAIT sockets.",
|
||||
"action": "Gracefully restarted the local public-tracker meshnet-node process on port 7000.",
|
||||
"startup_validation": {
|
||||
"device": "cuda",
|
||||
"capability_proof_ms": 336,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"relay_addr": "wss://meshnet.2.d-popov.com/rpc/7j77FsPY1evV8tuf-7000"
|
||||
}
|
||||
},
|
||||
"tracker_admission_after_recovery": {
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"alive": true,
|
||||
"status": "ready",
|
||||
"capability_state": "admitted",
|
||||
"routable": true,
|
||||
"route_hops": 1
|
||||
},
|
||||
"client_measurements": {
|
||||
"warmup": {
|
||||
"http_status": 200,
|
||||
"ttft_ms": 420.8,
|
||||
"elapsed_ms": 610.23,
|
||||
"response_text": "MeshNet Relay Benchmark Passed"
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"run": 1,
|
||||
"ttft_ms": 376.04,
|
||||
"elapsed_ms": 458.65,
|
||||
"response_text": "relay benchmark pass"
|
||||
},
|
||||
{
|
||||
"run": 2,
|
||||
"ttft_ms": 258.33,
|
||||
"elapsed_ms": 336.71,
|
||||
"response_text": "relay benchmark pass"
|
||||
},
|
||||
{
|
||||
"run": 3,
|
||||
"ttft_ms": 288.26,
|
||||
"elapsed_ms": 363.2,
|
||||
"response_text": "relay benchmark pass"
|
||||
}
|
||||
],
|
||||
"p50_ttft_ms": 288.26,
|
||||
"p50_elapsed_ms": 363.2
|
||||
},
|
||||
"tracker_relay_evidence": [
|
||||
{
|
||||
"status": 200,
|
||||
"relay": true,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"tokens": 11,
|
||||
"elapsed_seconds": 0.1686,
|
||||
"tokens_per_sec": 65.2541
|
||||
},
|
||||
{
|
||||
"status": 200,
|
||||
"relay": true,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"tokens": 11,
|
||||
"elapsed_seconds": 0.1891,
|
||||
"tokens_per_sec": 58.1799
|
||||
}
|
||||
],
|
||||
"scope_and_remaining_work": {
|
||||
"validated": "Public HTTPS chat endpoint routed a streaming request through the tracker relay to the local CUDA node and completed with HTTP 200.",
|
||||
"not_validated": "Two-node shard routing was not run because the remote node 5gMLrmyB-88f5cba044d0 still had an expired capability proof and was not routable.",
|
||||
"next_gate": "Refresh the remote node capability proof, then load a multi-node-compatible assignment and repeat the benchmark through the public tracker relay."
|
||||
},
|
||||
"reproduction": "Use a valid bearer API key with the public /v1/chat/completions endpoint and stream a short qwen2.5-0.5b-instruct request. Do not connect directly to private node HTTP endpoints; the tracker relay is the required path."
|
||||
}
|
||||
204
.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md
Normal file
204
.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md
Normal file
@@ -0,0 +1,204 @@
|
||||
# DGR-018 evidence — canonical Ralph and Gitea metadata schema
|
||||
|
||||
**Completed:** 2026-07-16
|
||||
**Branch:** `ralph/distributed-gguf-runtime`
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.
|
||||
|
||||
## Objective
|
||||
|
||||
Make `prd.json` the validated source from which Markdown (and, later, Gitea) issues
|
||||
can be generated losslessly, per
|
||||
`.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`.
|
||||
|
||||
## Pre-existing state found (not caused by this story)
|
||||
|
||||
Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
|
||||
already modified in the working tree relative to `HEAD` (commit `369b207`), with no corresponding
|
||||
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
|
||||
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
|
||||
objects, while `userStories` itself was byte-identical to `HEAD`. This looked like an abandoned,
|
||||
uncommitted partial edit from a prior session, not intentional current work — those fields are
|
||||
exactly the schema/quality-gate/audit-provenance content this story depends on, and their loss
|
||||
wasn't explained by any acceptance criterion. They were restored (see "Changes" below) rather than
|
||||
silently accepted or discarded, per the instruction to investigate unexplained working-tree state
|
||||
before building on top of it.
|
||||
|
||||
## Changes
|
||||
|
||||
### `scripts/ralph_prd_schema.py` (new)
|
||||
|
||||
Single module providing:
|
||||
|
||||
- **Parse:** `load_prd(path)` — JSON load with clear `PrdValidationError`s for missing file /
|
||||
invalid JSON / non-object document.
|
||||
- **Canonical schema registry:** `STORY_FIELDS` (name → required/type), `EXECUTION_MODES`,
|
||||
`EVIDENCE_CLASSES`, `HARDWARE_FLAGS`, `UPSTREAM_FLAGS`, `TRIAGE_VALUES`. Covers every field named
|
||||
in the acceptance criteria: stable `id`/`title`, `labels`, `milestone`, derived `type`
|
||||
(`derive_type`), `dependsOn`, derived `blocks`, `triage`, `evidenceClass`, and
|
||||
`hardware`/`model`/`upstream` flags.
|
||||
- **Structural validation:** `validate_schema(data)` — required fields, types, enum membership,
|
||||
ID convention, `type:`/`priority:` label cardinality, non-empty `acceptanceCriteria`.
|
||||
- **Semantic validation:** `validate_semantics(data)` — unique IDs, unique titles, `dependsOn`
|
||||
resolves to known stories (no self-dependency), dependency graph is acyclic (with a reported
|
||||
cycle path on failure), `blocks` matches the dependency graph exactly (sorted set equality, not
|
||||
superset), and `evidencePath` matches the per-story convention.
|
||||
- **Fresh vs. in-progress backlog:** `validate_fresh_backlog(data)` additionally requires every
|
||||
story to start `passes: false` (for a backlog that hasn't started execution yet);
|
||||
`validate_backlog(data)` is the composed check for a real, in-flight backlog where some stories
|
||||
have legitimately completed.
|
||||
- **Self-consistency check:** `validate_metadata_schema_consistency(data)` — when prd.json declares
|
||||
its own `metadataSchema`/`qualityGates` (as this one now does), verifies that self-documentation
|
||||
hasn't drifted from what the validator actually enforces (enum sets, required/optional field
|
||||
lists, presence of `qualityGates` and `generatedArtifactDisclaimer`). This is a no-op for minimal
|
||||
fixture PRDs that don't carry that documentation.
|
||||
- **Generation (one-directional, prd.json → artifact):** `render_issue_markdown(story, data)`
|
||||
renders the exact Markdown convention already used by
|
||||
`.scratch/distributed-gguf-runtime/issues/*.md`, sourcing the "Shared quality gates" bullets from
|
||||
`data["qualityGates"]` and the leading disclaimer from
|
||||
`data["metadataSchema"]["generatedArtifactDisclaimer"]` (falling back to a module default only
|
||||
when `data` omits them) — not from a duplicated Python string literal.
|
||||
`to_gitea_issue_payload(story, data)` wraps the same body into a Gitea create-issue-shaped payload
|
||||
(`title`, `body`, `labels`, `milestone`).
|
||||
- **Authority guard:** `check_generated_markdown_authority(text, disclaimer=...)` rejects generated
|
||||
Markdown that's missing the disclaimer or that contains a conflicting authority claim (e.g. "this
|
||||
file is authoritative"). There is deliberately no Markdown → prd.json parser, so a generated
|
||||
artifact structurally cannot feed `passes` (or anything else) back into the authoritative source.
|
||||
- CLI: `python scripts/ralph_prd_schema.py validate <prd.json> [--fresh]` and
|
||||
`... render <prd.json> <STORY-ID>`.
|
||||
|
||||
### `.scratch/distributed-gguf-runtime/prd.json`
|
||||
|
||||
- Restored the top-level `sourceOfTruth`, `qualityGates`, `milestones`, and `supersededStories`
|
||||
objects to their `HEAD` content (see "Pre-existing state" above); `userStories` was already
|
||||
identical to `HEAD` and is unchanged in content.
|
||||
- Extended `metadataSchema` (previously incomplete for this story's own acceptance criteria) with:
|
||||
`requiredStoryFields` now also lists `notes` and `blocks` (present on all 55 stories); new
|
||||
`optionalStoryFields: ["completionNotes"]`; new `hardwareValues`/`upstreamValues` enums (`model`
|
||||
is documented as an open convention, not a closed enum, since quantization/model targets are
|
||||
dynamic recipe inputs per `RALPH-CONTEXT.md`); new `typeDerivation` and `labelConventions`
|
||||
(reserved prefixes, cardinality); new `generatedArtifactDisclaimer` (the exact string generated
|
||||
artifacts must start with); extended `dependencyRules`/`authorityRule` prose to match what the
|
||||
validator enforces.
|
||||
- Reworded `sourceOfTruth`'s stale "All stories are unimplemented ... passes=false" clause, which
|
||||
was no longer accurate once DGR-017 completed.
|
||||
- Marked `DGR-018.passes = true` with `completionNotes` recording this story's outcome.
|
||||
|
||||
### `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`
|
||||
|
||||
Regenerated via `render_issue_markdown` to reflect `passes: true` (checked acceptance criteria,
|
||||
"completed" status line, "Verified evidence" handoff line) — matching the same convention DGR-017's
|
||||
issue file already used for a completed story.
|
||||
|
||||
### `tests/test_ralph_prd_schema.py` (new)
|
||||
|
||||
108 deterministic, model-download-free, GPU-free tests:
|
||||
|
||||
- **Parse** (4 tests): real backlog parses to 55 stories; missing file, invalid JSON, and
|
||||
non-object documents raise `PrdValidationError`.
|
||||
- **Structural/semantic validation against the real backlog** (7 tests): passes `validate_schema`,
|
||||
`validate_semantics`, and the composed `validate_backlog`; unique IDs/titles; all `dependsOn`
|
||||
resolve; `blocks` matches the derived dependency graph for all 55 stories; no cycle; every
|
||||
`passes: true` story carries `completionNotes` and an existing evidence README (a durable
|
||||
invariant, not a hardcoded list of which stories have completed — that list will keep growing).
|
||||
- **Structural/semantic failure-mode fixtures** (13 tests): missing required field, bad enum, wrong
|
||||
type, empty `acceptanceCriteria`, multiple `type:` labels, duplicate ID, duplicate title, unknown
|
||||
dependency, self-dependency, dependency cycle, mismatched `blocks`, bad `evidencePath`.
|
||||
- **Fresh-backlog invariant** (3 tests): accepts all-`false`, rejects a premature `passes: true`,
|
||||
and confirms `validate_backlog` (the in-progress variant) permits completed stories.
|
||||
- **prd.json-as-source-of-truth for boilerplate** (9 tests): `qualityGates`/`metadataSchema`
|
||||
self-consistency checks (no-op without them, catches a drifted enum, catches a missing
|
||||
`qualityGates`), `quality_gate_bullets` flattening order, `authority_disclaimer` precedence and
|
||||
fallback, and 3 tests asserting the real backlog's declared schema matches the code, its 7
|
||||
quality-gate bullets are intact, and its disclaimer matches the module default.
|
||||
- **`derive_type`** (4 tests): label-derived type, release-gate synthetic type for HITL gate
|
||||
stories, `None` when absent, and confirmation that the real backlog's two release-gate stories
|
||||
(`DGR-054`, `DGR-070`) derive `release-gate`.
|
||||
- **Markdown generation round trips** (55 parametrized + 6 tests): `render_issue_markdown` for
|
||||
every story `DGR-017`..`DGR-071` is byte-for-byte identical to the corresponding file already in
|
||||
`.scratch/distributed-gguf-runtime/issues/`; determinism; leading disclaimer; `Blocks (derived)`
|
||||
rendering (`None` vs. listed); checkbox reflects `passes`; filename convention.
|
||||
- **Authority-claim rejection** (4 tests): accepts real generated text, rejects a missing
|
||||
disclaimer, rejects an overriding claim, and confirms every committed issue file in
|
||||
`.scratch/distributed-gguf-runtime/issues/` passes the check.
|
||||
- **Gitea payload generation** (3 tests): payload shape, body carries no information beyond what's
|
||||
in prd.json, and every real story's payload is well-formed and authority-clean.
|
||||
|
||||
## Commands and results
|
||||
|
||||
```bash
|
||||
python3 -m pytest -q tests/test_ralph_prd_schema.py
|
||||
```
|
||||
```text
|
||||
108 passed in 0.16s
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 -m compileall -q packages tests
|
||||
```
|
||||
Exit code 0, no output (all files compile).
|
||||
|
||||
```bash
|
||||
git diff --check
|
||||
```
|
||||
Exit code 0 (no whitespace errors).
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||
```
|
||||
```text
|
||||
OK: 55 stories validated.
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json --fresh
|
||||
```
|
||||
```text
|
||||
ERROR: DGR-017: fresh backlog requires passes=false, got True
|
||||
ERROR: DGR-018: fresh backlog requires passes=false, got True
|
||||
2 validation error(s).
|
||||
```
|
||||
Expected: `--fresh` is the invariant for a backlog that hasn't started execution; this backlog has
|
||||
legitimately completed two stories, so it correctly fails that stricter check while passing the
|
||||
plain (in-progress) `validate` command above.
|
||||
|
||||
### Baseline: full repository suite (ad hoc `python3`, not a project venv)
|
||||
|
||||
```bash
|
||||
python3 -m pytest -q
|
||||
```
|
||||
```text
|
||||
20 failed, 776 passed, 13 skipped, 2 warnings in 244.17s (0:04:04)
|
||||
```
|
||||
None of the failures touch `scripts/ralph_prd_schema.py` or `tests/test_ralph_prd_schema.py`
|
||||
(neither file existed before this story; this story adds no changes to `packages/`). Four of the
|
||||
20 failures reproduce exactly the pre-existing baseline defects DGR-017's evidence already recorded
|
||||
(`test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`,
|
||||
`test_torch_node_applies_tracker_load_shard_directive`,
|
||||
`test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`,
|
||||
`test_a_node_with_an_unusable_precision_covers_no_layers`). The remaining 16 (activation
|
||||
compression, dynamic routing, gossip/relay, manual route benchmark, openai gateway, TOPLoC
|
||||
calibration dispatch, tracker control plane) include a `ModuleNotFoundError: langchain` failure,
|
||||
indicating this ad hoc `python3` lacks the project's `dev` extras (`langchain-openai`, etc.) rather
|
||||
than a real regression; this environment has no project virtualenv (e.g. no `.venv-rocm`) to run
|
||||
against instead. Not investigated further as out of scope for this story.
|
||||
|
||||
## Limitations
|
||||
|
||||
- No real Gitea instance or API integration exists; `to_gitea_issue_payload` defines the payload
|
||||
shape (title/body/labels/milestone) only. Creating issues against a live Gitea server is future
|
||||
work, not claimed here.
|
||||
- `model` is intentionally validated as an open string, not a closed enum, per
|
||||
`RALPH-CONTEXT.md`'s "Quantization and placement are dynamic recipe inputs" constraint; the schema
|
||||
documents (`metadataSchema.modelConvention`) but does not restrict its value set.
|
||||
- Validation and generation were exercised only against this feature's `prd.json`
|
||||
(`.scratch/distributed-gguf-runtime/prd.json`); `docs/prd.json` and other `.scratch/*/prd.json`
|
||||
files in this repo use a materially different (simpler) shape and are out of scope.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
DGR-021 and DGR-025 (this story's derived `blocks`) may treat `prd.json`'s `metadataSchema`,
|
||||
`qualityGates`, and this validator/generator as stable. Any future field addition to a story shape
|
||||
must extend `STORY_FIELDS` in `scripts/ralph_prd_schema.py` and the corresponding
|
||||
`metadataSchema.requiredStoryFields`/`optionalStoryFields` in `prd.json` together —
|
||||
`validate_metadata_schema_consistency` fails closed if they drift apart.
|
||||
15
.scratch/distributed-gguf-runtime/evidence/README.md
Normal file
15
.scratch/distributed-gguf-runtime/evidence/README.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# Distributed GGUF Runtime evidence
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
## Authority and classes
|
||||
|
||||
Evidence supports but never overrides `prd.json`. Valid classes are `model-free`, `fixture`, `real-model`, `real-hardware`, and `release`; lower classes cannot satisfy higher-class acceptance. Legacy DGR-001..016 directories remain unchanged for DGR-017 provenance audit and confer no completion credit.
|
||||
|
||||
## Future story layout
|
||||
|
||||
Each DGR-017..071 story writes `evidence/<ID>/README.md` with summary, exact changed files, exact commands and real outputs, limitations, compatibility/migration notes, and dependent-story handoff. Machine-readable contracts, manifests, metrics, and raw logs live beside it. Never fabricate output.
|
||||
|
||||
Real runs record exact model SHA and all split hashes, tokenizer, quant/recipe, llama.cpp pin+patch identity, backend/driver/toolchain, host/hardware/network, commands/environment (without secrets), raw parity/performance/resource results, and evidence class. Models live on configured mounted-drive storage, never `/home`.
|
||||
|
||||
Routing certification records prove only the exact exercised backend/model/recipe lane. Compile-only, fixture, failed, or unavailable lanes remain registered-dark. V4 cache/state evidence must show KV and CSA/HCA/SWA/indexer/compressor data remain shard-local/session-keyed; route recovery evidence must show cache miss plus re-prefill/restart, not migration.
|
||||
335
.scratch/distributed-gguf-runtime/gitea-issues.json
Normal file
335
.scratch/distributed-gguf-runtime/gitea-issues.json
Normal file
@@ -0,0 +1,335 @@
|
||||
{
|
||||
"repository": "https://git.d-popov.com/popov/neuron-tai",
|
||||
"stories": {
|
||||
"DGR-017": {
|
||||
"number": 1,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/1",
|
||||
"state": "closed",
|
||||
"status": "completed"
|
||||
},
|
||||
"DGR-018": {
|
||||
"number": 2,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/2",
|
||||
"state": "closed",
|
||||
"status": "completed"
|
||||
},
|
||||
"DGR-019": {
|
||||
"number": 3,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/3",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-020": {
|
||||
"number": 4,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/4",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-021": {
|
||||
"number": 5,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/5",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-022": {
|
||||
"number": 6,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/6",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-023": {
|
||||
"number": 7,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/7",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-024": {
|
||||
"number": 8,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/8",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-025": {
|
||||
"number": 9,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/9",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-026": {
|
||||
"number": 10,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/10",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-027": {
|
||||
"number": 11,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/11",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-028": {
|
||||
"number": 12,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/12",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-029": {
|
||||
"number": 13,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/13",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-030": {
|
||||
"number": 14,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/14",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-031": {
|
||||
"number": 15,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/15",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-032": {
|
||||
"number": 16,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/16",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-033": {
|
||||
"number": 17,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/17",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-034": {
|
||||
"number": 18,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/18",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-035": {
|
||||
"number": 19,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/19",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-036": {
|
||||
"number": 20,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/20",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-037": {
|
||||
"number": 21,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/21",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-038": {
|
||||
"number": 22,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/22",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-039": {
|
||||
"number": 23,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/23",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-040": {
|
||||
"number": 24,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/24",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-041": {
|
||||
"number": 25,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/25",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-042": {
|
||||
"number": 26,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/26",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-043": {
|
||||
"number": 27,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/27",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-044": {
|
||||
"number": 28,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/28",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-045": {
|
||||
"number": 29,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/29",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-046": {
|
||||
"number": 30,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/30",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-047": {
|
||||
"number": 31,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/31",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-048": {
|
||||
"number": 32,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/32",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-049": {
|
||||
"number": 33,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/33",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-050": {
|
||||
"number": 34,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/34",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-051": {
|
||||
"number": 35,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/35",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-052": {
|
||||
"number": 36,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/36",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-053": {
|
||||
"number": 37,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/37",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-054": {
|
||||
"number": 38,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/38",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-055": {
|
||||
"number": 39,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/39",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-056": {
|
||||
"number": 40,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/40",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-057": {
|
||||
"number": 41,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/41",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-058": {
|
||||
"number": 42,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/42",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-059": {
|
||||
"number": 43,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/43",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-060": {
|
||||
"number": 44,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/44",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-061": {
|
||||
"number": 45,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/45",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-062": {
|
||||
"number": 46,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/46",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-063": {
|
||||
"number": 47,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/47",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-064": {
|
||||
"number": 48,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/48",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-065": {
|
||||
"number": 49,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/49",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-066": {
|
||||
"number": 50,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/50",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-067": {
|
||||
"number": 51,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/51",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-068": {
|
||||
"number": 52,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/52",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-069": {
|
||||
"number": 53,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/53",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-070": {
|
||||
"number": 54,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/54",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-071": {
|
||||
"number": 55,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/55",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
}
|
||||
}
|
||||
}
|
||||
40
.scratch/distributed-gguf-runtime/implementation-strategy.md
Normal file
40
.scratch/distributed-gguf-runtime/implementation-strategy.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Distributed GGUF Runtime implementation strategy
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
|
||||
## Execution model
|
||||
|
||||
Execute one numerically ordered, dependency-ready story per fresh Ralph context. Read `RALPH-CONTEXT.md`, source issue, and dependency evidence first; use TDD/fixture-first verification; finish with exact evidence. `prd.json` is the only state authority.
|
||||
|
||||
## Sequence
|
||||
|
||||
1. **M0 DGR-017..020:** reconcile legacy reality, lock metadata/performance contracts, and run the independent whole-model baseline.
|
||||
2. **M1 DGR-021..033:** protocol/lifecycle/codegen, exact identities and split artifacts, pinned upstream/patches, CPU then accelerator builds, `ShardEngine`, fixtures, fake worker.
|
||||
3. **M2 DGR-034..043:** dense ranged ownership/boundary/parity/local state, worker integration, supervision/direct-relay, and measured GGUF inputs to unchanged routing.
|
||||
4. **M3 DGR-044..054:** pin/inventory V4, adapt upstream boundary/local state/MoE/hash execution, pass parity and real 2–4 scenario, then enforce alpha with MTP off.
|
||||
5. **M4 DGR-055..067:** batching/backpressure/failure/recovery/long-context, existing-routing 10+ certification, real scale, measured optimization/compression, MTP contract+implementation, hardware certification.
|
||||
6. **M5 DGR-068..071:** packages, human upstream collaboration, beta gate (including MTP), and pin/patch/certification maintenance.
|
||||
|
||||
## Guardrails
|
||||
|
||||
|
||||
## Locked scope
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
|
||||
## Target identities
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
DGR-020 cannot use distributed results. DGR-054 does not depend on MTP. DGR-070 depends on DGR-066. Compile support and scenario success never imply general routability.
|
||||
@@ -1,19 +0,0 @@
|
||||
# 01 — Route Session lifecycle
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## What to build
|
||||
|
||||
Add the narrowest end-to-end Route Session lifecycle that can be used by distributed inference routes: create a session, bind it to a selected Inference Route, expose status, and close it cleanly. This slice does not need real model cache yet; it proves stable session identity across the control plane and activation plane.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A request can create a Route Session with a stable `session_id`, `route_id`, model preset, backend id, and route membership.
|
||||
- [ ] Every downstream activation request carries the same session identity and fails clearly if the session or route id does not match.
|
||||
- [ ] Session status reports phase, route nodes, model preset, backend id, created time, and last activity time.
|
||||
- [ ] Closing a session releases all registered per-session state.
|
||||
- [ ] Tests cover create, status, close, stale-session rejection, and wrong-route rejection.
|
||||
|
||||
## Blocked by
|
||||
|
||||
None - can start immediately.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-017: Reconcile and clean the superseded DGR backlog
|
||||
|
||||
- **Status / triage:** completed; `passes: true`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** None
|
||||
- **Blocks (derived):** `DGR-018`, `DGR-019`, `DGR-027`, `DGR-054`
|
||||
- **Labels:** `area:provenance`, `area:cleanup`, `type:audit`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-018: Define canonical Ralph and Gitea metadata schema
|
||||
|
||||
- **Status / triage:** completed; `passes: true`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-021`, `DGR-025`
|
||||
- **Labels:** `area:planning`, `area:gitea`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
||||
- [x] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
||||
- [x] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
||||
- [x] Add deterministic model-free tests for parse, validation, and generation round trips.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||
@@ -0,0 +1,40 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-019: Lock alpha and beta performance contracts
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-020`, `DGR-044`, `DGR-054`
|
||||
- **Labels:** `area:performance`, `type:contract`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `generic+deepseek-v4`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
|
||||
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
|
||||
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
|
||||
- [ ] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
|
||||
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-019/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,20 +0,0 @@
|
||||
# 02 — Prefill/decode binary HTTP protocol
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## What to build
|
||||
|
||||
Split the activation protocol into explicit prefill and decode-step calls using the existing binary HTTP direction from ADR-0008. The completed slice should work against a stub backend so payload shape, route/session headers, relay preservation, and failure behavior are testable before real KV cache work begins.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Prefill accepts chunked binary activations with route/session metadata and forwards them through the selected route.
|
||||
- [ ] Decode-step accepts a one-step binary activation and forwards a one-step activation through the selected route.
|
||||
- [ ] Decode-step payload size is independent of prompt length in protocol tests.
|
||||
- [ ] Relay forwarding preserves route/session headers, shape, dtype, position, and wire version.
|
||||
- [ ] Legacy `/forward` either remains as a compatibility wrapper or fails with a clear wire-version error.
|
||||
- [ ] Tests cover prefill chunking, decode-step shape validation, relay preservation, and malformed header rejection.
|
||||
|
||||
## Blocked by
|
||||
|
||||
- 01 — Route Session lifecycle.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-020: Run the controlled whole-model GGUF baseline
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-019`
|
||||
- **Blocks (derived):** `DGR-054`
|
||||
- **Labels:** `area:performance`, `type:benchmark`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
|
||||
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
|
||||
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
|
||||
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-021: Define the versioned named-tensor stream envelope
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-018`
|
||||
- **Blocks (derived):** `DGR-022`, `DGR-023`, `DGR-025`, `DGR-031`, `DGR-035`, `DGR-046`
|
||||
- **Labels:** `area:protocol`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md`, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
|
||||
- [ ] Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
|
||||
- [ ] Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
|
||||
- [ ] Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-021/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-022: Define Shard lifecycle and structured status RPCs
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`
|
||||
- **Blocks (derived):** `DGR-024`, `DGR-033`, `DGR-037`
|
||||
- **Labels:** `area:protocol`, `area:lifecycle`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
|
||||
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
|
||||
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
|
||||
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-023: Make Python and C++ protobuf generation reproducible
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`
|
||||
- **Blocks (derived):** `DGR-024`, `DGR-037`
|
||||
- **Labels:** `area:protocol`, `area:build`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
|
||||
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
|
||||
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests.
|
||||
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-024: Implement in-memory fake gRPC seam transport
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-022`, `DGR-023`
|
||||
- **Blocks (derived):** `DGR-033`, `DGR-042`
|
||||
- **Labels:** `area:protocol`, `area:testing`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
|
||||
- [ ] Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
|
||||
- [ ] Verify direct and opaque-relay framing preserve identical protobuf bytes.
|
||||
- [ ] Tests require no sockets outside localhost, model downloads, or native accelerator.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-025: Define exact artifact and runtime recipe identity
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-018`, `DGR-021`
|
||||
- **Blocks (derived):** `DGR-026`, `DGR-031`, `DGR-041`, `DGR-044`
|
||||
- **Labels:** `area:identity`, `area:admission`, `type:domain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md`, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
|
||||
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant.
|
||||
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
|
||||
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-026: Provision exact split-GGUF artifacts outside /home
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-025`
|
||||
- **Blocks (derived):** `DGR-044`, `DGR-045`
|
||||
- **Labels:** `area:artifacts`, `area:provenance`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
|
||||
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
|
||||
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
|
||||
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-026/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-028`, `DGR-029`, `DGR-044`
|
||||
- **Labels:** `area:upstream`, `area:build`, `type:provenance`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
|
||||
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
|
||||
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
|
||||
- [ ] Offline reuse is supported only after the cached tree’s exact identity is verified.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-028: Implement numbered patch-stack apply and verification
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-027`
|
||||
- **Blocks (derived):** `DGR-029`, `DGR-034`, `DGR-069`
|
||||
- **Labels:** `area:upstream`, `area:patches`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
|
||||
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
|
||||
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
|
||||
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-029: Create the native CMake skeleton and deterministic CPU lane
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-027`, `DGR-028`
|
||||
- **Blocks (derived):** `DGR-030`, `DGR-034`
|
||||
- **Labels:** `area:build`, `type:toolchain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
|
||||
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
|
||||
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
|
||||
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-029/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,21 +0,0 @@
|
||||
# 03 — Generation Telemetry and streaming response contract
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## What to build
|
||||
|
||||
Expose realtime Generation Telemetry for active Route Sessions and stream token deltas when the serving path can produce them. This slice should make long distributed requests observable before real large-model work begins.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A client can observe route-session phase changes: queued, loading, prefill, decode, finalizing, completed, failed.
|
||||
- [ ] Telemetry includes prefill progress, generated token count, rolling tokens/sec, average tokens/sec, active route nodes, and failure reason.
|
||||
- [ ] Telemetry is available before the first output token.
|
||||
- [ ] A streaming response can include token deltas while telemetry remains available.
|
||||
- [ ] A non-streaming fallback still exposes telemetry until final answer or failure.
|
||||
- [ ] Route-node failure reports the last known phase and reason.
|
||||
- [ ] Tests cover telemetry updates, streaming token deltas, non-streaming fallback, and structured failure closeout.
|
||||
|
||||
## Blocked by
|
||||
|
||||
- 01 — Route Session lifecycle.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-030: Add accelerator build presets and native CI matrix
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-029`
|
||||
- **Blocks (derived):** `DGR-053`, `DGR-067`, `DGR-068`
|
||||
- **Labels:** `area:build`, `area:hardware`, `type:toolchain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md`, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
|
||||
- [ ] Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
|
||||
- [ ] Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
|
||||
- [ ] Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-030/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-031: Introduce the project-owned `ShardEngine` interface
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`, `DGR-025`
|
||||
- **Blocks (derived):** `DGR-032`, `DGR-034`, `DGR-035`, `DGR-037`
|
||||
- **Labels:** `area:engine`, `type:architecture`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md`, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
|
||||
- [ ] Use project-owned request/result/state types; expose no `ggml_tensor`, llama context, scheduler, or ABI-owned structure.
|
||||
- [ ] Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
|
||||
- [ ] Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-031/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-032: Implement deterministic fake `ShardEngine`
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-031`
|
||||
- **Blocks (derived):** `DGR-033`
|
||||
- **Labels:** `area:engine`, `area:testing`, `type:fixture`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md`, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
|
||||
- [ ] Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
|
||||
- [ ] Support configurable delay, memory pressure, malformed output, and crash injection.
|
||||
- [ ] Contract tests distinguish fixture evidence from real-model certification.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-032/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-033: Build a standalone fake C++ gRPC Shard worker
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-022`, `DGR-024`, `DGR-032`
|
||||
- **Blocks (derived):** `DGR-036`, `DGR-040`
|
||||
- **Labels:** `area:worker`, `area:grpc`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
|
||||
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
|
||||
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
|
||||
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-033/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-034: Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-028`, `DGR-029`, `DGR-031`
|
||||
- **Blocks (derived):** `DGR-035`, `DGR-037`, `DGR-051`
|
||||
- **Labels:** `area:llama`, `area:weights`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md`, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
|
||||
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state.
|
||||
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
|
||||
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-035: Implement dense architecture boundary input/output
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-021`, `DGR-031`, `DGR-034`
|
||||
- **Blocks (derived):** `DGR-036`, `DGR-037`, `DGR-069`
|
||||
- **Labels:** `area:llama`, `area:boundary`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md`, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
|
||||
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail returns logits or sampled-token output under an explicit contract.
|
||||
- [ ] Uncertified architectures and incompatible boundary schemas fail closed.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-036: Prove dense fixture and real-model range parity
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-033`, `DGR-035`
|
||||
- **Blocks (derived):** `DGR-039`
|
||||
- **Labels:** `area:parity`, `area:testing`, `type:certification`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md`, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
|
||||
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity.
|
||||
- [ ] At least 32 greedy decode tokens match the locked tolerance.
|
||||
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-037: Bind llama.cpp to the standalone worker
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-022`, `DGR-023`, `DGR-031`, `DGR-034`, `DGR-035`
|
||||
- **Blocks (derived):** `DGR-038`, `DGR-039`, `DGR-040`, `DGR-051`
|
||||
- **Labels:** `area:worker`, `area:llama`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp `ShardEngine` implementation without changing the public worker contract.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
|
||||
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
|
||||
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
|
||||
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,39 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-038: Implement isolated shard-local Hot KV State
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-037`
|
||||
- **Blocks (derived):** `DGR-039`, `DGR-052`, `DGR-055`, `DGR-069`
|
||||
- **Labels:** `area:kv-cache`, `area:concurrency`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md`, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
|
||||
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
|
||||
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-038/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user