Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai
This commit is contained in:
18
.agents/skills/claude-handoff/SKILL.md
Normal file
18
.agents/skills/claude-handoff/SKILL.md
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
---
|
||||||
|
name: claude-handoff
|
||||||
|
description: Hand the current conversation off to a fresh background agent that picks up the work immediately.
|
||||||
|
argument-hint: "What will the next session be used for?"
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Write a handoff summary of the current conversation so a fresh agent can continue the work. Instead of saving it, launch a background agent seeded with the summary as its prompt: `claude --bg --name "<descriptive name>" "<handoff summary>"`. It starts in the current working directory and returns immediately; the user manages it with `claude agents`.
|
||||||
|
|
||||||
|
Always pass `-n`/`--name` with a descriptive name (e.g. `--name "Fix login bug"`) — it sets the display name shown in the job list, session picker, and terminal title.
|
||||||
|
|
||||||
|
Include a "suggested skills" section in the summary, which suggests skills that the agent should invoke.
|
||||||
|
|
||||||
|
Do not duplicate content already captured in other artifacts (PRDs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
||||||
|
|
||||||
|
Redact any sensitive information, such as API keys, passwords, or personally identifiable information — the summary becomes the agent's prompt.
|
||||||
|
|
||||||
|
If the user passed arguments, treat them as a description of what the next session will focus on and tailor the summary accordingly.
|
||||||
89
.agents/skills/code-review/SKILL.md
Normal file
89
.agents/skills/code-review/SKILL.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
---
|
||||||
|
name: code-review
|
||||||
|
description: Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes — Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/PRD asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to "review since X".
|
||||||
|
---
|
||||||
|
|
||||||
|
Two-axis review of the diff between `HEAD` and a fixed point the user supplies:
|
||||||
|
|
||||||
|
- **Standards** — does the code conform to this repo's documented coding standards?
|
||||||
|
- **Spec** — does the code faithfully implement the originating issue / PRD / spec?
|
||||||
|
|
||||||
|
Both axes run as **parallel sub-agents** so they don't pollute each other's context, then this skill aggregates their findings.
|
||||||
|
|
||||||
|
The issue tracker should have been provided to you — run `/setup-matt-pocock-skills` if `docs/agents/issue-tracker.md` is missing.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
### 1. Pin the fixed point
|
||||||
|
|
||||||
|
Whatever the user said is the fixed point — a commit SHA, branch name, tag, `main`, `HEAD~5`, etc. If they didn't specify one, ask for it.
|
||||||
|
|
||||||
|
Capture the diff command once: `git diff <fixed-point>...HEAD` (three-dot, so the comparison is against the merge-base). Also note the list of commits via `git log <fixed-point>..HEAD --oneline`.
|
||||||
|
|
||||||
|
Before going further, confirm the fixed point resolves (`git rev-parse <fixed-point>`) and the diff is non-empty. A bad ref or empty diff should fail here — not inside two parallel sub-agents.
|
||||||
|
|
||||||
|
### 2. Identify the spec source
|
||||||
|
|
||||||
|
Look for the originating spec, in this order:
|
||||||
|
|
||||||
|
1. Issue references in the commit messages (`#123`, `Closes #45`, GitLab `!67`, etc.) — fetch via the workflow in `docs/agents/issue-tracker.md`.
|
||||||
|
2. A path the user passed as an argument.
|
||||||
|
3. A PRD/spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature.
|
||||||
|
4. If nothing is found, ask the user where the spec is. If they say there isn't one, the **Spec** sub-agent will skip and report "no spec available".
|
||||||
|
|
||||||
|
### 3. Identify the standards sources
|
||||||
|
|
||||||
|
Anything in the repo that documents how code should be written, such as `CODING_STANDARDS.md` or `CONTRIBUTING.md`.
|
||||||
|
|
||||||
|
On top of whatever the repo documents, the Standards axis always carries the **smell baseline** below — a fixed set of Fowler code smells (_Refactoring_, ch.3) that applies even when a repo documents nothing. Two rules bind it:
|
||||||
|
|
||||||
|
- **The repo overrides.** A documented repo standard always wins; where it endorses something the baseline would flag, suppress the smell.
|
||||||
|
- **Always a judgement call.** Each smell is a labelled heuristic ("possible Feature Envy"), never a hard violation — and, like any standard here, skip anything tooling already enforces.
|
||||||
|
|
||||||
|
Each smell reads *what it is* → *how to fix*; match it against the diff:
|
||||||
|
|
||||||
|
- **Mysterious Name** — a function, variable, or type whose name doesn't reveal what it does or holds. → rename it; if no honest name comes, the design's murky.
|
||||||
|
- **Duplicated Code** — the same logic shape appears in more than one hunk or file in the change. → extract the shared shape, call it from both.
|
||||||
|
- **Feature Envy** — a method that reaches into another object's data more than its own. → move the method onto the data it envies.
|
||||||
|
- **Data Clumps** — the same few fields or params keep travelling together (a type wanting to be born). → bundle them into one type, pass that.
|
||||||
|
- **Primitive Obsession** — a primitive or string standing in for a domain concept that deserves its own type. → give the concept its own small type.
|
||||||
|
- **Repeated Switches** — the same `switch`/`if`-cascade on the same type recurs across the change. → replace with polymorphism, or one map both sites share.
|
||||||
|
- **Shotgun Surgery** — one logical change forces scattered edits across many files in the diff. → gather what changes together into one module.
|
||||||
|
- **Divergent Change** — one file or module is edited for several unrelated reasons. → split so each module changes for one reason.
|
||||||
|
- **Speculative Generality** — abstraction, parameters, or hooks added for needs the spec doesn't have. → delete it; inline back until a real need shows.
|
||||||
|
- **Message Chains** — long `a.b().c().d()` navigation the caller shouldn't depend on. → hide the walk behind one method on the first object.
|
||||||
|
- **Middle Man** — a class or function that mostly just delegates onward. → cut it, call the real target direct.
|
||||||
|
- **Refused Bequest** — a subclass or implementer that ignores or overrides most of what it inherits. → drop the inheritance, use composition.
|
||||||
|
|
||||||
|
### 4. Spawn both sub-agents in parallel
|
||||||
|
|
||||||
|
Send a single message with two `Agent` tool calls. Use the `general-purpose` subagent for both.
|
||||||
|
|
||||||
|
**Standards sub-agent prompt** — include:
|
||||||
|
|
||||||
|
- The full diff command and commit list.
|
||||||
|
- The list of standards-source files you found in step 3, **plus the smell baseline from step 3** pasted in full — the sub-agent has no other access to it.
|
||||||
|
- The brief: "Report — per file/hunk where relevant — (a) every place the diff violates a documented standard: cite the standard (file + the rule); and (b) any baseline smell you spot: name it and quote the hunk. Distinguish hard violations from judgement calls — documented-standard breaches can be hard, but baseline smells are always judgement calls, and a documented repo standard overrides the baseline. Skip anything tooling enforces. Under 400 words."
|
||||||
|
|
||||||
|
**Spec sub-agent prompt** — include:
|
||||||
|
|
||||||
|
- The diff command and commit list.
|
||||||
|
- The path or fetched contents of the spec.
|
||||||
|
- The brief: "Report: (a) requirements the spec asked for that are missing or partial; (b) behaviour in the diff that wasn't asked for (scope creep); (c) requirements that look implemented but where the implementation looks wrong. Quote the spec line for each finding. Under 400 words."
|
||||||
|
|
||||||
|
If the spec is missing, skip the Spec sub-agent and note this in the final report.
|
||||||
|
|
||||||
|
### 5. Aggregate
|
||||||
|
|
||||||
|
Present the two reports under `## Standards` and `## Spec` headings, verbatim or lightly cleaned. Do **not** merge or rerank findings — the two axes are deliberately separate (see _Why two axes_).
|
||||||
|
|
||||||
|
End with a one-line summary: total findings per axis, and the worst issue _within each axis_ (if any). Don't pick a single winner across axes — that's the reranking the separation exists to prevent.
|
||||||
|
|
||||||
|
## Why two axes
|
||||||
|
|
||||||
|
A change can pass one axis and fail the other:
|
||||||
|
|
||||||
|
- Code that follows every standard but implements the wrong thing → **Standards pass, Spec fail.**
|
||||||
|
- Code that does exactly what the issue asked but breaks the project's conventions → **Spec pass, Standards fail.**
|
||||||
|
|
||||||
|
Reporting them separately stops one axis from masking the other.
|
||||||
12
.agents/skills/research/SKILL.md
Normal file
12
.agents/skills/research/SKILL.md
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
---
|
||||||
|
name: research
|
||||||
|
description: Investigate a question against high-trust primary sources and capture the findings as a Markdown file in the repo. Use when the user wants a topic researched, docs or API facts gathered, or reading legwork delegated to a background agent.
|
||||||
|
---
|
||||||
|
|
||||||
|
Spin up a **background agent** to do the research, so you keep working while it reads.
|
||||||
|
|
||||||
|
Its job:
|
||||||
|
|
||||||
|
1. Investigate the question against **primary sources** — official docs, source code, specs, first-party APIs — not a secondary write-up of them. Follow every claim back to the source that owns it.
|
||||||
|
2. Write the findings to a single Markdown file, citing each claim's source.
|
||||||
|
3. Save it where the repo already keeps such notes; match the existing convention, and if there is none, put it somewhere sensible and say where.
|
||||||
101
.agents/skills/wayfinder/SKILL.md
Normal file
101
.agents/skills/wayfinder/SKILL.md
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
name: wayfinder
|
||||||
|
description: Plan a huge chunk of work — more than one agent session can hold — as a shared map of investigation tickets on your issue tracker, and resolve them one at a time until the way to the goal is clear.
|
||||||
|
---
|
||||||
|
|
||||||
|
A loose idea has arrived — too big for one agent session, and wrapped in fog: the route from here to a plan isn't visible yet. This skill charts it as a **shared map** on the repo's issue tracker, then works its tickets one at a time. The map is domain-agnostic — engineering work, course content, whatever fits the shape.
|
||||||
|
|
||||||
|
## Refer by name
|
||||||
|
|
||||||
|
Every map and ticket is an issue, so it has a **name** — its title. In everything the human reads — narration, the map's Decisions-so-far — refer to it by that name, never by a bare id, number, or slug. A wall of `#42, #43, #44` is illegible; names read at a glance. The id and URL don't vanish — a name wraps its link — but they ride *inside* the name, never stand in for it.
|
||||||
|
|
||||||
|
## The Map
|
||||||
|
|
||||||
|
The map is a single issue on this repo's issue tracker, labelled `wayfinder:map` — the canonical artifact. Its tickets are child issues of the map.
|
||||||
|
|
||||||
|
The map is an **index**, not a store. It lists the decisions made and points at the tickets that hold their detail; a decision lives in exactly one place — its ticket — so the map never restates it, only gists it and links.
|
||||||
|
|
||||||
|
**Where the map, its child tickets, blocking, and frontier queries physically live is tracker-specific.** Consult `docs/agents/issue-tracker.md` (the "Wayfinding operations" section) for how _this_ repo expresses them. If that doc is absent, default to the local-markdown tracker.
|
||||||
|
|
||||||
|
### The map body
|
||||||
|
|
||||||
|
The whole map at low resolution, loaded once per session. Open tickets are **not** listed — they are open child issues, found by query.
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
<domain; skills every session should consult; standing preferences for this effort>
|
||||||
|
|
||||||
|
## Decisions so far
|
||||||
|
|
||||||
|
<!-- the index — one line per closed ticket: enough to judge relevance, then zoom the link for the detail the ticket holds -->
|
||||||
|
|
||||||
|
- [<closed ticket title>](link) — <one-line gist of the answer>
|
||||||
|
|
||||||
|
## Fog
|
||||||
|
|
||||||
|
<!-- see "Fog of war" for what belongs here -->
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tickets
|
||||||
|
|
||||||
|
Each ticket is a **child issue** of the map; the tracker's issue id is its identity. Its body is the question, sized to one 100K token agent session:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Question
|
||||||
|
|
||||||
|
<the decision or investigation this ticket resolves>
|
||||||
|
```
|
||||||
|
|
||||||
|
Each ticket carries a `wayfinder:<type>` label — one of `research`, `prototype`, `grilling`, `task` (see [Ticket Types](#ticket-types)).
|
||||||
|
|
||||||
|
A session **claims** a ticket by assigning it to the dev driving the map, **first**, before any work, so concurrent sessions skip it. That assignee _is_ the claim: an open, unassigned ticket is unclaimed.
|
||||||
|
|
||||||
|
Blocking uses the tracker's **native** dependency relationship — essential because it renders the frontier _visually_ in the tracker's own UI, so the human sees what's takeable without opening the map. Only a tracker that lacks native blocking falls back to a body convention. A ticket is **unblocked** when every ticket blocking it is closed; the **frontier** is the open, unblocked, unclaimed children — the edge of the known.
|
||||||
|
|
||||||
|
The answer isn't part of the body — it's recorded on resolution (see [Work through the map](#work-through-the-map)). Assets created while resolving a ticket are linked from the issue, not pasted in.
|
||||||
|
|
||||||
|
## Ticket Types
|
||||||
|
|
||||||
|
- **Research**: Reading documentation, third-party APIs, or local resources like knowledge bases. Creates a markdown summary as a linked asset. Use when knowledge outside the current working directory is required.
|
||||||
|
- **Prototype**: Raise the fidelity of the discussion by making a cheap, rough, concrete artifact to react to — an outline, a rough take, a stub, or UI/logic code via the /prototype skill. Links the prototype as an asset. Use when "how should it look" or "how should it behave" is the key question.
|
||||||
|
- **Grilling**: Conversation with the agent. Uses the /grilling and /domain-modeling skills. Asks one question at a time. The default case.
|
||||||
|
- **Task**: Literal manual work that must be done before the discussion can move forward — nothing to decide, prototype, or research. Moving data, signing up for a service, provisioning access. The agent automates it where it can; otherwise it hands the human a precise checklist. Resolved when the work is done; the answer records what was done and any resulting facts (credentials location, new URLs, row counts) later tickets depend on.
|
||||||
|
|
||||||
|
## Fog of war
|
||||||
|
|
||||||
|
The map is _deliberately_ incomplete: don't chart what you can't yet see. Beyond the tickets lies fog — the dim view of decisions and investigations you can tell are coming but can't yet pin down, because they hang on questions still open. Resolving a ticket clears the fog ahead of it, graduating whatever's now specifiable into fresh tickets — one at a time, until the way to the goal is clear and no tickets remain.
|
||||||
|
|
||||||
|
The map's **Fog** section is where that dim view is written down: the suspected question, the area to revisit later, the risk you're deferring. Write as loosely or as fully as the view allows; it doubles as a signpost for collaborators reading where the effort is headed.
|
||||||
|
|
||||||
|
**Fog or ticket?** The test is whether you can state the question precisely now — _not_ whether you can answer it now.
|
||||||
|
|
||||||
|
- **Ticket when** the question is already sharp — even if it's blocked and you can't act on it yet.
|
||||||
|
- **Fog when** you can't yet phrase it that sharply. Don't pre-slice fog into ticket-sized pieces: it's coarser than a ticket, and one patch may graduate into several tickets, or none, once the frontier reaches it.
|
||||||
|
|
||||||
|
Fog excludes only what's already decided (that's Decisions so far) and what's already a ticket.
|
||||||
|
|
||||||
|
## Invocation
|
||||||
|
|
||||||
|
Two modes. Either way, **never resolve more than one ticket per session.**
|
||||||
|
|
||||||
|
### Chart the map
|
||||||
|
|
||||||
|
User invokes with a loose idea.
|
||||||
|
|
||||||
|
1. Run a `/grilling` and `/domain-modeling` session to surface the open decisions.
|
||||||
|
2. **Create the map** (label `wayfinder:map`): Notes filled in, Decisions-so-far empty, Fog sketched.
|
||||||
|
3. **Create the tickets you can specify now** as child issues of the map — then wire blocking edges in a **second pass** (issues need ids before they can reference each other). Wiring sorts them into the frontier and the blocked; everything you can't yet specify stays in the Fog.
|
||||||
|
4. Stop — charting the map is one session's work; do not also resolve tickets.
|
||||||
|
|
||||||
|
### Work through the map
|
||||||
|
|
||||||
|
User invokes with a map (URL or number). A ticket is **optional** — without one, you pick the next decision, not the user.
|
||||||
|
|
||||||
|
1. Load the **map** — the low-res view, not every ticket body.
|
||||||
|
2. Choose the ticket. If the user named one, use it. Otherwise take the first frontier ticket in order. **Claim it**: assign it to yourself before any work.
|
||||||
|
3. Resolve it — **zoom as needed**: fetch the full body of any related or closed ticket on demand; invoke the skills the `## Notes` block names. If in doubt, use `/grilling` and `/domain-modeling`.
|
||||||
|
4. Record the resolution: post the answer as a **resolution comment**, **close** the issue, and **append a context pointer** to the map's Decisions-so-far.
|
||||||
|
5. Add newly-surfaced tickets (create-then-wire); graduate any fog the answer has made specifiable, clearing each graduated patch from the Fog so it lives only as its new ticket. If the decision invalidates other parts of the map, update or delete those tickets.
|
||||||
|
|
||||||
|
The user may run unblocked tickets in parallel, so expect other sessions to be editing the tracker concurrently.
|
||||||
45
.agents/skills/wizard/SKILL.md
Normal file
45
.agents/skills/wizard/SKILL.md
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
---
|
||||||
|
name: wizard
|
||||||
|
description: Generate an interactive bash wizard that walks a human through a manual procedure — third-party setup, a one-off migration, an A→B state transition — opening URLs, capturing values, confirming each step, and writing .env files and GitHub Actions secrets.
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wizard
|
||||||
|
|
||||||
|
A **wizard** is a bash script that walks a human, step by step, through a manual procedure that's tedious to do by hand and tedious to re-explain to an AI every time. It opens each URL, says exactly what to click and copy, captures the values, writes them where they belong (`.env`, GitHub secrets), confirms at every stage, and shows how much is left. It might configure third-party services, run a one-off migration, or move the project from one state to another.
|
||||||
|
|
||||||
|
The delightful UX is already solved by [template.sh](template.sh) — progress with time-remaining, confirmation gates, cross-platform URL opening (including WSL), hidden secret entry, idempotent `.env` upserts, `gh secret`/`gh variable` writes, and a closing summary. **Your job is only to scope the procedure and author its stages.** The library above the `STAGES` marker is identical in every wizard; that consistency is the point — never hand-edit it.
|
||||||
|
|
||||||
|
A wizard is ephemeral by default — built for one run, saved to a scratch or `scripts/` path, deleted when the job's done. Commit it only when the user wants a repeatable setup path that should live in the repo.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
### 1. Scope the procedure
|
||||||
|
|
||||||
|
Work out every manual step the human must take and every value that gets captured along the way. Read the repo first — don't ask cold:
|
||||||
|
|
||||||
|
- For setup: `.env`, `.env.example`, `.env.*`, `README`, `docker-compose*`, framework config, and `.github/workflows/*` (every `secrets.*` / `vars.*` reference is a value the wizard must produce).
|
||||||
|
- For a migration or transition: the current state, the target state, and the irreversible actions between them.
|
||||||
|
|
||||||
|
Then show the user the ordered list of stages and the values each produces, and confirm — they may add, drop, or reorder.
|
||||||
|
|
||||||
|
**Done when:** every stage is named in order, and for each captured value you know (a) where the human gets it, (b) where it's written (`.env`, a GitHub secret, both, or nowhere — some stages are pure actions), and (c) whether it's secret (hidden entry) or public.
|
||||||
|
|
||||||
|
### 2. Map each stage's journey
|
||||||
|
|
||||||
|
For each stage, write the precise path a human follows: which URL to open, what to do there, where a value is shown, which variable it fills — e.g. "Dashboard → Developers → API keys → Reveal test key → copy". Where you don't actually know the current UI or the exact command, say so and ask the user or check the docs — never invent steps that may not exist.
|
||||||
|
|
||||||
|
**Done when:** every stage traces to concrete instructions a stranger could follow.
|
||||||
|
|
||||||
|
### 3. Author the wizard
|
||||||
|
|
||||||
|
Copy `template.sh` to the target path. Replace the example stage with one `stage` per step, in dependency order. Use the library helpers — `stage`, `say`/`step`, `open_url`, `ask`/`ask_secret`, `write_env`, `set_secret`/`set_var`, `pause`/`confirm` — and set `TOTAL_STAGES` and `TOTAL_MINUTES` to honest estimates (this drives the time-remaining display).
|
||||||
|
|
||||||
|
Hold the bar the template sets: open the URL before asking for its value, use `ask_secret` for anything secret, `write_env` every persisted value, `set_secret` only the values CI actually needs, and `confirm` before any irreversible action. Each `stage` clears the screen so only the current step is visible — keep a stage to one focused task so nothing the human needs scrolls away. Don't touch the library above the marker.
|
||||||
|
|
||||||
|
### 4. Verify and hand off
|
||||||
|
|
||||||
|
- `bash -n <script>`; run `shellcheck` if available.
|
||||||
|
- `chmod +x <script>`.
|
||||||
|
- Don't run it end-to-end yourself — it opens browsers and blocks on human input. Trace it statically instead: every value from step 1 is captured and lands where step 1 said, and every `set_secret` name exactly matches a `secrets.*` reference in CI.
|
||||||
|
- Tell the user how to run it. If it's a repeatable setup path, commit it and link it from the README so the next person runs the script instead of asking an AI.
|
||||||
@@ -2,4 +2,5 @@
|
|||||||
|
|
||||||
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
||||||
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
||||||
- [Project status](project-status.md) — 29/30 done; US-030 (manual route + hop benchmark) is the only open story
|
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
|
||||||
|
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../.scratch/alpha-hardening/README.md), [handoff](../.scratch/alpha-hardening/handoff.md))
|
||||||
|
|||||||
@@ -23,6 +23,10 @@ Suite: 222 passed, 3 skipped (openai/langchain packages missing in .venv — pre
|
|||||||
**Why:** design locked in ADR-0015 (USDT custodial settlement; TAI deferred, protocol cut = future TAI liquidity).
|
**Why:** design locked in ADR-0015 (USDT custodial settlement; TAI deferred, protocol cut = future TAI liquidity).
|
||||||
**How to apply:** next steps are live devnet verification (run devnet_setup.py, start tracker with --solana-rpc-url/--usdt-mint/--treasury-keypair --billing-db), then the TAI mint when volume justifies it. Work not yet committed to git as of session end — check git status.
|
**How to apply:** next steps are live devnet verification (run devnet_setup.py, start tracker with --solana-rpc-url/--usdt-mint/--treasury-keypair --billing-db), then the TAI mint when volume justifies it. Work not yet committed to git as of session end — check git status.
|
||||||
|
|
||||||
|
## Alpha hardening (2026-07-04)
|
||||||
|
|
||||||
|
Planning complete in `.scratch/alpha-hardening/`: research report, ADRs 0016–0019, 22 issue files, README + handoff. **Bucket 1 trust-boundary blockers** (auth, persistence, starting credit, accounting, wallet binding) are next before fraud arc (TOPLOC, bisection, reputation routing). Prod audit thresholds gated on issue 21 (honest-noise calibration corpus).
|
||||||
|
|
||||||
## Windows CUDA node (working as of 2026-07-01)
|
## Windows CUDA node (working as of 2026-07-01)
|
||||||
- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118
|
- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118
|
||||||
- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200
|
- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200
|
||||||
|
|||||||
93
.scratch/alpha-hardening/README.md
Normal file
93
.scratch/alpha-hardening/README.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
# Alpha hardening — planning index
|
||||||
|
|
||||||
|
Pre-release alpha audit + grilling (2026-07-04). **Research complete; planning complete; Bucket 1 blockers next.**
|
||||||
|
|
||||||
|
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
||||||
|
|
||||||
|
## Artifacts
|
||||||
|
|
||||||
|
| Path | Status |
|
||||||
|
|---|---|
|
||||||
|
| [research-verifiable-inference.md](./research-verifiable-inference.md) | Complete — SOTA research, §8 layered scheme, TOPLOC adopt |
|
||||||
|
| [handoff.md](./handoff.md) | Session handoff — locked decisions, env notes |
|
||||||
|
| [docs/adr/0016–0019](../../docs/adr/) | Alpha scope, auth, fraud, multi-tracker design |
|
||||||
|
| [issues/](./issues/) | 22 work items (Buckets 1–3) |
|
||||||
|
|
||||||
|
## ADRs (this feature)
|
||||||
|
|
||||||
|
| ADR | Title |
|
||||||
|
|---|---|
|
||||||
|
| [0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md) | Alpha scope & known limitations |
|
||||||
|
| [0017](../../docs/adr/0017-tracker-authentication-and-authorization.md) | Tracker authentication & authorization |
|
||||||
|
| [0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) | Fraud detection, verification & reputation (flagship) |
|
||||||
|
| [0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) | Money-path consistency — design accepted, impl deferred |
|
||||||
|
| [0002](../../docs/adr/0002-dual-token-payment-model.md) | Amended — settlement superseded by 0015 |
|
||||||
|
| [0010](../../docs/adr/0010-p2p-gossip-and-nat-relay.md) | Amended — TLS alpha reality (relay only) |
|
||||||
|
|
||||||
|
## Recommended implementation order
|
||||||
|
|
||||||
|
**Implement Bucket 1 first.** Fraud arc depends on **auth (01–02, 20)** and **persistence (05)**.
|
||||||
|
|
||||||
|
### Phase 1 — Trust boundary (alpha blockers)
|
||||||
|
|
||||||
|
| Order | Issue | ID | Depends on |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | [Unified auth + gossip auth](./issues/02-a2-unified-auth-boundary.md) + [C1 gossip](./issues/01-c1-gossip-auth.md) + [Validator service token](./issues/20-validator-service-token.md) | A2, C1, — | — |
|
||||||
|
| 2 | [Persist strike/ban/reputation](./issues/05-a1-a5-persist-strike-ban-reputation.md) | A1/A5 | 02 |
|
||||||
|
| 3 | [Starting credit 0 + spend cap](./issues/03-c5-starting-credit-zero.md) | C5, M1 | 02 |
|
||||||
|
| 4 | [Tracker-authoritative accounting](./issues/04-h2-tracker-authoritative-accounting.md) | H2 | 02 |
|
||||||
|
| 5 | [Wallet binding proof](./issues/11-c6-wallet-binding-proof.md) | C6 | 02, 03 |
|
||||||
|
|
||||||
|
### Phase 2 — Fraud arc (after Phase 1)
|
||||||
|
|
||||||
|
| Order | Issue | Depends on |
|
||||||
|
|---|---|---|
|
||||||
|
| 6 | [TOPLOC integration](./issues/06-fraud-toploc-integration.md) | 05 |
|
||||||
|
| 7 | [Commitment + bisection blame](./issues/07-fraud-commitment-bisection-blame.md) | 06 |
|
||||||
|
| 8 | [Reputation model](./issues/08-fraud-reputation-model-persistence.md) | 05, 07 |
|
||||||
|
| 9 | [Routing + adaptive audit](./issues/09-fraud-reputation-routing-adaptive-audit.md) | 08 |
|
||||||
|
| 10 | [Penalty calibration wiring](./issues/10-fraud-penalty-calibration-wiring.md) | 07, 08, 02 |
|
||||||
|
|
||||||
|
**Prod gate:** [21 honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) must complete before enabling production TOPLOC audit thresholds (issues 09–10 in prod). Dev/staging TOPLOC wiring (06–08) may proceed in parallel.
|
||||||
|
|
||||||
|
### Phase 3 — Bucket 2 (post-alpha, design tracked)
|
||||||
|
|
||||||
|
| Issue | ADR |
|
||||||
|
|---|---|
|
||||||
|
| [12 C2 on-chain idempotency](./issues/12-c2-on-chain-idempotency.md) | 0019 §1 |
|
||||||
|
| [13 C3/C4 consensus-gated settlement](./issues/13-c3-c4-consensus-gated-settlement.md) | 0019 §2 |
|
||||||
|
| [14 A3 durable Raft term/vote](./issues/14-a3-raft-durable-term-vote.md) | 0019 §3 |
|
||||||
|
| [15 H1 commutative forfeit](./issues/15-h1-commutative-forfeit.md) | 0019 §4 |
|
||||||
|
|
||||||
|
### Phase 4 — Doc hygiene (parallel anytime)
|
||||||
|
|
||||||
|
| Issue |
|
||||||
|
|---|
|
||||||
|
| [16 US-006 + fraud issue reconciliation](./issues/16-doc-us006-reconciliation.md) |
|
||||||
|
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
||||||
|
| [18 Operational runbooks](./issues/18-doc-operational-runbooks.md) |
|
||||||
|
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env.md) |
|
||||||
|
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status.md) |
|
||||||
|
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
||||||
|
|
||||||
|
## First 3 to implement
|
||||||
|
|
||||||
|
1. **01 + 02 + 20** — Gossip auth + unified auth boundary + validator service token (unblocks financial endpoints)
|
||||||
|
2. **05** — Persist strike/ban/reputation (penalties must survive restart)
|
||||||
|
3. **03** — Starting credit 0 + funded-account gate (closes free-credit faucet)
|
||||||
|
|
||||||
|
## Research anchor
|
||||||
|
|
||||||
|
Fraud design cites [.scratch/alpha-hardening/research-verifiable-inference.md](./research-verifiable-inference.md):
|
||||||
|
|
||||||
|
- **ADOPT** TOPLOC (§8–9, build-vs-adopt table)
|
||||||
|
- **On-demand** commitments, not every request (§8 layer 1 footnote; ADR-0018 §3)
|
||||||
|
- **5% audit budget** as target, not cap (§1.1, §6)
|
||||||
|
- **19× deterrence** via full pending forfeiture (§1.1)
|
||||||
|
- **Hop bisection** blame pattern (§1.2, §8 layer 3)
|
||||||
|
- **Honest-noise corpus** before prod thresholds (§8 layer 3; issue 21)
|
||||||
|
- **Roadmap-only:** zkML, TEE, Gensyn RepOps (§9)
|
||||||
|
|
||||||
|
## Comments
|
||||||
|
|
||||||
|
<!-- Append triage / implementation notes below -->
|
||||||
111
.scratch/alpha-hardening/handoff.md
Normal file
111
.scratch/alpha-hardening/handoff.md
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
# neuron-tai — Alpha Hardening Handoff
|
||||||
|
|
||||||
|
**Date:** 2026-07-04
|
||||||
|
**Repo:** `D:/DEV/git.d-popov.com/neuron-tai`
|
||||||
|
**Prior session:** Pre-release alpha audit + grilling (design locked; planning artifacts complete)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Mission / where we are
|
||||||
|
|
||||||
|
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 0016–0019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Locked decisions
|
||||||
|
|
||||||
|
Point to artifacts — do not re-derive from this handoff.
|
||||||
|
|
||||||
|
| Decision | Status | Reference |
|
||||||
|
|---|---|---|
|
||||||
|
| Alpha scope: one settlement tracker, open join, devnet mock-USDT, reputation carries forward | Locked | `docs/adr/0016-alpha-scope-and-known-limitations.md` |
|
||||||
|
| Two fraud types: correctness (bad output) vs accounting (inflated tokens/shard span) | Locked | Session grilling; research §8 |
|
||||||
|
| Detection: **ADOPT TOPLOC** (MIT, `pip install toploc`); teacher-forced prefill; pin one canonical precision per model | Locked | `.scratch/alpha-hardening/research-verifiable-inference.md` §8 |
|
||||||
|
| Audit rate: **5% default, not a cap**; escalate on anomaly, low reputation, disputes | Locked | Research §1.1, §6, §8 |
|
||||||
|
| Blame: **on-demand** per-hop activation commitments + bisection to first divergent hop (Verde/Truebit **pattern only**) | Locked | Research §1.2; ADR-0018 §3–4 |
|
||||||
|
| Reputation: graduated multiplier (×0.8-per-strike shape), persisted, affects routing + audit rate | Locked | ADR-0018 §6; `packages/validator/README.md` |
|
||||||
|
| Penalty: **full pending forfeiture** is primary hammer (19× deterrence at 5% audit); ×0.8 is routing/payout decay | Locked | Research §1.1; ADR-0018 §1 |
|
||||||
|
| Accounting: tracker authoritative — count tokens from proxied stream; work units from **tracker-assigned** shard span | Locked | ADR-0018 §5; issue 04 |
|
||||||
|
| Persistence prerequisite: strike/ban/reputation/probation must survive tracker restart | Locked | Issue 05 |
|
||||||
|
| Validator service token for forfeit | Locked | ADR-0017 §4; issue 20 |
|
||||||
|
| Honest-noise corpus before prod audit thresholds | Locked | ADR-0018 consequences; issue 21 |
|
||||||
|
| Build vs adopt: TOPLOC **ADOPT**; Verde bisection **ADAPT**; zkML/TEE **roadmap-only** | Locked | Research §9 |
|
||||||
|
| Multi-tracker money-path (C2/C3/C4/A3/H1): **design now, implement later** | Locked | `docs/adr/0019-money-path-consistency-multi-tracker.md` |
|
||||||
|
| Routing layer | **Solid** — no redesign needed | ADR-0013 |
|
||||||
|
|
||||||
|
**Existing ADRs still relevant:** ADR-0003 (historical prototype), ADR-0015 (USDT custodial settlement).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Artifact index (read first)
|
||||||
|
|
||||||
|
| Path | What it contains |
|
||||||
|
|---|---|
|
||||||
|
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
|
||||||
|
| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 1–3) |
|
||||||
|
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
|
||||||
|
| `docs/adr/0016–0019` | Alpha scope, auth, fraud, multi-tracker design |
|
||||||
|
| `docs/agents/issue-tracker.md` | Issue file conventions |
|
||||||
|
| `packages/validator/meshnet_validator/__init__.py` | Current validator; `_final_text_node` blame bug |
|
||||||
|
| `packages/tracker/meshnet_tracker/server.py` | Auth gaps, gossip handlers, proxy accounting |
|
||||||
|
| `.claude/memory/MEMORY.md` | Agent memory index |
|
||||||
|
| Agent transcript (grilling session) | [Alpha audit grilling](4406ccbb-011a-4157-851d-b5b242bba7f7) |
|
||||||
|
|
||||||
|
### Bucket summaries
|
||||||
|
|
||||||
|
**Bucket 1 — Alpha blockers:** C1 gossip auth; A2 unified auth + issue 20 validator token; C5 starting credit; H2 tracker accounting; A1/A5 persistence; C6 wallet binding; fraud arc 06–10.
|
||||||
|
|
||||||
|
**Bucket 2 — Design deferred:** C2, C3/C4, A3, H1 (issues 12–15).
|
||||||
|
|
||||||
|
**Bucket 3 — Doc hygiene:** US-006/07/34 reconciliation (16), MEMORY index (22), runbooks (18), cryptography test env (19).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Recommended implementation order
|
||||||
|
|
||||||
|
See `.scratch/alpha-hardening/README.md` for full table.
|
||||||
|
|
||||||
|
**First 3:**
|
||||||
|
|
||||||
|
1. **01 + 02 + 20** — Gossip auth + unified auth boundary + validator service token
|
||||||
|
2. **05** — Persist strike/ban/reputation
|
||||||
|
3. **03** — Starting credit 0 + funded-account gate
|
||||||
|
|
||||||
|
**Prod gate:** issue **21** (honest-noise corpus) before enabling production audit thresholds.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Open questions / deferred
|
||||||
|
|
||||||
|
| Topic | State |
|
||||||
|
|---|---|
|
||||||
|
| Multi-tracker consensus & settlement idempotency | Bucket 2 (ADR-0019) |
|
||||||
|
| `/v1/gossip` node throughput auth | Out of scope alpha — ADR-0017 §3 note |
|
||||||
|
| Seed-synchronized exact-token audits | Optional complement; depends on sampler control |
|
||||||
|
| ADR-0010 TLS everywhere | Relay TLS only in alpha; ADR amended |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Environment notes
|
||||||
|
|
||||||
|
| Item | Detail |
|
||||||
|
|---|---|
|
||||||
|
| OS | Windows 10 |
|
||||||
|
| Repo path | `D:/DEV/git.d-popov.com/neuron-tai` |
|
||||||
|
| Python venv | `.venv/Scripts/python.exe` |
|
||||||
|
| CUDA node | RTX 4060 8 GB; tracker registration requires `https://` |
|
||||||
|
| Secrets | Do not commit `.env.devnet`, keypairs, treasury material |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Suggested skills
|
||||||
|
|
||||||
|
| Skill | When |
|
||||||
|
|---|---|
|
||||||
|
| **implement** | Bucket 1 code work |
|
||||||
|
| **tdd** | All Bucket 1 fixes |
|
||||||
|
| **diagnosing-bugs** | Auth/gossip/accounting failures |
|
||||||
|
| **domain-modeling** | ADR cross-links |
|
||||||
|
| **code-review** | After each blocker milestone |
|
||||||
|
|
||||||
|
Read `.claude/memory/MEMORY.md` at session start.
|
||||||
41
.scratch/alpha-hardening/issues/01-c1-gossip-auth.md
Normal file
41
.scratch/alpha-hardening/issues/01-c1-gossip-auth.md
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 01 — C1: Authenticate hive gossip endpoints
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Add authenticated peer identity to all tracker gossip mutation endpoints. Today any caller can push billing, account, and stats events without verification.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_gossip` (~2414–2427)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_accounts_gossip` (~2610–2623)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_stats_gossip` (~2355–2364)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `apply_events` (~301–311)
|
||||||
|
- `packages/tracker/meshnet_tracker/accounts.py` — `apply_events` (~220–226)
|
||||||
|
|
||||||
|
Implement per ADR-0017 §3: shared hive HMAC (body + timestamp) or mutual TLS between configured tracker peers. Reject unauthenticated gossip with 401.
|
||||||
|
|
||||||
|
**Note:** `/v1/gossip` (node throughput fan-out, `server.py` ~1331) is **not** in scope for this issue — see ADR-0017 §3 out-of-scope note.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: unauthenticated POST to `/v1/billing/gossip` applies a credit event today — test must fail after fix.
|
||||||
|
2. Red: authenticated peer with valid HMAC applies events; invalid/missing auth returns 401 and `applied: 0`.
|
||||||
|
3. Green: implement verifier + config (`--hive-secret` or peer cert paths).
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `/v1/billing/gossip`, `/v1/accounts/gossip`, `/v1/stats/gossip` reject requests without valid hive auth
|
||||||
|
- [ ] Authenticated peers replicate events as today (id-dedup preserved)
|
||||||
|
- [ ] Config documented for multi-tracker dev setups
|
||||||
|
- [ ] Tests cover reject + accept paths without live network
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
None — implement alongside or immediately before `02-a2-unified-auth-boundary.md`.
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 02 — A2: Unified auth boundary for privileged and financial reads
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Replace header-presence stubs with a single auth middleware that resolves API keys, admin sessions, validator service tokens, and hive peer identity. Close leaks on financial and operator endpoints.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_forfeit` (~2429–2464) — H3: non-empty `Authorization` only
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_benchmark_hop_penalty` (~2650–2658), `_handle_benchmark_results` (~2745–2748) — H3
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_summary` (~2366–2371) — H4
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_settlements` (~2407–2412) — H4
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_registry_wallets` (~2391–2405) — H4
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_session_account` (~2468+), `_handle_admin_accounts` (~2588–2608) — H4
|
||||||
|
- `packages/tracker/meshnet_tracker/accounts.py` — `session_account()`, `create_session()` only (session store; not handler wiring)
|
||||||
|
|
||||||
|
Per ADR-0017 §4: forfeit → validator or admin; benchmark → admin; billing summary/settlements/registry wallets → admin session. Forfeit validator identity: see `20-validator-service-token.md`.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: POST `/v1/billing/forfeit` with `Authorization: Bearer garbage` succeeds today — must require validator/admin identity.
|
||||||
|
2. Red: GET `/v1/billing/summary` without admin session returns 401/403.
|
||||||
|
3. Green: middleware + role checks; existing inference API-key path unchanged.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Single `_require_auth(role=...)` (or equivalent) used by all privileged handlers
|
||||||
|
- [ ] Forfeit accepts only validator service token or admin session — not arbitrary Bearer strings
|
||||||
|
- [ ] Financial read endpoints require admin session (alpha posture)
|
||||||
|
- [ ] Benchmark write/read require admin or service token
|
||||||
|
- [ ] Integration tests for each endpoint class (reject unauth, accept valid)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- `20-validator-service-token.md` — validator service token format, rotation, forfeit auth
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `01-c1-gossip-auth.md` (shared auth config)
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 03 — C5 + M1: Starting credit 0, funded-account gate, spend cap
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Close the free-credit faucet. New API keys start at **0 USDT**; inference requires a real deposit or admin credit. Add a configurable per-request spend cap (M1) to limit runaway charges on compromised keys.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `DEFAULT_STARTING_CREDIT = 1.0` (~22), `ensure_client` (~73–85), `has_funds` (~87–88), duplicate credit on charge (~130–138)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — billing gate before routing (~1667–1690)
|
||||||
|
|
||||||
|
Per ADR-0017 §2 and ADR-0016 §3.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: new API key gets 1.0 USDT implicit credit — test expects 0 balance until deposit.
|
||||||
|
2. Red: first inference without deposit returns 402.
|
||||||
|
3. Green: `DEFAULT_STARTING_CREDIT = 0.0`; optional `--max-charge-per-request` config.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `DEFAULT_STARTING_CREDIT` is 0.0; no automatic caller credit on first touch
|
||||||
|
- [ ] `has_funds` false for fresh keys; 402 before routing (server.py ~1684)
|
||||||
|
- [ ] Admin `credit_client` or bound-wallet deposit still funds accounts
|
||||||
|
- [ ] Configurable max charge per request (M1) rejects oversize completions with clear error
|
||||||
|
- [ ] Tests: fresh key blocked; after credit/deposit, inference proceeds
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)
|
||||||
|
- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md` (admin credit path secured)
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 04 — H2: Tracker-authoritative token and work-unit accounting
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Stop trusting node-reported usage for billing. The tracker already proxies responses — count tokens from the proxied stream/body and compute work units from the **route it constructed**, not node declarations.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `node_work` from route construction (~1776–1782, ~1781–1782)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — streaming token/chunk billing (~1890–1921)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — non-streaming `_usage_total_tokens` (~1938–1943)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `charge_request` node_work split (~104–151)
|
||||||
|
|
||||||
|
Accounting fraud = inflating tokens or shard span. Per ADR-0018 §5.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: mock upstream returns inflated `usage.total_tokens` in body but tracker bills that value — test expects tracker-measured count.
|
||||||
|
2. Red: node registers false `shard_end`; billing uses tracker route span, not registration field alone.
|
||||||
|
3. Green: authoritative counters; ignore node-reported work units on charge path.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Token count for billing derived from tracker-parsed proxy response — **prefer tracker-measured stream chunk count**; when upstream `usage.total_tokens` is present, use the **minimum** of tracker count and node-reported value (cap inflated node usage)
|
||||||
|
- [ ] Work units = tracker-computed layer span per hop at route build time (~1781–1782)
|
||||||
|
- [ ] Nodes cannot increase payout by lying about shard range mid-request
|
||||||
|
- [ ] Integration test: malicious node metadata does not inflate `charge_request` shares
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §5
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md`
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 05 — A1/A5: Persist strike, ban, and reputation state
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Registry strike/ban/reputation state today lives in RAM-only `_LocalContractState` — tracker restart wipes penalties. Persist to SQLite (same pattern as `BillingLedger` and `AccountStore`) so reputation carries forward per ADR-0016 §4.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/contracts/meshnet_contracts/__init__.py` — `RegistryContract`, `RegistryWallet`, in-memory `_state.registry` (~103–206)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — SQLite persistence pattern (~60, event log)
|
||||||
|
- `packages/tracker/meshnet_tracker/accounts.py` — SQLite + event replication (~40–56)
|
||||||
|
|
||||||
|
Include fields for: `strike_count`, `banned`, `completed_job_count`, graduated **reputation score** (float, default 1.0), `last_audit_ts`, probation tracking.
|
||||||
|
|
||||||
|
**Scope split:** this issue owns **schema + persistence + load/reload** only. Reputation **scoring deltas** (audit pass/fail adjustments, decay rules) belong in issue 08.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: record strike, restart tracker process, strike count is 0 — must fail.
|
||||||
|
2. Green: persist + reload; gossip replicates strike events if multi-tracker.
|
||||||
|
3. Red: banned wallet registers node — must reject (wire to routing).
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Strike/ban/reputation survive tracker restart (SQLite or equivalent)
|
||||||
|
- [ ] `RegistryContract.list_wallets` reflects persisted state
|
||||||
|
- [ ] Banned wallet rejected at registration and excluded from routes
|
||||||
|
- [ ] Reputation score field present for routing/audit issues (08–09)
|
||||||
|
- [ ] Event-sourced mutations compatible with future Raft (ADR-0019)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md) §4
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §6
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md`
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 06 — FRAUD: TOPLOC integration (teacher-forced audit primitive)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Adopt [TOPLOC](https://github.com/PrimeIntellect-ai/toploc) (MIT, `pip install toploc`) for activation fingerprint commit and verify. Replace string-equality validator checks with teacher-forced prefill + TOPLOC tolerance matching.
|
||||||
|
|
||||||
|
**Estimated effort:** 2+ sessions — split into subtasks below without separate issue files.
|
||||||
|
|
||||||
|
| Subtask | Owner package | Deliverable |
|
||||||
|
|---|---|---|
|
||||||
|
| Validator audit primitive | `packages/validator/` | Teacher-forced prefill, TOPLOC verify, unit tests with stub tensors |
|
||||||
|
| Node runtime commitments | `packages/node/` (if prover-side) | On-demand activation fingerprint generation on audit-selected requests |
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/validator/meshnet_validator/__init__.py` — `_run_reference`, `_outputs_match` (~92–148)
|
||||||
|
- `packages/validator/README.md` — deterrence math (update for 19× at p=0.05)
|
||||||
|
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layers 1–2, build-vs-adopt table
|
||||||
|
|
||||||
|
Pin one canonical precision/quantization per model preset. Add `toploc` to validator (and node if prover-side) dependencies.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: validator compares final text strings — fails on cross-GPU honest divergence (document expected).
|
||||||
|
2. Green: stub activation tensors + TOPLOC proofs round-trip in unit test.
|
||||||
|
3. Integration: reference node teacher-forces tokens; verify accepts honest proof, rejects swapped precision.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `toploc` dependency declared; `build_proofs_*` / `verify_proofs_*` wired
|
||||||
|
- [ ] Validator re-runs claimed token sequence as prefill, not free generation
|
||||||
|
- [ ] Model preset documents canonical dtype/quantization
|
||||||
|
- [ ] README updated: 19× deterrence at 5% audit (research §1.1)
|
||||||
|
- [ ] Tests with deterministic stub tensors (no GPU required in CI)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §2
|
||||||
|
- Research: [research-verifiable-inference.md](../research-verifiable-inference.md) §8, §9 build-vs-adopt
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `05-a1-a5-persist-strike-ban-reputation.md`
|
||||||
|
|
||||||
|
**Prod gate:** do not enable production audit thresholds until `21-honest-noise-calibration-corpus.md` completes (see README Phase 2 note).
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 07 — FRAUD: On-demand commitment + hop bisection blame
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
On audit selection, require nodes to supply TOPLOC-style fingerprints of **output boundary activations** per hop (on-demand, brief retention). On verify failure, referee identifies the **first divergent hop** — not always the last text node.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/validator/meshnet_validator/__init__.py` — `_slash_route`, `_final_text_node` bug (~102–140) — blames `max(shard_end)` only
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — route hop construction (~1774–1783) — cut-points for bisection
|
||||||
|
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §1.2, §8 layer 3 (Verde **pattern**, not on-chain game)
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: two-hop route, corrupt hop-0 activations — `_final_text_node` blames hop-1 — test must fail.
|
||||||
|
2. Green: bisection selects hop-0; forfeit targets hop-0 wallet.
|
||||||
|
3. On-demand: commitment requested only when audit flag set on proxied request.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Audit requests carry tracker RNG/VRF flag indistinguishable from normal traffic (research §6)
|
||||||
|
- [ ] Nodes retain recent boundary activations for on-demand commit window (configurable TTL)
|
||||||
|
- [ ] Validator/tracker compares fingerprints at each hop cut-point; first mismatch = culprit
|
||||||
|
- [ ] `_final_text_node` removed or limited to text-only fallback
|
||||||
|
- [ ] Integration test: multi-hop pipeline, fault injected at known hop
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §3–4
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `06-fraud-toploc-integration.md`
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 08 — FRAUD: Reputation model + persistence
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Implement graduated reputation per ADR-0018 §6: score derives only from tracker audit outcomes + uptime/latency. Slow build, instant loss, inactivity decay. ×0.8 routing multiplier per strike (not whole penalty — forfeiture stays full pending).
|
||||||
|
|
||||||
|
**Scope split:** issue 05 owns **schema + SQLite persistence**; this issue owns **scoring rules** (deltas, decay, strike→multiplier wiring) on top of persisted fields.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/contracts/meshnet_contracts/__init__.py` — extend `RegistryWallet` / persistence from issue 05
|
||||||
|
- `packages/validator/meshnet_validator/__init__.py` — `_slash_route` forfeiture path (~125–133)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292)
|
||||||
|
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §6
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: no reputation field — add schema + default 1.0 for new wallets.
|
||||||
|
2. Green: clean audit +0.05 (tunable); failed audit −0.3 and strike; three strikes → ban persisted.
|
||||||
|
3. Inactivity decay after N days without completed jobs.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `reputation_score` persisted in registry SQLite (issue 05)
|
||||||
|
- [ ] Audit pass/fail updates score with documented deltas
|
||||||
|
- [ ] Strike applies ×0.8 multiplier to routing weight (separate from forfeiture amount)
|
||||||
|
- [ ] Ban at 3 strikes; probation job count still enforced
|
||||||
|
- [ ] No peer-to-peer reputation inputs
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §6
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `05-a1-a5-persist-strike-ban-reputation.md`
|
||||||
|
- `07-fraud-commitment-bisection-blame.md` (audit outcomes feed reputation)
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 09 — FRAUD: Reputation-weighted routing + adaptive audit rate
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Wire reputation into route selection and audit sampling. Default network audit budget ≈5% — **not a cap**. New/low-reputation nodes: 20–30% audit rate; veterans: 2–3% floor ≥2%. Tripwires escalate rate without direct punishment.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — route selection `_select_route`, `_effective_throughput` (~1747, routing helpers)
|
||||||
|
- `packages/validator/meshnet_validator/__init__.py` — `sample_rate=0.05`
|
||||||
|
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §1.1, §6, §8 layers 2–4
|
||||||
|
|
||||||
|
Audit selection must be unpredictable at request time (tracker RNG after commitment window opens).
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: uniform 5% sample regardless of reputation — test expects higher rate for low-reputation wallet.
|
||||||
|
2. Green: budget balancer keeps fleet-wide average ≈ configured target.
|
||||||
|
3. Routing prefers higher reputation among equal throughput candidates.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Per-wallet audit probability function of reputation (newcomer high, veteran low, floor ≥2%)
|
||||||
|
- [ ] Fleet-wide audit budget configurable (~5% default target); over ≥1000 requests with fixed seed, measured fleet audit rate within **±1.0 percentage point** of configured target (e.g. 4.0–6.0% at 5% default)
|
||||||
|
- [ ] Route scoring includes reputation multiplier (earnings scale with tenure)
|
||||||
|
- [ ] Passive tripwire flags (perplexity/repetition) bump audit rate only
|
||||||
|
- [ ] Tests: deterministic seed for sampling distribution checks
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §1, §6–7
|
||||||
|
- [ADR-0013](../../docs/adr/0013-rolling-stats-smart-routing.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `08-fraud-reputation-model-persistence.md`
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 10 — FRAUD: Penalty calibration wiring (forfeit + strike + ban)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
End-to-end wiring: confirmed audit failure → atomic pending forfeiture + strike + reputation decay + audit-rate snap to max. Ensure payout cannot race penalty (ADR-0015). Document 19× deterrence math in validator README.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/validator/meshnet_validator/__init__.py` — `_slash_route` (~102–134)
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_forfeit` (~2429–2464)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292), payout exclusion for banned (~3337–3344 in settlement loop)
|
||||||
|
- `packages/validator/README.md` — update 20× → 19× at p=0.05
|
||||||
|
|
||||||
|
Per ADR-0018: **full pending forfeiture** is primary penalty; ×0.8 is routing decay per strike, not partial forfeit.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: integration from issue 34 — extend with multi-hop blame wallet from issue 07.
|
||||||
|
2. Green: node with pending balance → audit fail → pending zero, strike++, banned on 3rd, excluded from next settlement.
|
||||||
|
3. Settlement loop skips banned wallets (~3337–3344).
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Audit failure triggers forfeiture + strike in one tracker transaction
|
||||||
|
- [ ] Banned nodes excluded from `payables` / settlement
|
||||||
|
- [ ] Validator uses authenticated forfeit endpoint (issue 02)
|
||||||
|
- [ ] README: `L > 19× g` at p=0.05; pending balance = collateral
|
||||||
|
- [ ] Integration test: 60-request fraud scenario → ban within threshold
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md)
|
||||||
|
- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||||
|
- Research: [research-verifiable-inference.md](../research-verifiable-inference.md) §1.1
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `07-fraud-commitment-bisection-blame.md`
|
||||||
|
- `08-fraud-reputation-model-persistence.md`
|
||||||
|
- `02-a2-unified-auth-boundary.md`
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 11 — C6: Wallet binding ownership proof + binding overwrite safety
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
`POST /v1/wallet/register` binds a client Solana wallet to an API key for deposit attribution. Today any Bearer key can bind any wallet string without proving ownership. Prevent hijack and accidental overwrite.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_handle_wallet_register` (~2625–2648)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `bind_wallet` (~153+), `_wallet_bindings` / direct overwrite on apply (~351)
|
||||||
|
|
||||||
|
Require signed message from wallet pubkey (ed25519 via `cryptography` / solders). Reject rebinding without admin or signed release. Use explicit overwrite policy — today `~351` overwrites binding directly; gossip apply must reject conflicting binds instead of silently clobbering.
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: bind wallet A with only API key, no signature — must fail after fix.
|
||||||
|
2. Red: wallet already bound to key1; key2 cannot steal without proof.
|
||||||
|
3. Green: valid signature binds; deposit watcher credits correct API key.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Wallet binding requires cryptographic proof of pubkey ownership
|
||||||
|
- [ ] One wallet → one API key (or documented admin override)
|
||||||
|
- [ ] Gossip `bind` events cannot overwrite existing binding via direct overwrite at `~351`
|
||||||
|
- [ ] Tests with deterministic keypairs (local adapter)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md) §5
|
||||||
|
- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md`
|
||||||
|
- `03-c5-starting-credit-zero.md`
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Status: ready-for-human
|
||||||
|
|
||||||
|
# 12 — C2: On-chain settlement idempotency (deferred)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Harden payout idempotency so Solana transaction retries never double-pay. Design accepted in ADR-0019 §1; **implementation deferred post-alpha**.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — `_settlement_loop` resend (~3331–3356), `_send_settlement` (~3358–3376)
|
||||||
|
- `packages/contracts/meshnet_contracts/solana_adapter.py` — `send_payouts` (~186–213)
|
||||||
|
|
||||||
|
Today: pending debited before broadcast with stable `settlement_id`; unconfirmed batches resent. Gap: on-chain confirmation vs ledger state if tx succeeds but confirm fails.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `confirm_settlement` only after RPC finalized confirmation
|
||||||
|
- [ ] Retry path reuses same `settlement_id` and detects already-confirmed signature
|
||||||
|
- [ ] Property test: N retries → single on-chain transfer per wallet per settlement_id
|
||||||
|
- [ ] Document recovery procedure for stuck unconfirmed batches
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §1
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
Alpha release (ADR-0016 single settlement tracker)
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
Status: ready-for-human
|
||||||
|
|
||||||
|
# 13 — C3/C4: Consensus-gated money mutations (deferred)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Route money-affecting ledger events through Raft commit, not gossip-only apply. Extend `raft.py` command set beyond register/deregister. Settlement remains leader-only with treasury key.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py` — settlement leader gate (~3331–3332), payout batch (~3353–3356)
|
||||||
|
- `packages/tracker/meshnet_tracker/raft.py` — log entry types (~26–27)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `apply_events` (~301–311)
|
||||||
|
|
||||||
|
Design: ADR-0019 §2. **Deferred post-alpha** while single operator holds settlement.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `charge`, `payout`, `forfeit`, `credit`, `settlement`, `bind` commit via Raft log
|
||||||
|
- [ ] Followers reject direct gossip money mutations
|
||||||
|
- [ ] Leader-only `_settlement_loop` unchanged in semantics
|
||||||
|
- [ ] Migration plan from gossip-only billing to Raft-backed log
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §2
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `12-c2-on-chain-idempotency.md`
|
||||||
|
- `14-a3-raft-durable-term-vote.md`
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
Status: ready-for-human
|
||||||
|
|
||||||
|
# 14 — A3: Durable Raft term and vote state (deferred)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Persist Raft `currentTerm`, `votedFor`, and log metadata to disk. In-memory-only term (~26) risks split leadership after tracker restart → duplicate settlement epochs.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/raft.py` — `LogEntry.term` (~25–27), election state in `RaftNode`
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Term/vote persisted alongside tracker data dir
|
||||||
|
- [ ] Restart resumes as follower/candidate with monotonic term
|
||||||
|
- [ ] Test: kill leader mid-settlement, restart, no duplicate payout batch
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §3
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
Alpha single-settlement posture
|
||||||
27
.scratch/alpha-hardening/issues/15-h1-commutative-forfeit.md
Normal file
27
.scratch/alpha-hardening/issues/15-h1-commutative-forfeit.md
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
Status: ready-for-human
|
||||||
|
|
||||||
|
# 15 — H1: Commutative forfeit event ordering (deferred)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Define deterministic ordering when `forfeit`, `charge`, and `payout` events replicate concurrently. Forfeit snapshots amount at creation (~287) but apply order can desync pending balances under gossip.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292), `_apply_locked` forfeit branch (~345–349)
|
||||||
|
- `packages/tracker/meshnet_tracker/billing.py` — `_pending_since.setdefault` (~324), wallet bind direct overwrite (~351)
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Documented commit order: charges before forfeit before payout for same wallet epoch
|
||||||
|
- [ ] Forfeit events carry pending snapshot or `(term, index)` for tie-break
|
||||||
|
- [ ] `setdefault` replaced with explicit merge rules on out-of-order apply
|
||||||
|
- [ ] Property tests under shuffled event delivery
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §4
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `13-c3-c4-consensus-gated-settlement.md`
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 16 — DOC: US-006 reconciliation note
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Reconcile stale US-006 (Solana testnet stake contracts) with ADR-0015/0016 devnet custodial settlement. Issue `docs/issues/06-solana-stake-and-settlement.md` says "never devnet"; ADR-0015 explicitly targets devnet mock-USDT.
|
||||||
|
|
||||||
|
Also reconcile legacy fraud issues with the alpha-hardening fraud arc:
|
||||||
|
|
||||||
|
- `docs/issues/07-fraud-detection-slash.md` — on-chain stake slash model superseded by pending-balance forfeiture + TOPLOC (ADR-0018)
|
||||||
|
- `docs/issues/34-forfeiture-penalty.md` — partially implemented; remaining fraud work lives in `.scratch/alpha-hardening/issues/06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Add reconciliation comment atop `docs/issues/06-solana-stake-and-settlement.md` (Status: superseded for alpha — see ADR-0015, issue 33/34)
|
||||||
|
- [ ] Add **superseded** banner atop `docs/issues/07-fraud-detection-slash.md` → ADR-0018 + issues 06–10
|
||||||
|
- [ ] Add **superseded for remaining scope** banner atop `docs/issues/34-forfeiture-penalty.md` → ADR-0018 + issues 06–10 (note done items: basic forfeiture wired)
|
||||||
|
- [ ] Update `docs/prd.json` US-006 description footnote if present
|
||||||
|
- [ ] Cross-link ADR-0015 devnet decision
|
||||||
|
- [ ] No production code changes
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
None
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 17 — DOC: Duplicate US-020 issue dedup
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Two files share the US-020 number with different slugs:
|
||||||
|
|
||||||
|
- `docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md` (ready-for-agent)
|
||||||
|
- `docs/issues/20-tracker-node-hardening.md` (done)
|
||||||
|
|
||||||
|
Resolve numbering collision without losing history.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Document canonical mapping in this issue's Comments or a short `docs/issues/README.md` note
|
||||||
|
- [ ] Renumber or prefix disambiguation (e.g. keep done item as US-020a, renumber memory-budget to next slot) — **human approval before git mv**
|
||||||
|
- [ ] Update any prd.json / cross-links that reference US-020 ambiguously
|
||||||
|
- [ ] No production code changes
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
Human approval for renumbering
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 18 — DOC: Operational runbooks (stubs)
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Add operational runbook stubs for alpha operators under `docs/runbooks/` (or `.scratch/alpha-hardening/runbooks/` until close-feature):
|
||||||
|
|
||||||
|
1. **Ledger backup** — billing SQLite, accounts SQLite, registry DB paths; gossip pause procedure
|
||||||
|
2. **Treasury key rotation** — devnet mock-USDT mint + treasury keypair rotation without double-credit
|
||||||
|
3. **Upgrade path** — tracker rolling restart with persisted strike/reputation (post issue 05)
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Three markdown runbook stubs with prerequisites, steps, rollback
|
||||||
|
- [ ] Reference ADR-0015 settlement loop and ADR-0016 trust assumptions
|
||||||
|
- [ ] Secrets handling: never commit `.env.devnet`, keypairs
|
||||||
|
- [ ] No production code changes
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
None (stubs can land before issue 05; update after persistence ships)
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 19 — DOC: Cryptography dependency + test environment note
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Document and verify test/dev environment setup for wallet crypto paths. `packages/node/meshnet_node/wallet.py` uses `cryptography`; failures occur when `.venv` lacks deps.
|
||||||
|
|
||||||
|
**Code refs:**
|
||||||
|
|
||||||
|
- `packages/node/pyproject.toml` — `cryptography>=41` (verify declared)
|
||||||
|
- `packages/node/meshnet_node/wallet.py`
|
||||||
|
- Handoff: tests fail without `cryptography`, `openai`, `langchain` in `.venv`
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Confirm `cryptography` in node package deps; add to root/dev extras if tests import wallet without node install
|
||||||
|
- [ ] Add short **Test environment** section to `CONTRIBUTING.md` or `docs/dev/test-env.md`: use `.venv/Scripts/python.exe`, `pip install -e packages/node ...`, optional dep skips
|
||||||
|
- [ ] Note which tests require optional deps (`--ignore=test_openai_gateway,...`)
|
||||||
|
- [ ] No unrelated production code changes
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
None
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 20 — Validator service token for `/v1/billing/forfeit`
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Define and implement a **validator service token** distinct from client API keys and admin sessions. The validator process must authenticate when calling `POST /v1/billing/forfeit`; arbitrary Bearer strings and client API keys must be rejected.
|
||||||
|
|
||||||
|
Per [ADR-0017 §4](../../docs/adr/0017-tracker-authentication-and-authorization.md): forfeit accepts **validator service identity or admin session** only.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
| Item | Alpha default |
|
||||||
|
|---|---|
|
||||||
|
| Env var | `MESHNET_VALIDATOR_SERVICE_TOKEN` (tracker + validator) |
|
||||||
|
| Config flag | `--validator-service-token` / tracker config file equivalent |
|
||||||
|
| Header format | `Authorization: Bearer <service-token>` with a dedicated prefix or separate header scheme documented in runbooks (e.g. `Authorization: Service <token>` — pick one and test consistently) |
|
||||||
|
| Rotation | Manual: set new token on tracker + validator, restart both; document zero-downtime rotation as post-alpha |
|
||||||
|
|
||||||
|
## Rejection rules
|
||||||
|
|
||||||
|
- Client API keys (`sk-mesh-…`) → **403** on forfeit (even if valid for inference)
|
||||||
|
- Non-empty garbage Bearer → **401/403**
|
||||||
|
- Missing auth → **401**
|
||||||
|
- Valid validator service token → **200** (existing forfeit semantics)
|
||||||
|
- Admin session → **200** (operator override)
|
||||||
|
|
||||||
|
## Test-first
|
||||||
|
|
||||||
|
1. Red: validator (or test client) posts forfeit with a valid API key — must fail after fix.
|
||||||
|
2. Red: `Authorization: Bearer garbage` — must fail (covered by issue 02; this issue defines the accepted token).
|
||||||
|
3. Green: configured service token succeeds; wrong token fails.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Service token configurable via env/flag on tracker and validator
|
||||||
|
- [ ] Unified auth middleware resolves service token → `validator` role (issue 02)
|
||||||
|
- [ ] API keys explicitly rejected on forfeit path
|
||||||
|
- [ ] Integration test: validator client with service token forfeit succeeds; API key forfeit fails
|
||||||
|
- [ ] Runbook stub: rotation procedure (manual alpha)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md) §4
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md` — middleware + role checks
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `02-a2-unified-auth-boundary.md`
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
Status: ready-for-human
|
||||||
|
|
||||||
|
# 21 — Honest-noise TOPLOC calibration corpus
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Before enabling production TOPLOC audit thresholds, collect an **honest-noise baseline** across the heterogeneous volunteer fleet. Run identical inference jobs on every active node/GPU combo; measure the divergence envelope (TOPLOC exponent/mantissa deltas, logprob-rank spread) under real hardware variance.
|
||||||
|
|
||||||
|
Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and-reputation.md): threshold calibration requires an honest-noise corpus across the fleet before production thresholds.
|
||||||
|
|
||||||
|
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
||||||
|
|
||||||
|
## Deliverables
|
||||||
|
|
||||||
|
- [ ] Scripted benchmark job (fixed prompt, model preset, seed policy) runnable on all nodes
|
||||||
|
- [ ] Aggregated corpus artifact (per node: GPU model, dtype, TOPLOC deltas vs reference)
|
||||||
|
- [ ] Recommended tolerance thresholds documented (p99 honest envelope + safety margin)
|
||||||
|
- [ ] Gate checklist: production audit enable blocked until corpus covers ≥N distinct hardware profiles (define N in runbook, suggest ≥3)
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Corpus collected from current fleet (or documented subset + extrapolation note)
|
||||||
|
- [ ] Threshold constants in validator config derived from corpus, not guessed
|
||||||
|
- [ ] False-positive rate estimate documented at chosen thresholds
|
||||||
|
- [ ] README / runbook cross-link: **do not enable production audits** until this issue closes
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) — Consequences (honest-noise corpus)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
- `06-fraud-toploc-integration.md` (TOPLOC wired; calibration uses same primitive)
|
||||||
|
|
||||||
|
## Blocks (prod gate)
|
||||||
|
|
||||||
|
- Production enable of adaptive audit thresholds (issues 09–10 in prod)
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# 22 — DOC: MEMORY.md + project-status alpha-hardening index
|
||||||
|
|
||||||
|
## What to build
|
||||||
|
|
||||||
|
Update persistent memory files so agents and humans find the alpha-hardening feature without stale handoff paths.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] `.claude/memory/MEMORY.md` — index entry for alpha-hardening (`.scratch/alpha-hardening/`, ADRs 0016–0019, issue count)
|
||||||
|
- [ ] `.claude/memory/project-status.md` — brief alpha-hardening section: planning complete, Bucket 1 blockers next, link README
|
||||||
|
- [ ] Cross-link `.scratch/alpha-hardening/handoff.md` from README (not temp path)
|
||||||
|
|
||||||
|
## ADR links
|
||||||
|
|
||||||
|
- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)
|
||||||
|
|
||||||
|
## Blocked by
|
||||||
|
|
||||||
|
None — parallel with other Bucket 3 doc issues
|
||||||
281
.scratch/alpha-hardening/research-verifiable-inference.md
Normal file
281
.scratch/alpha-hardening/research-verifiable-inference.md
Normal file
@@ -0,0 +1,281 @@
|
|||||||
|
# Verifiable Inference & Fraud Layer — Research Findings
|
||||||
|
|
||||||
|
**Date:** 2026-07-04
|
||||||
|
**Scope:** Ground the neuron-tai fraud/verification layer in 2026 state-of-the-art and in what comparable decentralized compute networks actually do.
|
||||||
|
**Alpha context assumed throughout:** volunteer consumer GPUs (8 GB class, e.g. RTX 4060), pipeline-parallel multi-hop inference where different nodes hold different layer ranges, a single trusted tracker (coordinator), devnet mock-USDT payments but reputation that carries forward, and an owner-approved verification budget of roughly 5% redundant recomputation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Executive summary
|
||||||
|
|
||||||
|
1. **The proven, deployed pattern for exactly our setting is: optimistic acceptance + random teacher-forced re-verification + slashing/reputation penalties.** This is what Prime Intellect runs in production for INTELLECT-2 (TOPLOC validators, random sampling, slash-and-evict) and what Hyperbolic formalized game-theoretically (Proof of Sampling). At a 5% audit rate the math works if the penalty for a caught cheat exceeds ~19× the per-job gain — which carried-forward reputation plus forfeited pending balance easily provides.
|
||||||
|
2. **Teacher-forced logit/activation re-verification is a recognized, robust technique** — it is the core of TOPLOC (ICML 2025), Token-DiFR (2025), and log-probability auditing papers. It sidesteps autoregressive divergence entirely and is up to ~100× cheaper than the original generation because verification prefill is compute-bound while generation is memory-bound.
|
||||||
|
3. **Bitwise output equality is unattainable across heterogeneous volunteer GPUs**; honest nodes on different hardware/batch conditions produce different tokens even at temperature 0. All robust schemes compare in *logit/activation space with tolerances* (TOPLOC exponent/mantissa thresholds, DiFR divergence scores) or force bitwise reproducibility with special kernels (Gensyn RepOps) at a performance cost.
|
||||||
|
4. **zkML and GPU TEEs are roadmap-only.** zkML in 2025–2026 proves GPT-2-scale models in tens of seconds per inference (100–10,000× overhead); nothing near serving speed for multi-billion-parameter models. GPU confidential computing exists only on H100/Blackwell-class datacenter hardware paired with server CPUs (SEV-SNP/TDX) — categorically unavailable on consumer volunteer cards.
|
||||||
|
5. **Layer-skipping is NOT reliably detectable by output quality.** Careful pruning of ~25% of layers keeps ~90% of benchmark scores (ShortGPT); models degrade gracefully until a sharp collapse around 20–55% removal depending on family. But any layer skipping changes hidden states and logits drastically, so per-position activation/logit comparison catches it essentially always (TOPLOC reports 100% detection of model modifications in its evals). Verify in logit space, not by eyeballing text quality.
|
||||||
|
6. **Classic PoW is Sybil/hardware admission control, not correctness proof** — io.net's hourly PoW is the cautionary tale (it verifies a GPU exists and has claimed VRAM, nothing about job correctness). Self-computed activation checksums are *binding commitments*, not correctness proofs: a cheater simply commits to its wrong values. Commitments earn their keep only when a referee later recomputes and compares — i.e., for audit pinning and dispute bisection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Technique-by-technique review
|
||||||
|
|
||||||
|
### 1.1 Optimistic verification + spot-checking (probabilistic deterrence)
|
||||||
|
|
||||||
|
**Mechanism.** Accept results by default; re-run a random, unpredictable subset with an independent replica; punish divergences. Hyperbolic's Proof of Sampling (PoSP) paper proves a pure-strategy Nash equilibrium in which rational nodes are honest, and claims the spot-check approach adds well under 1% overhead when everyone is rational ([PoSP, arXiv:2405.00295](https://arxiv.org/html/2405.00295); [Hyperbolic spML breakdown](https://www.hyperbolic.ai/blog/spml-breakdown)). VeriLLM adds VRF-selected audit indices over Merkle-committed hidden states so verifiers can't be lazy or predictable ([VeriLLM, arXiv:2509.24257](https://arxiv.org/html/2509.24257v3)). Prime Intellect's production deployment: validators randomly sample committed batches, and "since the Inference Provider does not know which generations will be checked, they are incentivized to be honest on all generations" ([INTELLECT-2, arXiv:2505.07291](https://arxiv.org/html/2505.07291)).
|
||||||
|
|
||||||
|
**Sampling math.** With independent audit probability *p* per job:
|
||||||
|
|
||||||
|
- A node that cheats on every job survives *N* jobs undetected with probability (1−p)^N. At p = 0.05: ~36% survive 20 jobs, ~4.6% survive 60 jobs, ~0.6% survive 100 jobs. Detection is near-certain over any meaningful volume.
|
||||||
|
- A node that cheats on a fraction *f* of jobs is caught per job with probability *p·f*; expected jobs until caught = 1/(p·f). At p = 0.05, f = 0.2: caught within ~100 jobs on average. Low-rate cheating stretches time-to-detection linearly — this is why the penalty must scale with accumulated stake/reputation, not per-incident.
|
||||||
|
- **Deterrence condition (rational node):** cheating gains *g* per job and loses *L* when caught, so expected value of cheating is (1−p)·g − p·L. Honesty dominates iff **L > g·(1−p)/p ≈ 19·g at p = 0.05**. If a caught node forfeits its pending balance and a reputation that gates future routing (i.e., discounted future earnings), L is orders of magnitude above 19× a single job's margin. This is exactly the structure PoSP formalizes and the existing neuron-tai forfeiture design (pending-balance forfeiture + strikes + ban) already implements.
|
||||||
|
|
||||||
|
**The verifier-incentive caveat.** Truebit identified the classic failure mode: if the system works and nobody cheats, verifiers never earn anything and stop checking; Truebit's answer was "forced errors" with jackpot payouts, and it estimated the verification tax at 500–5000% of task cost to keep independent verifiers attentive ([Truebit whitepaper §2, §4](http://people.cs.uchicago.edu/~teutsch/papers/truebit.pdf)). **This whole problem disappears in a single-tracker alpha**: the tracker audits as a protocol function paid from the protocol cut, not as a profit-seeking third party. It becomes relevant again only when verification is decentralized.
|
||||||
|
|
||||||
|
**Feasibility for alpha: HIGH.** This is the anchor technique. 5% audit budget is in the same range as deployed systems (OTR proposes ρ ≈ 1% spot-checks as sufficient deterrent in a TEE-hybrid design — [arXiv:2512.20176](https://doi.org/10.48550/arxiv.2512.20176)); 5% gives comfortable margin for a young network with thin reputational stakes.
|
||||||
|
|
||||||
|
### 1.2 Interactive verification / refereed delegation (Truebit-style bisection)
|
||||||
|
|
||||||
|
**Mechanism.** Solver commits Merkle roots of intermediate execution states. On dispute, challenger and solver binary-search over the committed states to the first step where they diverge; a referee (smart contract or trusted party) recomputes only that single step and rules. One honest challenger suffices; the game costs O(log n) rounds ([Truebit whitepaper](http://people.cs.uchicago.edu/~teutsch/papers/truebit.pdf); [Truebit explainer, Medium](https://medium.com/truebit/truebit-the-marketplace-for-verifiable-computation-f51d1726798f)).
|
||||||
|
|
||||||
|
**ML adaptation.** Gensyn's Verde adapts this to neural networks with a two-phase bisection: first narrow to the training/inference *step*, then to the single *operator* in the computational graph; the referee recomputes one operator, needing ~two orders of magnitude less compute than the full job. Crucially, Verde only works because RepOps makes execution **bitwise reproducible across heterogeneous hardware** by fixing floating-point operation order — otherwise honest nodes diverge and bisection finds spurious "fraud" ([Verde paper, arXiv:2502.19405](https://arxiv.org/html/2502.19405v1); [Gensyn Verde blog](https://blog.gensyn.ai/verde-a-verification-system-for-machine-learning-over-untrusted-nodes/)).
|
||||||
|
|
||||||
|
**Mapping to our multi-hop pipeline.** Our pipeline is naturally pre-bisected: each hop's boundary activations are the "intermediate states." A workable scheme:
|
||||||
|
|
||||||
|
1. Each node commits a compact fingerprint (TOPLOC-style top-k encoding, not a raw hash — see §2) of its input and output boundary activations per request.
|
||||||
|
2. When an end-to-end audit fails, the referee (tracker) teacher-forces the full claimed token sequence through a reference model and computes reference boundary activations at each hop cut-point in one forward pass.
|
||||||
|
3. The first hop whose committed output fingerprint diverges from the reference (beyond tolerance) while its input fingerprint matched is the culprit. No interactive game is needed at hop granularity because the number of hops is small — the referee can check all cut-points in a single replay.
|
||||||
|
4. Bisection *within* a hop (to a layer or operator, Verde-style) is only needed if you must prove fault to a third party at fine granularity; for internal blame assignment, hop granularity is enough since payment and reputation are per node.
|
||||||
|
|
||||||
|
The main adaptation cost is the referee needing enough VRAM to run the full model (or to re-run hop layer-ranges one at a time, which any 8 GB card can do for its own range).
|
||||||
|
|
||||||
|
**Feasibility for alpha: MEDIUM-HIGH** in the simplified "referee replays and compares committed hop boundaries" form (no on-chain game, tracker as referee). The full trustless interactive game with bitwise-reproducible kernels is roadmap.
|
||||||
|
|
||||||
|
### 1.3 zkML (zero-knowledge proofs of inference)
|
||||||
|
|
||||||
|
**2025–2026 state.** The frontier moved fast but remains far from LLM serving speed:
|
||||||
|
|
||||||
|
- EZKL (ONNX→Halo2) is the most-used toolkit; generic overhead is characterized as 100–10,000× native execution, and much real usage is verifiable data science rather than LLMs ([Equilibrium Labs survey](https://equilibrium.co/writing/state-of-verifiable-inference)).
|
||||||
|
- zkGPT (USENIX Security 2025) proves a GPT-2 inference in **under 25 seconds** on a CPU server — 185× faster than the prior ZKML system ([zkGPT](https://www.usenix.org/conference/usenixsecurity25/presentation/qu-zkgpt)).
|
||||||
|
- NanoZK (2026) proves GPT-2-scale transformer blocks in ~43 s with 6.9 KB proofs via layerwise decomposition, 52–228× faster than EZKL ([NanoZK, arXiv:2603.18046](https://arxiv.org/html/2603.18046v1)).
|
||||||
|
- For a 7B model, one estimate puts pure-zkML verification at **over 20 minutes per inference** ([Optimistic TEE-Rollups, arXiv:2512.20176](https://doi.org/10.48550/arxiv.2512.20176)).
|
||||||
|
- Project reality check: Modulus Labs (authored the field-defining benchmark ["The Cost of Intelligence"](https://eprint.iacr.org/2026/1063), demonstrated proving of a multi-billion-parameter LLM as a feat, not a service) was acquired by Tools for Humanity in 2024 and now does World ID cryptography ([announcement](https://world.org/blog/announcements/modulus-labs-joins-tfh-support-applied-research-world)). Giza uses ZK proofs for *small, simple* DeFi-agent models on Starknet ([Equilibrium survey](https://equilibrium.co/writing/state-of-verifiable-inference)). EZKL itself has drifted toward verifiable data science and co-SNARKs.
|
||||||
|
|
||||||
|
**Honest read:** proving is ~GPT-2 scale (~1.5B params max in tens of seconds), on beefy servers, per short inference. For a volunteer network serving multi-billion-parameter models interactively, zkML is 3+ orders of magnitude away, and proof generation would crush an 8 GB consumer card regardless.
|
||||||
|
|
||||||
|
**Feasibility for alpha: NONE (roadmap-only).**
|
||||||
|
|
||||||
|
### 1.4 TEE / confidential computing on GPUs
|
||||||
|
|
||||||
|
**What it gives.** NVIDIA H100 is the first GPU with a hardware TEE anchored in an on-die root of trust: measured/secure boot, SPDM session with a driver inside a CPU confidential VM, and a signed **attestation report** covering GPU firmware/state that a remote party verifies against NVIDIA's attestation service before releasing work ([NVIDIA blog](https://developer.nvidia.com/blog/confidential-computing-on-h100-gpus-for-secure-and-trustworthy-ai/); [CACM: Creating the First Confidential GPUs](https://cacm.acm.org/practice/creating-the-first-confidential-gpus/)). Combined with CPU TEE attestation (AMD SEV-SNP or Intel TDX measuring the VM image), you get a verifiable claim that *a specific measured software stack loaded a specific model and ran on genuine hardware*, with only ~2–6% overhead (Blackwell figures, [GPUYard setup guide](https://www.gpuyard.com/tutorials/howto/nvidia-blackwell-confidential-computing-setup/)) or ~5–10% generally ([Equilibrium survey](https://equilibrium.co/writing/state-of-verifiable-inference)). Phala runs exactly this to sell "verifiable LLMs" through OpenRouter ([Phala blog](https://phala.com/posts/GPU-TEEs-is-Alive-on-OpenRouter)).
|
||||||
|
|
||||||
|
**Caveats.** Attestation proves the *environment*, not the mathematics — you trust NVIDIA/AMD/Intel hardware and are exposed to side channels and firmware bugs (the OTR paper layers ZK spot-checks on top of TEEs precisely because of compromised-TEE risk). And critically for us: CC mode requires Hopper/Blackwell datacenter GPUs plus server platforms with SEV-SNP/TDX enabled in BIOS and the open kernel modules ([NVIDIA deployment guide](https://docs.nvidia.com/cc-deployment-guide-tdx-snp.pdf)). **No RTX consumer card supports it.** Volunteer nodes on 4060-class hardware categorically cannot provide GPU TEE attestation; Intel SGX on client CPUs is likewise not a path (deprecated on consumer parts, and it would only cover CPU-side code anyway).
|
||||||
|
|
||||||
|
**Feasibility for alpha: NONE for volunteer nodes.** Roadmap option: a *TEE tier* — if datacenter H100/Blackwell operators ever join, attest them and route confidentiality-sensitive or high-value jobs there; also usable for the tracker's own reference validator.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. The non-determinism problem for output comparison
|
||||||
|
|
||||||
|
### 2.1 Why honest nodes disagree
|
||||||
|
|
||||||
|
- **Floating-point non-associativity** is the root cause, but the *mechanism* that surfaces it is batch-variant kernels: inference servers pick different kernels/split strategies depending on batch size and load, changing reduction order and hence results — the same request returns different logits depending on what else is in the batch ([Thinking Machines: Defeating Nondeterminism in LLM Inference](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)).
|
||||||
|
- **Even greedy (temperature-0) decoding diverges across configurations**: changing batch size, GPU count, GPU architecture, or tensor-parallel size measurably changes outputs under greedy decoding, and small numerical differences get **amplified autoregressively** — one flipped token early in a long generation cascades into a completely different continuation ([arXiv:2506.09501](https://arxiv.org/html/2506.09501v2); [arXiv:2511.17826](https://arxiv.org/html/2511.17826v2)).
|
||||||
|
- Temperature > 0 sampling adds outright randomness on top unless seeds and the sampling algorithm (e.g. Gumbel-max in vLLM) are synchronized.
|
||||||
|
|
||||||
|
**Consequence:** naive "re-run the prompt, compare the text" audits will flag honest heterogeneous nodes as cheaters. Never compare free-running generations token-for-token across different hardware.
|
||||||
|
|
||||||
|
### 2.2 Techniques that make honest-vs-honest comparison robust
|
||||||
|
|
||||||
|
1. **Teacher-forced re-verification (per-position logit/activation check) — yes, this is a recognized, state-of-the-art technique.**
|
||||||
|
- **TOPLOC** (Prime Intellect, ICML 2025): the prover commits top-k values/indices of the last hidden state per decode chunk (258 bytes per 32 tokens, ~1000× smaller than raw activations); the validator *re-runs the claimed token sequence as a single prefill* (teacher forcing) and checks the committed top-k against recomputed values using exponent/mantissa error thresholds. Reported: 100% detection of model, prompt, or precision substitutions with zero false positives across different GPUs, tensor-parallel layouts, and attention implementations, and validation up to **100× faster than the original inference** because prefill is compute-bound while decode is memory-bound ([TOPLOC, arXiv:2501.16007](https://arxiv.org/html/2501.16007v1); [GitHub](https://github.com/PrimeIntellect-ai/toploc)).
|
||||||
|
- **Token-DiFR / Activation-DiFR** (2025): synchronize the sampling seed (Gumbel-max), then compare the provider's tokens against a reference re-generation — with a shared seed, >98% of tokens match exactly for honest providers, so token divergence itself becomes the audit signal; detects 4-bit quantization with AUC > 0.999 within 300 tokens. Activation-DiFR compresses activations with random orthogonal projections and detects the same with 2 tokens. Released as a vLLM integration ([DiFR, arXiv:2511.20621](https://arxiv.org/pdf/2511.20621); [author explainer](https://technicallyprivate.substack.com/p/token-difr-llm-inference-verification)).
|
||||||
|
- **Log-probability auditing**: comparing per-token logprob distributions against a reference model detects even single-step fine-tuning or quantization changes; simple statistical tests (permutation test on per-token mean logprobs, KS tests) overcome logprob noise ([Logprob Tracking, arXiv:2512.03816](https://arxiv.org/html/2512.03816v1); [model-substitution audit, arXiv:2504.04715](https://arxiv.org/html/2504.04715v1)).
|
||||||
|
2. **Tolerance-based matching, not bitwise equality.** TOPLOC accepts bounded exponent-intersection and mantissa-error deviations; DiFR scores divergence-from-reference against an honest-noise baseline. Both are explicitly designed so cross-GPU numerical noise passes while quantization/model swaps fail.
|
||||||
|
3. **Beware scalar aggregate thresholds alone.** The DiFR authors note that a single statistic like mean cross-entropy can be gamed — a malicious provider can tune its sampling temperature until the aggregate matches expectation. Seed-synchronized token matching or per-position top-k activation checks leave far fewer degrees of freedom ([DiFR explainer](https://technicallyprivate.substack.com/p/token-difr-llm-inference-verification)).
|
||||||
|
4. **Bitwise-deterministic kernels** (Gensyn RepOps; Thinking Machines' batch-invariant kernels; TBIK for cross-tensor-parallel invariance) make exact comparison possible but require replacing the whole operator stack and sacrifice performance — practical for a controlled verification environment, unrealistic to impose on volunteer nodes running stock llama.cpp/transformers ([Gensyn](https://blog.gensyn.ai/verde-a-verification-system-for-machine-learning-over-untrusted-nodes/); [Thinking Machines](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/); [arXiv:2511.17826](https://arxiv.org/html/2511.17826v2)).
|
||||||
|
|
||||||
|
**Practical recipe for our audits:** record prompt, claimed output tokens, sampling params, and per-hop activation fingerprints. Audit = one teacher-forced forward pass of the claimed tokens on a reference node; compare per-position: (a) hop-boundary fingerprints TOPLOC-style, and/or (b) whether each claimed token is plausible under the reference distribution (rank/logprob within tolerance; exact match if seeds are synchronized). This is cheap (single prefill), robust to hardware noise, and immune to autoregressive divergence because the token sequence is fixed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Layer-skipping detectability
|
||||||
|
|
||||||
|
**How gracefully do transformers degrade when layers are dropped?**
|
||||||
|
|
||||||
|
- **Careful pruning degrades subtly.** ShortGPT removes ~25% of layers (chosen by Block Influence, training-free) and keeps ~90% of benchmark performance — e.g. LLaMA2-13B MMLU drops only 55.0→52.2 after removing 10 of 40 layers ([ShortGPT, arXiv:2403.03853](https://arxiv.org/pdf/2403.03853)). Gromov et al. find a "characteristic flat region of robust performance" on QA benchmarks followed by a **sharp collapse to random accuracy** at a model-dependent threshold: ~45–55% of layers for Llama-2 family, ~35% for Mistral-7B, ~25% for Phi-2, ~20% for Qwen — and up to ~half of Llama-2-70B's layers with healing finetuning ([The Unreasonable Ineffectiveness of the Deeper Layers, arXiv:2403.17887](https://arxiv.org/html/2403.17887v1)). LayerDrop showed a decade of precedent: networks *trained* with layer dropout can be pruned at inference with modest loss ([arXiv:1909.11556](https://arxiv.org/abs/1909.11556)).
|
||||||
|
- **So: would skipping ~36 of 100 layers produce gibberish?** Probably not gibberish. For a Llama-like model, 36% removal of *well-chosen* layers sits below the collapse threshold — output would be degraded but often fluent, and on easy prompts might look acceptable. A lazy cheater skipping a contiguous middle block *without* choosing layers carefully does worse — Gromov et al. show the naive heuristic's loss "begins to increase very rapidly even with small amounts of pruning" before healing — but "worse" still means plausible-looking text on many prompts, not obvious garbage. **Human-eyeball or coarse quality checks are not a reliable fraud detector.**
|
||||||
|
- **Statistical detection: yes, and easily.** Without healing, C4 validation loss rises sharply with pruning fraction (Gromov et al. Fig. 3), so perplexity of the node's outputs under a reference model shifts detectably given enough tokens. But the far stronger signal is direct: skipping any layer changes the hidden states and output logits at *every* position by amounts vastly exceeding floating-point noise. TOPLOC's evaluation explicitly includes detecting modified models with 100% accuracy; DiFR detects even 4-bit quantization (a much subtler perturbation than deleting layers) with AUC > 0.999 in ≤300 tokens. **A single teacher-forced audit pass catches layer-skipping essentially deterministically.**
|
||||||
|
- **Pipeline nuance:** in multi-hop inference, a node that skips layers inside its assigned range corrupts its output boundary activations; the hop-boundary fingerprint comparison in §1.2 localizes exactly which node did it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Why classic PoW does not solve verifiable useful computation
|
||||||
|
|
||||||
|
- **PoW's defining property is verification asymmetry over an *arbitrary* puzzle**: work is hard to produce, trivially cheap to check (hash preimage with leading zeros), and the puzzle content is irrelevant — it exists purely to price Sybil identities and block production ([Wikipedia: Proof of work](https://en.wikipedia.org/wiki/Proof_of_work)). Useful computation generally lacks this asymmetry: checking an ML result naively requires re-executing it, which is the whole problem. This is the central, repeatedly confirmed obstacle in the Proof-of-Useful-Work literature — "the main weakness that all presented PoUW approaches have in common is the verification of results" ([Challenges of PoUW, arXiv:2209.03865](https://doi.org/10.48550/arxiv.2209.03865); [SoK: Is Proof-of-Useful-Work Really Useful?, IACR 2025/1814](https://eprint.iacr.org/2025/1814.pdf)). Expensive verification then triggers the **verifier's dilemma**: rational verifiers skip checking, and security collapses ([Truebit whitepaper §1](http://people.cs.uchicago.edu/~teutsch/papers/truebit.pdf)).
|
||||||
|
- **Live case study:** io.net's hourly "Proof-of-Work" makes GPUs solve hash puzzles and proves *the hardware exists, has the claimed VRAM, and is online* ([io.net PoW docs](https://io.net/docs/guides/workers/proof-of-work)). It was bolted on after the April 2024 Sybil attack (~1.8M fake GPUs spoofed via a shared auth token to farm airdrops) and says nothing about whether any customer job was computed correctly ([io.net 2026 analysis](https://cryptoaianalysis.com/io-net-io-analysis-2026/)). PoW = admission control and capacity attestation; correctness needs a separate mechanism.
|
||||||
|
- **Self-computed checksums/commitments: confirmed — no standalone correctness guarantee.** A hash or Merkle commitment over activations is *binding* (the node can't later change its story) and possibly *hiding*, but the committed values are whatever the node computed — a malicious node honestly commits to its dishonest activations, and the checksum verifies perfectly. Correctness only enters when an independent party recomputes and compares against the commitment. That is precisely how the serious systems use commitments: Truebit's Merkle state roots and Verde's checkpoint hashes exist to make **dispute bisection** possible and to **pin claims before an unpredictable audit** (so a node can't retroactively fix its answer once it learns it's being checked), never as proof by themselves ([Truebit](http://people.cs.uchicago.edu/~teutsch/papers/truebit.pdf); [Verde, arXiv:2502.19405](https://arxiv.org/html/2502.19405v1); TOPLOC commitments are verified by validator recomputation — [arXiv:2501.16007](https://arxiv.org/html/2501.16007v1)).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. What comparable networks actually do
|
||||||
|
|
||||||
|
| Network | Verification of outputs | Sybil / fraud handling | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Prime Intellect** | TOPLOC activation commitments per sequence; validators teacher-force-recompute (sampled, up to 100× faster than generation); plus sampling-sanity and data-sanity checks | Invalid batches → node **slashed and evicted** from compute pool; hardware checks at registration | Deployed at scale for INTELLECT-2 (32B RL run over permissionless nodes) ([arXiv:2505.07291](https://arxiv.org/html/2505.07291)) |
|
||||||
|
| **Gensyn** | Verde refereed delegation: bisect disputes to a single operator; referee recomputes it; requires RepOps bitwise-reproducible kernels | Correct result guaranteed if ≥1 of the assigned providers is honest; economic penalties on losers of disputes | The most rigorous trustless design; reproducibility stack is the price ([Verde](https://arxiv.org/html/2502.19405v1); [docs](https://docs.gensyn.ai/core-components)) |
|
||||||
|
| **Bittensor** | No direct output verification: subnet **validators score miners**; Yuma Consensus aggregates stake-weighted scores into emissions | Registration costs + stake; the notorious **weight-copying** problem (validators free-riding by copying consensus weights) countered by Commit-Reveal v3: timelock-encrypted weights revealed epochs later so copiers only get stale data ([weight copying](https://docs.learnbittensor.org/concepts/weight-copying-in-bittensor); [commit-reveal](https://docs.learnbittensor.org/concepts/commit-reveal)) | Verification quality is per-subnet and heuristic; consensus rewards conformity, not proven correctness |
|
||||||
|
| **io.net** | None for job outputs | Hourly hash-puzzle PoW + VRAM check + Proof of Time-Lock (uptime), added after the Apr 2024 fake-GPU Sybil attack; failed checks → excluded from rewards/hiring ([PoW docs](https://io.net/docs/guides/workers/proof-of-work); [Messari overview](https://messari.io/report/understanding-io-net-a-comprehensive-overview)) | Marketplace model: hardware attestation only, correctness left to the customer |
|
||||||
|
| **Ritual (Infernet)** | Modular: nodes may attach ZK, optimistic, or TEE proofs; consumer contracts opt in ([Ritual blog](https://ritual.net/blog/celestia)) | ChainLight's security review: base Infernet has **no aggregation of multiple nodes, no reputation, no penalty for malicious nodes returning wrong results** ([ChainLight](https://blog.chainlight.io/ecosystem-explorer-exploring-security-risks-in-ai-blockchain-projects-2c490a726d13)) | "Verification-optional" in practice |
|
||||||
|
| **Hyperbolic** | PoSP/spML: random spot-check by a second node; disagreement → arbitration; slashing of the dishonest party ([PoSP, arXiv:2405.00295](https://arxiv.org/html/2405.00295)) | Nash-equilibrium argument that rational nodes stay honest; validator anonymity until finalization to resist collusion | Closest published formalization of our intended scheme |
|
||||||
|
| **Akash** | None (general compute marketplace) | **Auditor-attested provider tiers**: governance-approved auditors post on-chain attestations; no provider staking/slashing — ChainLight flags malicious-provider risk as an accepted gap ([ChainLight](https://blog.chainlight.io/ecosystem-explorer-exploring-security-risks-in-ai-blockchain-projects-2c490a726d13); [Akash docs](https://akash.network/docs/providers/getting-started/should-i-run-a-provider/)) | Reputation = audit badge + market history |
|
||||||
|
| **Petals** (closest architectural cousin: volunteer, pipeline-parallel, consumer GPUs) | **None** — the paper and docs openly state malicious peers can alter outputs; mitigation is "use a private swarm of people you trust"; devs planned a centralized points system for nodes that "consistently return correct results" ([GitHub](http://github.com/bigscience-workshop/petals); [dev comments on HN](https://news.ycombinator.com/item?id=34215665)) | Proof that the architecture works, and that the fraud layer is the missing piece we're building |
|
||||||
|
| **Together AI** | Centralized provider; no public trustless verification protocol — trust is contractual/reputational | n/a | Included for contrast: the "trusted provider" baseline |
|
||||||
|
| **Atoma / Ambient** (emerging) | Atoma: TEE where available, else user-configurable redundant sampling; Ambient: "Proof of Logits" fingerprinting with ~0.1% claimed overhead but no open implementation ([Equilibrium survey](https://equilibrium.co/writing/state-of-verifiable-inference)) | — | Signal of where the field is converging: logit fingerprints + sampling |
|
||||||
|
|
||||||
|
**Pattern:** nobody in production uses zkML for LLM serving; nobody offers TEE on consumer GPUs; every serious inference-verification deployment converges on *commit → sampled teacher-forced recompute → slash/reputation*, with refereed delegation as the trustless escalation path.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Reputation systems for routing and audit-rate weighting
|
||||||
|
|
||||||
|
**Foundations.** EigenTrust computes global trust as the principal eigenvector of normalized local trust ratings and remains the canonical design; in simulation it suppresses malicious collectives up to large fractions of the network, but it depends on pre-trusted peers and majority-honest aggregation ([EigenTrust, WWW'03](https://nlp.stanford.edu/pubs/eigentrust.pdf)).
|
||||||
|
|
||||||
|
**Documented pitfalls** (from EigenTrust's own threat models and the P2P reputation-attack literature — [survey](https://cnitarot.github.io/papers/p2p-reputation-survey.pdf)):
|
||||||
|
|
||||||
|
- **Sybil flooding / whitewashing:** free identities let attackers dominate the "unknown newcomer" pool and shed bad reputations by rejoining. Defense: entry cost (stake, invite, proof-of-hardware), newcomer probation with low routing weight and *elevated* audit rates.
|
||||||
|
- **Reputation farming / milking:** behave honestly on cheap jobs to build score, then cash in on expensive ones ("moles" that interleave honest work are documented in real systems like Maze). Defense: value-weighted reputation (a job's reputation contribution ∝ its audited value), and audit probability that never floors at zero for anyone.
|
||||||
|
- **Collusion rings:** peers mutually inflating ratings. Defense: in our alpha, reputation should derive **only from tracker-verified audit outcomes**, never from peer ratings — this deletes the collusion surface EigenTrust suffers from. (EigenTrust++ hardens propagation with feedback-similarity for when peer ratings do get introduced — [EigenTrust++](https://doi.org/10.4108/icst.collaboratecom.2012.250420).)
|
||||||
|
- **Weight-copying (Bittensor's lesson):** any published evaluation signal will be free-ridden by lazy evaluators; the fix is delaying/encrypting evaluations (commit-reveal timelock) until they're stale ([Bittensor docs](https://docs.learnbittensor.org/concepts/weight-copying-in-bittensor)). Relevant to us only when third-party validators exist; a single tracker has no one to copy from.
|
||||||
|
|
||||||
|
**Reputation-weighted routing + audit rates in practice.** io.net gates hiring and block rewards on rolling verification status; Akash routes via audit-tier filters; Bittensor routes emissions via consensus-weighted scores. The clean design for us, consistent with PoSP-style deterrence:
|
||||||
|
|
||||||
|
- **Routing weight** increases with reputation (and observed TPS, already tracked), so honest nodes earn more — making L (loss-on-catch, §1.1) grow with tenure.
|
||||||
|
- **Audit rate** decreases with reputation but has a hard floor (e.g. new node 20–30% → veteran 2–3%, floor ≥ 2%), keeping the overall budget ≈ 5% while making cheating-as-a-newcomer maximally risky and reputation-then-defect still irrational.
|
||||||
|
- **Asymmetric dynamics:** slow to build, instant to lose (existing 3-strike forfeiture design already matches this), plus decay on inactivity so parked identities can't hoard trust.
|
||||||
|
- **Unpredictability is load-bearing:** audit selection must be indistinguishable from normal traffic at request time (same request format, decided by tracker RNG/VRF after commitment), or nodes cheat only on non-audited requests — the exact property both PoSP and INTELLECT-2 rely on.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Feasibility table for the neuron-tai alpha
|
||||||
|
|
||||||
|
Alpha constraints: volunteer 8 GB consumer GPUs, single trusted tracker, mock-USDT but real carried-forward reputation, ~5% audit budget, small models today (0.5B-class) scaling up.
|
||||||
|
|
||||||
|
| Technique | Alpha feasibility | Overhead | What it buys | Verdict |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Optimistic accept + random teacher-forced audit (~5%) | **High** — tracker (or a trusted validator node) re-runs claimed tokens as one prefill | ~5% of serving compute; audits themselves up to ~100× cheaper than original generation | Detects wrong model, quantization, layer-skipping, garbage outputs; rational-cheating deterrence at L > 19× per-job gain | **Do now (anchor)** |
|
||||||
|
| Per-hop activation fingerprints (TOPLOC-style top-k, tolerance matching) | **High** — few hundred bytes per response per hop; nodes already stream activations | Negligible at inference; storage ~258 B/32 tokens/hop | Pins claims before audit; localizes blame to the exact hop on audit failure | **Do now** |
|
||||||
|
| Hop-boundary replay "bisection" (referee = tracker, non-interactive) | **Medium-high** — needs a reference node able to run the full model or per-hop ranges | Only on audit failure / dispute | Culprit identification in multi-hop pipelines; fair slashing | **Do now (simplified form)** |
|
||||||
|
| Seed-synchronized sampling (Token-DiFR-style exact-token audits) | **Medium** — requires controlling the sampler (Gumbel-max w/ shared seed) in node runtime | Zero at serve time | Strongest per-token evidence; >98% exact match for honest nodes | **Do if node runtime is ours; else logprob-rank tolerance** |
|
||||||
|
| Statistical logprob/perplexity monitoring (passive, on all traffic) | **High** — reference-free heuristics (output perplexity, repetition, truncation) + periodic logprob tests | Tiny | Cheap tripwire that raises a node's audit rate when outputs look off | **Do now (cheap complement)** |
|
||||||
|
| Reputation-weighted routing + reputation-weighted audit rate with floor | **High** — extends existing tracker reputation/strikes | None | Makes deterrence math work; concentrates audits on new/suspect nodes | **Do now** |
|
||||||
|
| Full interactive verification game (Truebit/Verde) with bitwise RepOps kernels | Low — requires replacing node operator stacks, decentralized referees | High engineering; runtime penalty for reproducible kernels | Trustless dispute resolution without a trusted tracker | **Roadmap (multi-tracker era)** |
|
||||||
|
| zkML proofs of inference | None — GPT-2-scale, tens of seconds per proof, server-class hardware ([zkGPT](https://www.usenix.org/conference/usenixsecurity25/presentation/qu-zkgpt); [NanoZK](https://arxiv.org/html/2603.18046v1)) | 100–10,000× | Cryptographic soundness | **Roadmap-only; re-evaluate yearly** |
|
||||||
|
| GPU TEE attestation | None on consumer cards (H100/Blackwell + SEV-SNP/TDX servers only — [NVIDIA guide](https://docs.nvidia.com/cc-deployment-guide-tdx-snp.pdf)) | ~2–10% where available | Hardware-rooted "right model+code ran" | **Roadmap: optional attested tier for datacenter contributors** |
|
||||||
|
| PoW-style hardware challenges (io.net-like) | Medium — trivial to add | Idle-time only | Sybil/VRAM/capacity attestation at registration; NOT output correctness | **Optional, registration-time only; don't confuse with fraud layer** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Recommended layered scheme for alpha
|
||||||
|
|
||||||
|
1. **Commit layer (on-demand, audit-selected):** when a request is selected for audit, each hop signs and reports a TOPLOC-style top-k fingerprint of its output boundary activations plus the sampling metadata (params, seed if controlled). Client-visible response carries the full claimed token sequence. Commitments are cheap and make retroactive lying impossible — they are *audit pins, not proofs* (§4).[^alpha-on-demand]
|
||||||
|
|
||||||
|
[^alpha-on-demand]: Alpha implements **on-demand** commitments per [ADR-0018 §3](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) — not every request. Nodes retain recent activations briefly; serving path stays uncommitted until audit selection.
|
||||||
|
2. **Audit layer (~5% of requests, VRF/tracker-RNG selected, indistinguishable ex ante):** a reference executor teacher-forces the claimed tokens in one prefill and checks (a) per-position token plausibility under the reference distribution (exact match if seed-synced; logprob-rank tolerance otherwise) and (b) hop-boundary fingerprints within TOPLOC-style exponent/mantissa tolerances. New/low-reputation nodes get 20–30% audit rates, veterans 2–3%, floor ≥ 2%, budget-balanced to ≈5% overall.
|
||||||
|
3. **Blame layer (on audit failure):** replay comparison across hop cut-points identifies the first divergent hop; that node eats the penalty (forfeit pending balance + strike, per the existing forfeiture design), and its audit rate snaps to maximum. Honest-noise false positives are handled by tolerance calibration against an honest-node baseline corpus (collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds).
|
||||||
|
4. **Reputation layer:** reputation derives exclusively from audit outcomes + uptime/latency, never peer ratings (no collusion surface). It weights routing (earnings) and inversely weights audit probability. Slow build, instant loss, decay on inactivity, entry probation for Sybil/whitewash resistance.
|
||||||
|
5. **Tripwire layer (all traffic, passive):** perplexity/repetition/truncation anomaly scoring on outputs; anomalies don't punish directly, they escalate the node's audit rate.
|
||||||
|
|
||||||
|
This is, deliberately, the Prime Intellect / Hyperbolic pattern adapted to multi-hop pipelines with the tracker as referee — every component has a deployed or peer-reviewed precedent.
|
||||||
|
|
||||||
|
## 9. Explicitly NOT feasible for alpha (roadmap-only)
|
||||||
|
|
||||||
|
- **zkML proofs of LLM inference** — 3+ orders of magnitude too slow for multi-billion-parameter serving; provers won't run on 8 GB cards. Revisit if layerwise/GKR provers (NanoZK, zkGPT lineage) reach real-time on 7B+ models.
|
||||||
|
- **GPU TEE / confidential-computing attestation on volunteer nodes** — hardware doesn't exist on consumer GPUs. Keep as an *attested datacenter tier* concept.
|
||||||
|
- **Fully trustless interactive verification games** (Truebit/Verde with on-chain referees and bitwise-reproducible kernels) — pointless while a single trusted tracker exists; becomes the correct escalation path when trackers decentralize. Requires shipping a reproducible-operator runtime to nodes.
|
||||||
|
- **Decentralized third-party verifier markets** (and their incentive pathologies: verifier's dilemma, forced-error jackpots, Truebit's 500–5000% verification tax) — single-tracker alpha sidesteps all of it; re-enters with multi-tracker.
|
||||||
|
- **Peer-rating reputation (EigenTrust-style transitive trust)** — collusion/Sybil surface with no benefit while the tracker is the sole verifier.
|
||||||
|
- **Commit-reveal timelock machinery for evaluator honesty** (Bittensor CRV3) — solves weight-copying among many validators; irrelevant until independent validators exist.
|
||||||
|
- **PoW puzzles as a correctness mechanism** — category error (§4); acceptable only as optional registration-time hardware attestation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
**Papers / peer-reviewed:**
|
||||||
|
- Truebit: [A scalable verification solution for blockchains](http://people.cs.uchicago.edu/~teutsch/papers/truebit.pdf) (Teutsch & Reitwießner)
|
||||||
|
- Gensyn Verde: [arXiv:2502.19405](https://arxiv.org/html/2502.19405v1); [Gensyn blog](https://blog.gensyn.ai/verde-a-verification-system-for-machine-learning-over-untrusted-nodes/); [Gensyn docs](https://docs.gensyn.ai/core-components)
|
||||||
|
- TOPLOC: [arXiv:2501.16007](https://arxiv.org/html/2501.16007v1) (ICML 2025); [GitHub](https://github.com/PrimeIntellect-ai/toploc)
|
||||||
|
- INTELLECT-2: [arXiv:2505.07291](https://arxiv.org/html/2505.07291)
|
||||||
|
- Proof of Sampling (Hyperbolic): [arXiv:2405.00295](https://arxiv.org/html/2405.00295); [spML blog](https://www.hyperbolic.ai/blog/spml-breakdown)
|
||||||
|
- DiFR (Token/Activation): [arXiv:2511.20621](https://arxiv.org/pdf/2511.20621); [explainer](https://technicallyprivate.substack.com/p/token-difr-llm-inference-verification)
|
||||||
|
- Logprob tracking: [arXiv:2512.03816](https://arxiv.org/html/2512.03816v1); Model-substitution auditing: [arXiv:2504.04715](https://arxiv.org/html/2504.04715v1)
|
||||||
|
- VeriLLM: [arXiv:2509.24257](https://arxiv.org/html/2509.24257v3); Optimistic TEE-Rollups: [arXiv:2512.20176](https://doi.org/10.48550/arxiv.2512.20176)
|
||||||
|
- zkGPT: [USENIX Security 2025](https://www.usenix.org/conference/usenixsecurity25/presentation/qu-zkgpt); NanoZK: [arXiv:2603.18046](https://arxiv.org/html/2603.18046v1); Modulus "Cost of Intelligence": [IACR 2026/1063](https://eprint.iacr.org/2026/1063)
|
||||||
|
- Nondeterminism: [Thinking Machines blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/); [arXiv:2506.09501](https://arxiv.org/html/2506.09501v2); [arXiv:2511.17826](https://arxiv.org/html/2511.17826v2); [arXiv:2511.00025](https://arxiv.org/html/2511.00025v1)
|
||||||
|
- Layer pruning: [arXiv:2403.17887](https://arxiv.org/html/2403.17887v1) (Gromov et al., ICLR 2025); [ShortGPT, arXiv:2403.03853](https://arxiv.org/pdf/2403.03853) (ACL 2025); LayerDrop: [arXiv:1909.11556](https://arxiv.org/abs/1909.11556)
|
||||||
|
- PoUW: [SoK, IACR 2025/1814](https://eprint.iacr.org/2025/1814.pdf); [arXiv:2209.03865](https://doi.org/10.48550/arxiv.2209.03865); [Wikipedia: Proof of work](https://en.wikipedia.org/wiki/Proof_of_work)
|
||||||
|
- EigenTrust: [WWW'03](https://nlp.stanford.edu/pubs/eigentrust.pdf); [EigenTrust++](https://doi.org/10.4108/icst.collaboratecom.2012.250420); [P2P reputation attack survey](https://cnitarot.github.io/papers/p2p-reputation-survey.pdf)
|
||||||
|
- Petals: [GitHub](http://github.com/bigscience-workshop/petals); [petals.dev](https://petals.dev/); [dev statements on incentives/correctness, HN](https://news.ycombinator.com/item?id=34215665)
|
||||||
|
|
||||||
|
**Official docs / engineering blogs:**
|
||||||
|
- NVIDIA CC: [H100 CC blog](https://developer.nvidia.com/blog/confidential-computing-on-h100-gpus-for-secure-and-trustworthy-ai/); [deployment guide (TDX/SNP)](https://docs.nvidia.com/cc-deployment-guide-tdx-snp.pdf); [CACM article](https://cacm.acm.org/practice/creating-the-first-confidential-gpus/); [Phala GPU-TEE on OpenRouter](https://phala.com/posts/GPU-TEEs-is-Alive-on-OpenRouter)
|
||||||
|
- Bittensor: [weight copying](https://docs.learnbittensor.org/concepts/weight-copying-in-bittensor); [commit reveal](https://docs.learnbittensor.org/concepts/commit-reveal); [OTF blog](https://blog.bittensor.com/weight-copying-in-bittensor-422585ab8fa5)
|
||||||
|
- io.net: [PoW docs](https://io.net/docs/guides/workers/proof-of-work); [Messari overview](https://messari.io/report/understanding-io-net-a-comprehensive-overview); [2026 risk analysis incl. Sybil attack history](https://cryptoaianalysis.com/io-net-io-analysis-2026/)
|
||||||
|
- Ritual: [ritual.net blog](https://ritual.net/blog/celestia); [ChainLight security review of AI-chain projects (Ritual, Akash)](https://blog.chainlight.io/ecosystem-explorer-exploring-security-risks-in-ai-blockchain-projects-2c490a726d13)
|
||||||
|
- Akash: [provider docs](https://akash.network/docs/providers/getting-started/should-i-run-a-provider/)
|
||||||
|
- Landscape survey: [Equilibrium Labs — State of Verifiable Inference](https://equilibrium.co/writing/state-of-verifiable-inference)
|
||||||
|
- Modulus Labs acquisition: [world.org announcement](https://world.org/blog/announcements/modulus-labs-joins-tfh-support-applied-research-world)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open-source reusability & build-vs-adopt
|
||||||
|
|
||||||
|
**Added 2026-07-04.** Focused follow-up: which of the projects above ship code we can actually reuse, and is verified decentralized inference a "solved problem"? All repo facts below checked against the GitHub repos/APIs on 2026-07-04.
|
||||||
|
|
||||||
|
### TOPLOC (Prime Intellect) — a real, adoptable library
|
||||||
|
|
||||||
|
- **Repo & license:** [PrimeIntellect-ai/toploc](https://github.com/PrimeIntellect-ai/toploc), **MIT** (both the repo license and the `pyproject.toml` declaration). Published on PyPI as `pip install toploc`. Separate [toploc-experiments](https://github.com/PrimeIntellect-ai/toploc-experiments) repo holds the paper's vLLM-integrated experiment code.
|
||||||
|
- **Maintenance:** actively maintained — created 2025-01-28, last push **2026-07-02** (two days before this writing), 56 stars / 12 forks / 12 open issues. Small single-purpose codebase (Python + a compiled extension; ships wheels, depends only on `torch`/`numpy`).
|
||||||
|
- **What it actually is:** a *library*, not a framework. Two function families ([README](https://github.com/PrimeIntellect-ai/toploc)):
|
||||||
|
- `build_proofs_bytes/base64(activations, decode_batching_size, topk, skip_prefill)` — prover side: takes a list of activation tensors (prefill tensor + one last-hidden-state tensor per generated token, bf16 in the examples) and returns compact proofs (~10 bytes each; 258 bytes per 32 tokens per the paper).
|
||||||
|
- `verify_proofs_bytes/base64(recomputed_activations, proofs, ...)` — verifier side: returns per-chunk `VerificationResult(exp_intersections, mant_err_mean, mant_err_median)`; **the accept/reject thresholds are ours to set and calibrate** (the paper gives reference thresholds validated across GPU types).
|
||||||
|
- **Assumptions:** model- and runtime-agnostic — it operates on tensors you extract yourself. It does *not* hook into vLLM/transformers for you; capturing the last hidden state per decode step in the node runtime, and recomputing activations via teacher-forced prefill on the validator, is our integration work. Detects precision changes by design, so **we must pin one canonical precision/quantization per served model** or honest quantized volunteers will fail verification.
|
||||||
|
- **Integration estimate for our pipeline-parallel case:** modest. (1) Node runtime: capture final-hidden-state per token, call `build_proofs_*`, attach to response — days, not weeks. (2) Tracker/validator: teacher-forced prefill re-run + `verify_proofs_*` — days. (3) The genuinely new part is *per-hop* commitments: TOPLOC as published commits only the **final** hidden state (whole-pipeline check, catches that fraud happened); applying the same encoding to hop-boundary activations for per-node blame is our own straightforward extension since the functions are tensor-generic, but its thresholds across heterogeneous 8 GB cards must be calibrated by us — that empirical honest-noise calibration is the real work. (4) Threshold calibration corpus across the volunteer fleet — the long pole, but required for any tolerance-based scheme regardless of library.
|
||||||
|
|
||||||
|
### Gensyn — papers and proprietary binaries, not a reusable verification library
|
||||||
|
|
||||||
|
What is actually public in [github.com/gensyn-ai](https://github.com/gensyn-ai) (org listing checked 2026-07-04):
|
||||||
|
|
||||||
|
- **[rl-swarm](https://github.com/gensyn-ai/rl-swarm)** — MIT, 1.7k stars: a framework for *RL training swarms*, plus [rl-swarm-contracts](https://github.com/gensyn-ai/rl-swarm-contracts) (coordination contracts). Not inference verification; not relevant to our fraud layer.
|
||||||
|
- **Verde (dispute resolution): paper only.** No `verde` repository exists in the org. The bisection/arbitration protocol from [arXiv:2502.19405](https://arxiv.org/html/2502.19405v1) has no published implementation. If we want it, we implement it from the paper.
|
||||||
|
- **RepOps / reproducible execution: shipping, but proprietary.** [repops-demo](https://github.com/gensyn-ai/repops-demo) is demo scripts with **no OSS license file** (only a `LICENSE-LLAMA` for the model weights) driving a prebuilt Docker image; supported targets include consumer RTX 3090/4070/4090 (CC 7.5–9, CUDA 12.6+). The successor [ree](https://github.com/gensyn-ai/ree) (Reproducible Execution Environment) is explicit in its README: the SDK wrapper is MIT, but **"the REE compiler binary and the REE reproducible-operators binary … are not licensed under the MIT License"** — they fall under a proprietary REE Binary License. REE does reproducible *LLM inference* with run/verify receipts, pipeline parallelism up to 72B (v0.2.0 notes), and needs Docker + NVIDIA driver ≥570. So bitwise cross-GPU determinism is real and demonstrably achievable on consumer GPUs — but as a closed-source appliance, not an embeddable library.
|
||||||
|
- **"Judge"** (verifiable AI evaluator, [docs](https://docs.gensyn.ai/core-components)) — no public library repo found in the org.
|
||||||
|
- **Assessment of the RepOps requirement:** deterministic cross-GPU execution means reimplementing/fixing the reduction order of every operator (matmul, attention, norms) for every hardware target — a compiler/kernel-engineering effort Gensyn keeps proprietary, and Thinking Machines' open batch-invariant kernels ([blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)) only address batch-invariance on one platform, not cross-GPU bitwise equality. **This is a large lift with no off-the-shelf OSS solution; it is exactly what TOPLOC-style tolerance matching lets us avoid.** Adopting REE would mean forcing volunteers into Gensyn's Docker appliance (proprietary binaries, driver floor, perf overhead) — a non-starter for our alpha; a possible future option for the *validator/referee* environment only.
|
||||||
|
|
||||||
|
### Bittensor — an incentive market, not verification code (confirmed)
|
||||||
|
|
||||||
|
- **Confirmed from primary sources:** the [whitepaper](https://bittensor.com/whitepaper) frames Bittensor as peers ranking peers ("intelligence measured by intelligence"), and the [Yuma Consensus docs](https://docs.learnbittensor.org/yuma-consensus) describe the on-chain mechanism precisely: each subnet validator submits a **weight vector ranking miners**; YC resolves the stake-weighted matrix into emissions, **clipping** any weight above the level supported by κ (default 0.5) of stake to punish out-of-consensus/collusive over-evaluation, and paying validators via EMA **bonds** that reward staying near consensus. There is **no cryptographic verification of any computation anywhere in the mechanism** — correctness of miner outputs is whatever each subnet's own validator code decides to score. The chain ([opentensor/subtensor](https://github.com/opentensor/subtensor), Rust, The Unlicense, actively developed) implements the consensus/emissions math, not inference checking.
|
||||||
|
- **What subnets actually do:** subnet owners write off-chain validator logic (Python, typically on the MIT-licensed [bittensor SDK](https://github.com/opentensor/bittensor)) that queries miners with tasks and scores responses — reference-model comparison, heuristics, or LLM-judging, entirely subnet-specific. Quality of validation is therefore uneven by construction.
|
||||||
|
- **Known weaknesses & mitigations (documented by Bittensor itself):** validator **weight-copying** (free-riding on the public weight matrix) — mitigated by Commit-Reveal v3 timelock-encrypted weights revealed epochs later ([weight copying](https://docs.learnbittensor.org/concepts/weight-copying-in-bittensor); [commit reveal](https://docs.learnbittensor.org/concepts/commit-reveal)); lazy/colluding validators — mitigated only economically via clipping and bond penalties; and the docs concede commit-reveal fails if miner rankings are too static.
|
||||||
|
- **Reusable for us:** the *design patterns* — stake-weighted score aggregation with clipping, EMA-smoothed trust, commit-reveal for future multi-validator honesty — not code. Subtensor's Rust consensus math is chain-embedded and solves a multi-validator problem we don't have while the single tracker is the only scorer.
|
||||||
|
|
||||||
|
### Other OSS in this niche
|
||||||
|
|
||||||
|
- **EZKL** ([zkonduit/ezkl](https://github.com/zkonduit/ezkl)) — **real, mature library** (Rust with Python/JS bindings, **Apache-2.0** per its [npm package](https://registry.npmjs.org/ezkl) and README, ~1.2k stars, last push 2026-02). It proves ONNX-graph inference in ZK and genuinely works — for *small* models (MLPs, small CNNs/regressors; §1.3 overheads apply). Irrelevant to serving multi-billion-parameter LLMs, but adoptable later if we ever need to prove a tiny model (e.g., a routing/scoring model) on-chain.
|
||||||
|
- **Petals** ([bigscience-workshop/petals](https://github.com/bigscience-workshop/petals)) — MIT, 10.3k stars, but **effectively dormant (last push Sep 2024)** and contains **zero trust machinery**: its own docs say malicious peers can alter outputs and recommend private swarms (§5). Its `hivemind` DHT/networking stack (MIT, also mirrored by Gensyn) is reusable *infrastructure*, not verification. Petals is validation of our architecture and a warning, not a component.
|
||||||
|
- **Hyperbolic PoSP/spML** — **paper + marketing only**. The [HyperbolicLabs GitHub org](https://github.com/HyperbolicLabs) contains agent kits, MCP servers, and a k8s OS — no PoSP/spML implementation is published. Adapt the pattern from [arXiv:2405.00295](https://arxiv.org/html/2405.00295); there is nothing to adopt.
|
||||||
|
- **Ritual Infernet** — the node ([ritual-net/infernet-node](https://github.com/ritual-net/infernet-node), Python, BSD-3-Clause-Clear per its community port; direct repo access was flaky at check time) plus [infernet-deploy](https://github.com/ritual-net/infernet-deploy)/[infernet-sdk](https://github.com/ritual-net/infernet-sdk) are open **coordination/oracle plumbing** (request routing, container orchestration, on-chain delivery). Proofs are a pluggable slot, not shipped verification logic, and ChainLight's review found no output aggregation, reputation, or penalties in the base system (§5). Nothing here advances our fraud layer.
|
||||||
|
- **DiFR** — the Token/Activation-DiFR authors state they released an **open-source vLLM integration** ([arXiv:2511.20621](https://arxiv.org/pdf/2511.20621)); worth tracking as a second adoptable audit primitive alongside TOPLOC, especially the seed-synchronized exact-token variant.
|
||||||
|
|
||||||
|
### Bottom line: is it solved?
|
||||||
|
|
||||||
|
**The audit primitive is solved and adoptable; the system around it is not.** No OSS project ships an end-to-end "verified decentralized inference network in a box" — every network either built theirs in-house (Prime Intellect), kept the hard part proprietary (Gensyn REE binaries), or doesn't verify at all (Petals, Akash, base Infernet, Bittensor's chain). What *is* genuinely reusable is small, high-quality, and MIT-licensed: TOPLOC. Everything else we need is a pattern to adapt or thin logic to build on our tracker, which is appropriately bespoke (it's our economics).
|
||||||
|
|
||||||
|
| Capability | Verdict | Component / reference | Rationale |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Detection / audit** (teacher-forced re-verification) | **ADOPT** | [`toploc`](https://github.com/PrimeIntellect-ai/toploc) (MIT, PyPI, maintained) — track [DiFR's vLLM integration](https://arxiv.org/pdf/2511.20621) as a complement | Proven encoding + verifier with cross-GPU tolerance semantics; our work is runtime hooks + threshold calibration, not algorithm development |
|
||||||
|
| **Blame attribution / bisection** (per-hop) | **ADAPT + BUILD** | Verde's commit-then-recompute pattern ([arXiv:2502.19405](https://arxiv.org/html/2502.19405v1)); reuse TOPLOC's encoding on hop-boundary activations | No published Verde code; full bisection is overkill for a handful of hops — a single referee replay over committed hop boundaries suffices (§1.2) |
|
||||||
|
| **Reputation / incentive** | **ADAPT + BUILD** | PoSP deterrence math ([arXiv:2405.00295](https://arxiv.org/html/2405.00295)); Yuma patterns (clipping, EMA trust, [commit-reveal](https://docs.learnbittensor.org/concepts/commit-reveal)) for the future multi-validator era; EigenTrust pitfalls list (§6) | No code to adopt (PoSP unpublished; Yuma is chain-embedded Rust for a different topology); our tracker-side logic is small and economics-specific |
|
||||||
|
| **Cryptographic proof** (zkML / TEE / bitwise-reproducible execution) | **NEITHER (roadmap)** | EZKL (Apache-2.0) if a tiny provable model ever needs it; Gensyn REE as a proprietary-appliance option for a future referee tier; GPU TEE per §1.4 | All three are unusable on 8 GB volunteer cards today (§1.3–1.4); REE's reproducible operators are closed-source binaries |
|
||||||
|
|
||||||
|
**Direct answer to the owner:** Gensyn and Bittensor are the right things to read but the wrong things to build on — Gensyn's verification core is a paper plus proprietary binaries aimed at training-grade trustlessness we don't need under a single trusted tracker, and Bittensor ships an incentive market that deliberately contains no computation verification at all. The one shelf-ready piece for our exact audit step is TOPLOC (MIT, pip-installable, updated this week), and the recommended alpha scheme in §8 remains: adopt TOPLOC for the audit primitive, adapt PoSP/Verde patterns for deterrence and blame, and build the thin tracker-side reputation/audit-rate logic ourselves.
|
||||||
@@ -66,6 +66,8 @@ services:
|
|||||||
BILLING_ARGS=""
|
BILLING_ARGS=""
|
||||||
if [ "$${ENABLE_BILLING_DB:-1}" = "1" ]; then
|
if [ "$${ENABLE_BILLING_DB:-1}" = "1" ]; then
|
||||||
BILLING_ARGS="--billing-db /var/lib/meshnet/billing.sqlite"
|
BILLING_ARGS="--billing-db /var/lib/meshnet/billing.sqlite"
|
||||||
|
else
|
||||||
|
BILLING_ARGS="--no-billing"
|
||||||
fi
|
fi
|
||||||
TREASURY_ARGS=""
|
TREASURY_ARGS=""
|
||||||
if [ -n "$${SOLANA_RPC_URL:-}" ] || [ -n "$${USDT_MINT:-}" ] || [ -n "$${MESHNET_TREASURY_KEYPAIR_B64:-}" ]; then
|
if [ -n "$${SOLANA_RPC_URL:-}" ] || [ -n "$${USDT_MINT:-}" ] || [ -n "$${MESHNET_TREASURY_KEYPAIR_B64:-}" ]; then
|
||||||
@@ -84,6 +86,7 @@ services:
|
|||||||
--self-url "$${PUBLIC_TRACKER_URL}" \
|
--self-url "$${PUBLIC_TRACKER_URL}" \
|
||||||
--relay-url "$${RELAY_URL}" \
|
--relay-url "$${RELAY_URL}" \
|
||||||
--stats-db /var/lib/meshnet/tracker-stats.sqlite \
|
--stats-db /var/lib/meshnet/tracker-stats.sqlite \
|
||||||
|
--accounts-db /var/lib/meshnet/accounts.sqlite \
|
||||||
$${BILLING_ARGS} \
|
$${BILLING_ARGS} \
|
||||||
$${TREASURY_ARGS} \
|
$${TREASURY_ARGS} \
|
||||||
$${PEER_ARGS}
|
$${PEER_ARGS}
|
||||||
|
|||||||
@@ -45,6 +45,8 @@ services:
|
|||||||
BILLING_ARGS=""
|
BILLING_ARGS=""
|
||||||
if [ "$${ENABLE_BILLING_DB:-1}" = "1" ]; then
|
if [ "$${ENABLE_BILLING_DB:-1}" = "1" ]; then
|
||||||
BILLING_ARGS="--billing-db /var/lib/meshnet/billing.sqlite"
|
BILLING_ARGS="--billing-db /var/lib/meshnet/billing.sqlite"
|
||||||
|
else
|
||||||
|
BILLING_ARGS="--no-billing"
|
||||||
fi
|
fi
|
||||||
TREASURY_ARGS=""
|
TREASURY_ARGS=""
|
||||||
if [ -n "$${SOLANA_RPC_URL:-}" ] || [ -n "$${USDT_MINT:-}" ] || [ -n "$${MESHNET_TREASURY_KEYPAIR_B64:-}" ]; then
|
if [ -n "$${SOLANA_RPC_URL:-}" ] || [ -n "$${USDT_MINT:-}" ] || [ -n "$${MESHNET_TREASURY_KEYPAIR_B64:-}" ]; then
|
||||||
@@ -63,6 +65,7 @@ services:
|
|||||||
--self-url "$${PUBLIC_TRACKER_URL}" \
|
--self-url "$${PUBLIC_TRACKER_URL}" \
|
||||||
--relay-url "$${RELAY_URL}" \
|
--relay-url "$${RELAY_URL}" \
|
||||||
--stats-db /var/lib/meshnet/tracker-stats.sqlite \
|
--stats-db /var/lib/meshnet/tracker-stats.sqlite \
|
||||||
|
--accounts-db /var/lib/meshnet/accounts.sqlite \
|
||||||
$${BILLING_ARGS} \
|
$${BILLING_ARGS} \
|
||||||
$${TREASURY_ARGS} \
|
$${TREASURY_ARGS} \
|
||||||
$${PEER_ARGS}
|
$${PEER_ARGS}
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||
# TAI token: revenue-backed rewards for nodes, USDT/SOL payments for clients
|
# TAI token: revenue-backed rewards for nodes, USDT/SOL payments for clients
|
||||||
|
|
||||||
|
## Status: Accepted (settlement mechanics superseded by ADR-0015)
|
||||||
|
|
||||||
|
> **Settlement update (2026-07-04):** Alpha and near-term production use **USDT-direct custodial settlement** with pending-balance forfeiture penalties (ADR-0015). TAI emission, backing-price buyback, and stake deposits described below remain the **roadmap tokenomics** — not the live payment path. When TAI ships, it will mint from accumulated protocol cut; this ADR's incentive structure still governs long-term design.
|
||||||
|
|
||||||
## Core principle
|
## Core principle
|
||||||
|
|
||||||
Nodes must get paid reliably for inference work. This is the primary growth engine. All tokenomics decisions serve this goal first. The model is intentionally simple and sound — parameters can be tweaked later, but the structure must have no fatal flaws from day one.
|
Nodes must get paid reliably for inference work. This is the primary growth engine. All tokenomics decisions serve this goal first. The model is intentionally simple and sound — parameters can be tweaked later, but the structure must have no fatal flaws from day one.
|
||||||
|
|||||||
@@ -16,10 +16,12 @@ The reference implementation (Petals) solves this with libp2p — GossipSub for
|
|||||||
|
|
||||||
### 1. TLS everywhere
|
### 1. TLS everywhere
|
||||||
|
|
||||||
All HTTP between nodes, tracker, and gateway uses HTTPS (TLS 1.3). Self-signed certificates are auto-generated on first node start and stored in `~/.config/meshnet/`. The certificate fingerprint is included in every heartbeat and gossip envelope. Nodes use TOFU (trust on first use) — they accept a peer's cert on first contact and pin the fingerprint; connections from the same peer with a different fingerprint are rejected.
|
**Intent:** All HTTP between nodes, tracker, and gateway uses HTTPS (TLS 1.3). Self-signed certificates are auto-generated on first node start and stored in `~/.config/meshnet/`. The certificate fingerprint is included in every heartbeat and gossip envelope. Nodes use TOFU (trust on first use) — they accept a peer's cert on first contact and pin the fingerprint; connections from the same peer with a different fingerprint are rejected.
|
||||||
|
|
||||||
The relay node uses a real CA-signed certificate (Let's Encrypt) because it is the internet-facing bootstrap point.
|
The relay node uses a real CA-signed certificate (Let's Encrypt) because it is the internet-facing bootstrap point.
|
||||||
|
|
||||||
|
> **Alpha reality correction (2026-07-04, ADR-0016):** As implemented, **TLS applies to the relay path** (`packages/relay`, WSS circuit proxy) and P2P packages that generate certs — **not** to plain HTTP on tracker ↔ node inference and registration in typical LAN/dev deployments. Tracker registration already requires `https://` URLs for production trackers, but node-to-tracker proxy traffic may still be plaintext on alpha. Full end-to-end TLS remains the target; alpha documents this as a **known limitation**, not a silent guarantee. Hive gossip auth (ADR-0017) is the immediate mitigation for tracker replication integrity.
|
||||||
|
|
||||||
### 2. mDNS for LAN peer discovery
|
### 2. mDNS for LAN peer discovery
|
||||||
|
|
||||||
Python `zeroconf` library. Service type: `_meshnet._tcp.local.`. A node announces itself on startup and browses for existing peers. This is zero-config discovery for home and lab networks. mDNS does not traverse routers, which is correct — LAN discovery should not bleed into the internet.
|
Python `zeroconf` library. Service type: `_meshnet._tcp.local.`. A node announces itself on startup and browses for existing peers. This is zero-config discovery for home and lab networks. mDNS does not traverse routers, which is correct — LAN discovery should not bleed into the internet.
|
||||||
|
|||||||
52
docs/adr/0016-alpha-scope-and-known-limitations.md
Normal file
52
docs/adr/0016-alpha-scope-and-known-limitations.md
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
# ADR-0016: Alpha scope and known limitations
|
||||||
|
|
||||||
|
## Status: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Pre-release audits found the routing layer is solid (ADR-0013) but the money and trust paths are not alpha-ready: unauthenticated gossip, free starting credit, node-reported accounting, ephemeral strike/ban state, and plaintext tracker HTTP undermine release. The owner locked an **alpha scope** that ships real inference and devnet settlement while explicitly deferring multi-tracker trustlessness and cryptographic verification.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### 1. Single settlement tracker
|
||||||
|
|
||||||
|
One operator-designated tracker holds the treasury keypair and runs the settlement loop (ADR-0015). Third-party trackers may join the hive for routing and state replication but **never** sign payouts. Raft elects a leader for shard-assignment commands only; settlement leadership is operator-configured, not democratic.
|
||||||
|
|
||||||
|
### 2. Open node join
|
||||||
|
|
||||||
|
Any wallet may register a node without invite, stake deposit, or hardware attestation. Anti-sybil cost comes from probation (first N jobs unpaid), carried-forward reputation, and fraud penalties — not from closed admission.
|
||||||
|
|
||||||
|
### 3. Devnet mock-USDT
|
||||||
|
|
||||||
|
Development and alpha target **Solana devnet** with a self-created mock-USDT SPL mint (ADR-0015). Mainnet USDT is a config change, not an architecture change. TAI settlement (ADR-0002) remains deferred; the 10% protocol cut accumulates as future TAI liquidity.
|
||||||
|
|
||||||
|
### 4. Reputation carries forward
|
||||||
|
|
||||||
|
Strike count, ban status, graduated reputation, and audit history **persist across tracker restarts** and are intended to survive alpha → mainnet cutover. A banned operator cannot evade penalties by restarting the tracker or re-registering under a fresh node id with the same wallet.
|
||||||
|
|
||||||
|
### 5. Known alpha limitations (explicit, not bugs)
|
||||||
|
|
||||||
|
| Limitation | Alpha posture | Post-alpha path |
|
||||||
|
|---|---|---|
|
||||||
|
| Tracker is trusted referee for audits and blame | Acceptable — see ADR-0018 | Decentralized verifier market; Verde-style on-chain games |
|
||||||
|
| Plaintext HTTP on tracker/node paths | Acceptable on LAN/dev; relay TLS only (ADR-0010 amended) | Full TLS + cert pinning per ADR-0010 intent |
|
||||||
|
| Unauthenticated gossip replication | **Blocker** — fix before alpha (ADR-0017) | HMAC/mTLS between hive members |
|
||||||
|
| zkML / GPU TEE verification | Roadmap-only | ADR-0018 §9; re-evaluate yearly |
|
||||||
|
| Multi-tracker money-path safety | Design now (ADR-0019), implement later | Consensus-gated settlement, idempotent payouts |
|
||||||
|
| Bitwise output equality across GPUs | Not required — logit/activation tolerance (TOPLOC) | Optional Gensyn RepOps tier for referee |
|
||||||
|
|
||||||
|
### 6. Fraud must be bounded
|
||||||
|
|
||||||
|
Because reputation carries forward, correctness fraud (bad outputs) and accounting fraud (inflated tokens/shard spans) must be **economically irrational** at the locked ~5% audit budget. Detection, blame, and penalty design are ADR-0018; implementation is Bucket 1 in `.scratch/alpha-hardening/`.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Alpha release criteria are Bucket 1 blockers in `.scratch/alpha-hardening/README.md`, not "all 35 user stories done."
|
||||||
|
- Documentation and runbooks must state trust assumptions plainly — custodial treasury, trusted tracker, devnet funds.
|
||||||
|
- Bucket 2 (multi-tracker) issues are tracked but not alpha-blocking.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- Supersedes the spirit of prototype-only trust in ADR-0003 for production alpha behavior — see ADR-0018.
|
||||||
|
- Settlement mechanics: ADR-0015.
|
||||||
|
- Research grounding: `.scratch/alpha-hardening/research-verifiable-inference.md`.
|
||||||
75
docs/adr/0017-tracker-authentication-and-authorization.md
Normal file
75
docs/adr/0017-tracker-authentication-and-authorization.md
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
# ADR-0017: Tracker authentication and authorization
|
||||||
|
|
||||||
|
## Status: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The tracker exposes three overlapping trust domains:
|
||||||
|
|
||||||
|
1. **Client API access** — Bearer API keys for inference and billing (partially gated today).
|
||||||
|
2. **Operator/admin access** — session-based accounts in `packages/tracker/meshnet_tracker/accounts.py` (registration, login, admin listing).
|
||||||
|
3. **Hive replication** — gossip endpoints that mutate billing, accounts, and stats without authentication today.
|
||||||
|
|
||||||
|
Financial and registry endpoints leak through inconsistent checks: some handlers require only a non-empty `Authorization` header (header-presence stub), while gossip accepts arbitrary peer events. Alpha requires a **single auth boundary** before any state mutation or privileged read.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### 1. Account subsystem (`accounts.py`)
|
||||||
|
|
||||||
|
- First registered account is **admin**; subsequent accounts are `user`.
|
||||||
|
- Login identifiers: email **or** wallet address + PBKDF2-SHA256 password.
|
||||||
|
- API keys (`sk-mesh-…`) are scoped to an account; revocation is event-sourced and gossip-replicated.
|
||||||
|
- **Sessions are local** (bearer tokens in memory, 7-day TTL) — not replicated across the hive. Each tracker validates its own sessions.
|
||||||
|
- Account mutations persist to SQLite and replicate via `/v1/accounts/gossip` events.
|
||||||
|
|
||||||
|
### 2. Client inference auth
|
||||||
|
|
||||||
|
- `POST /v1/chat/completions` requires a valid, non-revoked API key and positive ledger balance (402 if broke).
|
||||||
|
- **Starting credit is 0** for new API keys (ADR-0016 / issue C5). No implicit faucet; clients must deposit mock-USDT via bound wallet (US-032) or admin credit.
|
||||||
|
|
||||||
|
### 3. Gossip auth (alpha blocker)
|
||||||
|
|
||||||
|
All hive mutation endpoints require **authenticated peer identity**:
|
||||||
|
|
||||||
|
- `/v1/billing/gossip` — billing event replication
|
||||||
|
- `/v1/accounts/gossip` — account/key replication
|
||||||
|
- `/v1/stats/gossip` — rolling RPM merge
|
||||||
|
|
||||||
|
Acceptable alpha mechanisms (pick one in implementation):
|
||||||
|
|
||||||
|
- **Shared hive secret** (HMAC over request body + timestamp, configured on all trackers), or
|
||||||
|
- **Mutual TLS** between tracker peers (extends ADR-0010 relay TLS pattern).
|
||||||
|
|
||||||
|
Followers must reject events from unauthenticated callers. Read-only endpoints (`/dashboard`, `/v1/stats` GET) may remain public on LAN alpha deployments.
|
||||||
|
|
||||||
|
**Out of scope (alpha):** `POST /v1/gossip` — node throughput / peer fan-out gossip (`server.py` ~1331), distinct from hive mutation endpoints above. Document as **unauthenticated alpha limitation**; authenticate in a future ADR amendment when node identity binding exists (issue 01 tracks hive endpoints only).
|
||||||
|
|
||||||
|
### 4. Privileged operator endpoints
|
||||||
|
|
||||||
|
These require **real** auth — not header presence alone:
|
||||||
|
|
||||||
|
| Endpoint | Required role |
|
||||||
|
|---|---|
|
||||||
|
| `POST /v1/billing/forfeit` | Validator service identity or admin session |
|
||||||
|
| `POST /v1/benchmark/hop-penalty` | Admin session or service token |
|
||||||
|
| `GET /v1/benchmark/results` | Admin session or service token |
|
||||||
|
| `GET /v1/admin/accounts` | Admin session |
|
||||||
|
| `GET /v1/billing/summary`, `/v1/billing/settlements`, `/v1/registry/wallets` | Admin session (alpha: restrict financial reads) |
|
||||||
|
|
||||||
|
The unified auth middleware resolves: API key → account; session token → account + role; service token → validator; hive HMAC → peer.
|
||||||
|
|
||||||
|
### 5. Node registration
|
||||||
|
|
||||||
|
Node `POST /v1/register` is open (ADR-0016) but **banned wallets are rejected** at registration and excluded from routing. Wallet binding for client deposits (`POST /v1/wallet/register`) requires a valid API key and cryptographic ownership proof (issue C6).
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Gossip authentication (C1) and unified auth boundary (A2) are prerequisites for fraud penalties and billing fixes — an attacker can otherwise forge credits, strikes, or forfeit events.
|
||||||
|
- Session locality means admin login is per-tracker URL; document in runbooks.
|
||||||
|
- Multi-tracker auth evolution (client-signed commands) is deferred to ADR-0019.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- ADR-0016 (alpha scope)
|
||||||
|
- ADR-0015 (billing gate on inference)
|
||||||
|
- `.scratch/alpha-hardening/issues/01-c1-gossip-auth.md`, `02-a2-unified-auth-boundary.md`
|
||||||
92
docs/adr/0018-fraud-detection-verification-and-reputation.md
Normal file
92
docs/adr/0018-fraud-detection-verification-and-reputation.md
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
# ADR-0018: Fraud detection, verification, and reputation
|
||||||
|
|
||||||
|
## Status: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
ADR-0003 established optimistic sampling with stake slashing; ADR-0015 replaced stake with **pending-balance forfeiture** as collateral. Pre-alpha audits identified two distinct fraud types:
|
||||||
|
|
||||||
|
1. **Correctness fraud** — wrong model, layer-skipping, garbage outputs.
|
||||||
|
2. **Accounting fraud** — inflated token counts or shard-span work units reported by nodes.
|
||||||
|
|
||||||
|
The validator today compares final text only and always blames the last hop (`_final_text_node` in `packages/validator/meshnet_validator/__init__.py` ~137–140) — wrong for multi-hop pipelines.
|
||||||
|
|
||||||
|
Research (`.scratch/alpha-hardening/research-verifiable-inference.md`, 2026-07-04) grounds the alpha design in deployed patterns (Prime Intellect TOPLOC, Hyperbolic PoSP, Gensyn Verde blame **patterns**). This ADR is the flagship decision record for alpha hardening.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### 1. Anchor technique: optimistic accept + teacher-forced audit
|
||||||
|
|
||||||
|
- Default audit probability **p ≈ 5%** — a **budget target**, not a hard cap. Anomalies, low reputation, and disputes escalate rate; veterans floor at ≥ 2% (research §6, §8).
|
||||||
|
- **Deterrence condition:** at p = 0.05, penalty L must exceed **L > g·(1−p)/p ≈ 19×** per-job gain g (research §1.1). **Full pending forfeiture** is the primary penalty; three strikes → ban. The ×0.8-per-strike multiplier applies to **routing/payout weight** (reputation decay), not to the forfeiture amount.
|
||||||
|
- Single-tracker alpha: the tracker (or a designated reference node) is the auditor — no verifier market, no verifier's dilemma (research §1.1).
|
||||||
|
|
||||||
|
### 2. Detection primitive: ADOPT TOPLOC
|
||||||
|
|
||||||
|
- **`pip install toploc`** (MIT, [PrimeIntellect-ai/toploc](https://github.com/PrimeIntellect-ai/toploc)) for activation fingerprint commit + verify (research build-vs-adopt table).
|
||||||
|
- Teacher-forced prefill re-verification — compare in **logit/activation space with tolerances**, never free-running token equality (research §2).
|
||||||
|
- Pin **one canonical precision/quantization** per served model; TOPLOC detects precision drift by design.
|
||||||
|
- Per-hop boundary fingerprints extend TOPLOC's final-hidden-state encoding for **multi-hop blame** (research §1.2, §8 layer 1).
|
||||||
|
|
||||||
|
### 3. Commit layer: on-demand activation commitments
|
||||||
|
|
||||||
|
- Nodes commit compact TOPLOC-style fingerprints of **output boundary activations** per hop when selected for audit (on-demand, not every request — lower serving latency; brief retention window for recent activations).
|
||||||
|
- Commitments are **audit pins, not proofs** — correctness requires independent recomputation (research §4).
|
||||||
|
|
||||||
|
### 4. Blame layer: hop-boundary bisection (adapt Verde pattern)
|
||||||
|
|
||||||
|
On audit failure:
|
||||||
|
|
||||||
|
1. Referee (tracker) teacher-forces claimed token sequence through reference model.
|
||||||
|
2. Compare committed hop-boundary fingerprints to reference at each cut-point.
|
||||||
|
3. **First divergent hop** is the culprit — fixes `_final_text_node` last-hop-only bug.
|
||||||
|
4. Full interactive Truebit/Verde on-chain game and bitwise RepOps kernels: **roadmap-only** (research §1.2, §9).
|
||||||
|
|
||||||
|
### 5. Accounting fraud: tracker-authoritative metering
|
||||||
|
|
||||||
|
- **Token counts** come from the tracker's proxied stream/non-stream response parsing (`server.py` ~1890–1943), not node self-reports.
|
||||||
|
- **Work units** derive from **tracker-assigned shard span** at route construction (`server.py` ~1776–1782), not node-declared ranges at billing time.
|
||||||
|
- See issue H2.
|
||||||
|
|
||||||
|
### 6. Reputation model (graduated, persisted)
|
||||||
|
|
||||||
|
Reputation derives **only from tracker-verified audit outcomes** + uptime/latency — never peer ratings (research §6, collusion surface).
|
||||||
|
|
||||||
|
| Signal | Effect |
|
||||||
|
|---|---|
|
||||||
|
| Clean audits | Slow reputation build; higher routing weight |
|
||||||
|
| Strike | ×0.8 routing multiplier per strike (graduated decay) |
|
||||||
|
| Failed audit | Full pending forfeiture + strike; audit rate → maximum |
|
||||||
|
| Ban (3 strikes) | Registration rejected; excluded from routes; pending never paid |
|
||||||
|
| New/low reputation | Elevated audit rate (20–30% target for newcomers) |
|
||||||
|
| Inactivity | Reputation decay |
|
||||||
|
|
||||||
|
Persist strike/ban/reputation in SQLite alongside billing (issue A1/A5). Probation (first N jobs unpaid) retained as re-entry cost.
|
||||||
|
|
||||||
|
### 7. Passive tripwires
|
||||||
|
|
||||||
|
Perplexity/repetition/truncation heuristics on all traffic raise audit rate without direct punishment (research §8 layer 5).
|
||||||
|
|
||||||
|
### 8. Roadmap-only (explicitly NOT alpha)
|
||||||
|
|
||||||
|
- zkML proofs of LLM inference (research §1.3)
|
||||||
|
- GPU TEE attestation on consumer cards (research §1.4)
|
||||||
|
- Fully trustless Verde interactive games + RepOps bitwise kernels (research §9)
|
||||||
|
- Decentralized verifier markets
|
||||||
|
- Peer-rating reputation (EigenTrust)
|
||||||
|
- PoW as correctness proof — registration-time hardware attestation only, optional (research §4)
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Validator must be rewired: TOPLOC verify, hop blame, tracker-authoritative events — not string compare on final text alone.
|
||||||
|
- Threshold calibration requires an **honest-noise corpus** across the volunteer fleet before production thresholds (research §8).
|
||||||
|
- ADR-0003 remains historical; penalty mechanics follow ADR-0015 + this ADR.
|
||||||
|
- Implementation order: auth + persistence → accounting → TOPLOC → bisection → reputation routing (`.scratch/alpha-hardening/README.md`).
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` (§8 recommended scheme, §9 roadmap, build-vs-adopt table)
|
||||||
|
- ADR-0015 (forfeiture collateral)
|
||||||
|
- ADR-0016 (alpha scope)
|
||||||
|
- ADR-0017 (validator/forfeit auth)
|
||||||
|
- Issues: `06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`
|
||||||
66
docs/adr/0019-money-path-consistency-multi-tracker.md
Normal file
66
docs/adr/0019-money-path-consistency-multi-tracker.md
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
# ADR-0019: Money-path consistency (multi-tracker)
|
||||||
|
|
||||||
|
## Status: Accepted (design); implementation deferred post-alpha
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
ADR-0015 assumes a **single settlement tracker** with Raft-replicated ledger events. The hive already gossips billing, accounts, and stats between trackers. When multiple trackers participate, several money-path races appear:
|
||||||
|
|
||||||
|
- Settlement retries may double-pay if idempotency keys diverge from on-chain reality.
|
||||||
|
- Any tracker follower can apply `forfeit` / `charge` events — fine under one trusted operator, dangerous with adversarial peers.
|
||||||
|
- Raft term/vote state is in-memory — restart can duplicate leadership epochs.
|
||||||
|
- `forfeit` events are not commutative with `charge`/`payout` ordering under concurrent replication.
|
||||||
|
|
||||||
|
Alpha ships with **one settlement-capable tracker** (ADR-0016). This ADR records the accepted multi-tracker design so Bucket 2 issues do not block alpha while preventing silent architectural drift.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### 1. Settlement idempotency (C2)
|
||||||
|
|
||||||
|
- Every on-chain payout batch carries a **stable `settlement_id`** already debiting pending before broadcast (`server.py` ~3353–3356, `_send_settlement` ~3358).
|
||||||
|
- **Required hardening:** `solana_adapter.send_payouts` (~186–213) and the settlement loop must treat the Solana transaction signature as the idempotency anchor — retries reuse the same `settlement_id`; confirmed signatures must never spawn a second debit.
|
||||||
|
- Deposit watcher already uses `deposit-<sig>` event ids — extend the same pattern to payouts.
|
||||||
|
|
||||||
|
### 2. Consensus-gated money mutations (C3/C4)
|
||||||
|
|
||||||
|
Money-affecting events (`charge`, `payout`, `forfeit`, `credit`, `settlement`, `bind`) must commit through **Raft** (or successor consensus), not best-effort gossip alone:
|
||||||
|
|
||||||
|
- Only the Raft leader appends money commands to the replicated log.
|
||||||
|
- Followers apply committed entries in order; gossip becomes a transport, not the source of truth.
|
||||||
|
- Settlement signing remains **leader-only** and **treasury-key-bearing** (ADR-0015).
|
||||||
|
|
||||||
|
Shard-assignment commands already use Raft (`raft.py`); extend the log command set.
|
||||||
|
|
||||||
|
### 3. Durable Raft metadata (A3)
|
||||||
|
|
||||||
|
Persist `currentTerm`, `votedFor`, and log index to disk (`raft.py` ~26 — today term lives only in memory). Without this, restart can elect split leaders and duplicate settlement epochs.
|
||||||
|
|
||||||
|
### 4. Commutative forfeit semantics (H1)
|
||||||
|
|
||||||
|
Define explicit ordering rules when replicating `forfeit` vs `charge`/`payout`:
|
||||||
|
|
||||||
|
- Forfeit amount is snapshotted at event creation time (`billing.py` ~287).
|
||||||
|
- Apply forfeit **after** pending balance reflects all prior committed charges in the same epoch, or use signed pending snapshots in the event payload.
|
||||||
|
- `setdefault` on `_pending_since` (~324) and wallet bindings (~351) must not resurrect stale state on out-of-order apply — use last-writer-wins with `(term, index)` tie-break.
|
||||||
|
|
||||||
|
### 5. Alpha posture
|
||||||
|
|
||||||
|
| Item | Alpha | Post-alpha |
|
||||||
|
|---|---|---|
|
||||||
|
| Single treasury holder | Required | Optional multi-sig |
|
||||||
|
| Gossip-authenticated billing | Required (ADR-0017) | Superseded by Raft commit for money |
|
||||||
|
| On-chain idempotency | Best-effort resend | Hard guarantee (C2) |
|
||||||
|
| Consensus-gated settlement | Leader check only (~3331) | Full Raft gate (C3/C4) |
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Bucket 2 issues in `.scratch/alpha-hardening/` are **tracked but not alpha-blocking**.
|
||||||
|
- Implementing C3/C4 before alpha would delay release without security benefit while only one operator runs settlement.
|
||||||
|
- When enabling third-party settlement trackers, **all four** hardening items must ship together — partial deployment is worse than single-tracker alpha.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- ADR-0015 (settlement loop, pending collateral)
|
||||||
|
- ADR-0016 (single settlement tracker for alpha)
|
||||||
|
- ADR-0017 (gossip auth prerequisite)
|
||||||
|
- Issues: `12-c2-on-chain-idempotency.md` through `15-h1-commutative-forfeit.md`
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
Status: ready-for-agent
|
Status: superseded for alpha — see [ADR-0018](../adr/0018-fraud-detection-verification-and-reputation.md) and `.scratch/alpha-hardening/issues/06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`. On-chain stake slashing replaced by pending-balance forfeiture (ADR-0015).
|
||||||
|
|
||||||
# 07 — Fraud detection: validator + on-chain slash
|
# 07 — Fraud detection: validator + on-chain slash
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
Status: done
|
Status: done (core forfeiture wired); remaining fraud scope superseded — see [ADR-0018](../adr/0018-fraud-detection-verification-and-reputation.md) and `.scratch/alpha-hardening/issues/06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`.
|
||||||
|
|
||||||
# 34 — Hardened proof-of-work: pending-balance forfeiture penalty
|
# 34 — Hardened proof-of-work: pending-balance forfeiture penalty
|
||||||
|
|
||||||
|
|||||||
299
packages/tracker/meshnet_tracker/accounts.py
Normal file
299
packages/tracker/meshnet_tracker/accounts.py
Normal file
@@ -0,0 +1,299 @@
|
|||||||
|
"""Tracker user accounts: registration, login, API-key management.
|
||||||
|
|
||||||
|
Accounts are identified by email address or wallet address (either works as
|
||||||
|
the login identifier) and authenticated with a password (PBKDF2-SHA256).
|
||||||
|
The first account ever registered becomes the admin; everyone after is a
|
||||||
|
regular user.
|
||||||
|
|
||||||
|
Mutations are append-only events with unique ids — the same replication
|
||||||
|
model as ``BillingLedger`` — so accounts and API keys converge across the
|
||||||
|
tracker hive via gossip, and every dashboard can serve registration/login.
|
||||||
|
Sessions are deliberately local to each tracker (bearer tokens in memory).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import secrets
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
DEFAULT_ACCOUNTS_DB_PATH = "accounts.sqlite"
|
||||||
|
SESSION_TTL = 7 * 86400.0 # seconds
|
||||||
|
PBKDF2_ITERATIONS = 200_000
|
||||||
|
MIN_PASSWORD_LENGTH = 8
|
||||||
|
API_KEY_PREFIX = "sk-mesh-"
|
||||||
|
|
||||||
|
_EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
|
||||||
|
|
||||||
|
|
||||||
|
def _hash_password(password: str, salt: str) -> str:
|
||||||
|
return hashlib.pbkdf2_hmac(
|
||||||
|
"sha256", password.encode(), bytes.fromhex(salt), PBKDF2_ITERATIONS
|
||||||
|
).hex()
|
||||||
|
|
||||||
|
|
||||||
|
class AccountStore:
|
||||||
|
"""Thread-safe account/API-key store with SQLite persistence and event replication."""
|
||||||
|
|
||||||
|
def __init__(self, db_path: str | None = None) -> None:
|
||||||
|
self._db_path = db_path
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._accounts: dict[str, dict] = {} # account_id -> record
|
||||||
|
self._by_identifier: dict[str, str] = {} # lowercased email/wallet -> account_id
|
||||||
|
self._active_keys: dict[str, str] = {} # api_key -> account_id
|
||||||
|
self._revoked_keys: set[str] = set()
|
||||||
|
self._sessions: dict[str, dict] = {} # token -> {account_id, expires}
|
||||||
|
self._seen_event_ids: set[str] = set()
|
||||||
|
self._event_log: list[dict] = []
|
||||||
|
self._dirty = False
|
||||||
|
if self._db_path:
|
||||||
|
self._init_db()
|
||||||
|
self._load_from_db()
|
||||||
|
|
||||||
|
# ---- registration & login ----
|
||||||
|
|
||||||
|
def register(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
email: str | None = None,
|
||||||
|
wallet: str | None = None,
|
||||||
|
password: str,
|
||||||
|
) -> dict:
|
||||||
|
"""Create an account. The first account becomes the admin.
|
||||||
|
|
||||||
|
Raises ValueError with a user-facing message on invalid input.
|
||||||
|
"""
|
||||||
|
email = (email or "").strip().lower() or None
|
||||||
|
wallet = (wallet or "").strip() or None
|
||||||
|
if not email and not wallet:
|
||||||
|
raise ValueError("provide an email or a wallet address")
|
||||||
|
if email and not _EMAIL_RE.match(email):
|
||||||
|
raise ValueError("invalid email address")
|
||||||
|
if len(password or "") < MIN_PASSWORD_LENGTH:
|
||||||
|
raise ValueError(f"password must be at least {MIN_PASSWORD_LENGTH} characters")
|
||||||
|
with self._lock:
|
||||||
|
for identifier in filter(None, (email, wallet)):
|
||||||
|
if identifier.lower() in self._by_identifier:
|
||||||
|
raise ValueError(f"an account for {identifier!r} already exists")
|
||||||
|
salt = secrets.token_hex(16)
|
||||||
|
event = {
|
||||||
|
"id": f"register-{uuid.uuid4().hex}",
|
||||||
|
"type": "register",
|
||||||
|
"account_id": f"acct-{uuid.uuid4().hex}",
|
||||||
|
"email": email,
|
||||||
|
"wallet": wallet,
|
||||||
|
"role": "admin" if not self._accounts else "user",
|
||||||
|
"password_hash": _hash_password(password, salt),
|
||||||
|
"salt": salt,
|
||||||
|
"ts": time.time(),
|
||||||
|
}
|
||||||
|
self._apply_locked(event)
|
||||||
|
return self._public_view(self._accounts[event["account_id"]])
|
||||||
|
|
||||||
|
def verify_login(self, identifier: str, password: str) -> dict | None:
|
||||||
|
"""Return the public account view when credentials match, else None."""
|
||||||
|
with self._lock:
|
||||||
|
account_id = self._by_identifier.get((identifier or "").strip().lower())
|
||||||
|
if account_id is None:
|
||||||
|
return None
|
||||||
|
record = self._accounts[account_id]
|
||||||
|
if _hash_password(password or "", record["salt"]) != record["password_hash"]:
|
||||||
|
return None
|
||||||
|
return self._public_view(record)
|
||||||
|
|
||||||
|
# ---- sessions (local to this tracker) ----
|
||||||
|
|
||||||
|
def create_session(self, account_id: str) -> str:
|
||||||
|
token = secrets.token_urlsafe(32)
|
||||||
|
with self._lock:
|
||||||
|
self._sessions[token] = {
|
||||||
|
"account_id": account_id,
|
||||||
|
"expires": time.time() + SESSION_TTL,
|
||||||
|
}
|
||||||
|
return token
|
||||||
|
|
||||||
|
def session_account(self, token: str | None) -> dict | None:
|
||||||
|
if not token:
|
||||||
|
return None
|
||||||
|
with self._lock:
|
||||||
|
session = self._sessions.get(token)
|
||||||
|
if session is None:
|
||||||
|
return None
|
||||||
|
if session["expires"] < time.time():
|
||||||
|
del self._sessions[token]
|
||||||
|
return None
|
||||||
|
record = self._accounts.get(session["account_id"])
|
||||||
|
return self._public_view(record) if record else None
|
||||||
|
|
||||||
|
def destroy_session(self, token: str | None) -> None:
|
||||||
|
if not token:
|
||||||
|
return
|
||||||
|
with self._lock:
|
||||||
|
self._sessions.pop(token, None)
|
||||||
|
|
||||||
|
# ---- API keys ----
|
||||||
|
|
||||||
|
def create_api_key(self, account_id: str) -> str:
|
||||||
|
api_key = API_KEY_PREFIX + secrets.token_hex(24)
|
||||||
|
with self._lock:
|
||||||
|
if account_id not in self._accounts:
|
||||||
|
raise ValueError("unknown account")
|
||||||
|
self._apply_locked({
|
||||||
|
"id": f"createkey-{uuid.uuid4().hex}",
|
||||||
|
"type": "create_key",
|
||||||
|
"account_id": account_id,
|
||||||
|
"api_key": api_key,
|
||||||
|
"ts": time.time(),
|
||||||
|
})
|
||||||
|
return api_key
|
||||||
|
|
||||||
|
def revoke_api_key(self, account_id: str, api_key: str) -> bool:
|
||||||
|
"""Revoke a key owned by ``account_id``. Returns False if not owned."""
|
||||||
|
with self._lock:
|
||||||
|
if self._active_keys.get(api_key) != account_id:
|
||||||
|
return False
|
||||||
|
self._apply_locked({
|
||||||
|
"id": f"revokekey-{uuid.uuid4().hex}",
|
||||||
|
"type": "revoke_key",
|
||||||
|
"account_id": account_id,
|
||||||
|
"api_key": api_key,
|
||||||
|
"ts": time.time(),
|
||||||
|
})
|
||||||
|
return True
|
||||||
|
|
||||||
|
def keys_for(self, account_id: str) -> list[str]:
|
||||||
|
with self._lock:
|
||||||
|
return sorted(
|
||||||
|
key for key, owner in self._active_keys.items() if owner == account_id
|
||||||
|
)
|
||||||
|
|
||||||
|
def is_key_revoked(self, api_key: str) -> bool:
|
||||||
|
with self._lock:
|
||||||
|
return api_key in self._revoked_keys
|
||||||
|
|
||||||
|
# ---- views ----
|
||||||
|
|
||||||
|
def _public_view(self, record: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"account_id": record["account_id"],
|
||||||
|
"email": record.get("email"),
|
||||||
|
"wallet": record.get("wallet"),
|
||||||
|
"role": record["role"],
|
||||||
|
"created_ts": record.get("ts", 0.0),
|
||||||
|
}
|
||||||
|
|
||||||
|
def get_account(self, account_id: str) -> dict | None:
|
||||||
|
with self._lock:
|
||||||
|
record = self._accounts.get(account_id)
|
||||||
|
return self._public_view(record) if record else None
|
||||||
|
|
||||||
|
def list_accounts(self) -> list[dict]:
|
||||||
|
with self._lock:
|
||||||
|
views = []
|
||||||
|
for record in self._accounts.values():
|
||||||
|
view = self._public_view(record)
|
||||||
|
view["api_keys"] = sorted(
|
||||||
|
key for key, owner in self._active_keys.items()
|
||||||
|
if owner == record["account_id"]
|
||||||
|
)
|
||||||
|
views.append(view)
|
||||||
|
return sorted(views, key=lambda v: v["created_ts"])
|
||||||
|
|
||||||
|
def account_count(self) -> int:
|
||||||
|
with self._lock:
|
||||||
|
return len(self._accounts)
|
||||||
|
|
||||||
|
# ---- replication (same model as BillingLedger) ----
|
||||||
|
|
||||||
|
def events_since(self, index: int) -> tuple[list[dict], int]:
|
||||||
|
with self._lock:
|
||||||
|
return list(self._event_log[index:]), len(self._event_log)
|
||||||
|
|
||||||
|
def apply_events(self, events: list[dict]) -> int:
|
||||||
|
applied = 0
|
||||||
|
with self._lock:
|
||||||
|
for event in events:
|
||||||
|
event_id = event.get("id")
|
||||||
|
if not event_id or event_id in self._seen_event_ids:
|
||||||
|
continue
|
||||||
|
self._apply_locked(event)
|
||||||
|
applied += 1
|
||||||
|
return applied
|
||||||
|
|
||||||
|
def _apply_locked(self, event: dict) -> None:
|
||||||
|
etype = event.get("type")
|
||||||
|
if etype == "register":
|
||||||
|
account_id = event["account_id"]
|
||||||
|
record = {
|
||||||
|
"account_id": account_id,
|
||||||
|
"email": event.get("email"),
|
||||||
|
"wallet": event.get("wallet"),
|
||||||
|
"role": event.get("role", "user"),
|
||||||
|
"password_hash": event["password_hash"],
|
||||||
|
"salt": event["salt"],
|
||||||
|
"ts": float(event.get("ts", 0.0)),
|
||||||
|
}
|
||||||
|
self._accounts[account_id] = record
|
||||||
|
for identifier in filter(None, (record["email"], record["wallet"])):
|
||||||
|
self._by_identifier.setdefault(identifier.lower(), account_id)
|
||||||
|
elif etype == "create_key":
|
||||||
|
api_key = event["api_key"]
|
||||||
|
if api_key not in self._revoked_keys:
|
||||||
|
self._active_keys[api_key] = event["account_id"]
|
||||||
|
elif etype == "revoke_key":
|
||||||
|
api_key = event["api_key"]
|
||||||
|
self._active_keys.pop(api_key, None)
|
||||||
|
self._revoked_keys.add(api_key)
|
||||||
|
else:
|
||||||
|
return
|
||||||
|
self._seen_event_ids.add(event["id"])
|
||||||
|
self._event_log.append(event)
|
||||||
|
self._dirty = True
|
||||||
|
|
||||||
|
# ---- persistence ----
|
||||||
|
|
||||||
|
def _init_db(self) -> None:
|
||||||
|
con = sqlite3.connect(self._db_path) # type: ignore[arg-type]
|
||||||
|
con.execute(
|
||||||
|
"CREATE TABLE IF NOT EXISTS account_events "
|
||||||
|
"(event_id TEXT PRIMARY KEY, payload TEXT NOT NULL, ts REAL NOT NULL)"
|
||||||
|
)
|
||||||
|
con.commit()
|
||||||
|
con.close()
|
||||||
|
|
||||||
|
def _load_from_db(self) -> None:
|
||||||
|
con = sqlite3.connect(self._db_path) # type: ignore[arg-type]
|
||||||
|
rows = con.execute(
|
||||||
|
"SELECT payload FROM account_events ORDER BY ts, event_id"
|
||||||
|
).fetchall()
|
||||||
|
con.close()
|
||||||
|
with self._lock:
|
||||||
|
for (payload,) in rows:
|
||||||
|
try:
|
||||||
|
event = json.loads(payload)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
if event.get("id") not in self._seen_event_ids:
|
||||||
|
self._apply_locked(event)
|
||||||
|
self._dirty = False
|
||||||
|
|
||||||
|
def save_to_db(self) -> None:
|
||||||
|
if not self._db_path:
|
||||||
|
return
|
||||||
|
with self._lock:
|
||||||
|
if not self._dirty:
|
||||||
|
return
|
||||||
|
events = list(self._event_log)
|
||||||
|
self._dirty = False
|
||||||
|
con = sqlite3.connect(self._db_path) # type: ignore[arg-type]
|
||||||
|
con.executemany(
|
||||||
|
"INSERT OR IGNORE INTO account_events (event_id, payload, ts) VALUES (?, ?, ?)",
|
||||||
|
[(e["id"], json.dumps(e), float(e.get("ts", 0.0))) for e in events],
|
||||||
|
)
|
||||||
|
con.commit()
|
||||||
|
con.close()
|
||||||
@@ -20,6 +20,7 @@ import uuid
|
|||||||
|
|
||||||
DEFAULT_PRICE_PER_1K_TOKENS = 0.02 # USDT
|
DEFAULT_PRICE_PER_1K_TOKENS = 0.02 # USDT
|
||||||
DEFAULT_STARTING_CREDIT = 1.0 # USDT of Caller Credit for a new API key
|
DEFAULT_STARTING_CREDIT = 1.0 # USDT of Caller Credit for a new API key
|
||||||
|
DEFAULT_BILLING_DB_PATH = "billing.sqlite"
|
||||||
NODE_REVENUE_SHARE = 0.90 # nodes 90% / protocol cut 10% (ADR-0015)
|
NODE_REVENUE_SHARE = 0.90 # nodes 90% / protocol cut 10% (ADR-0015)
|
||||||
|
|
||||||
|
|
||||||
@@ -364,6 +365,34 @@ class BillingLedger:
|
|||||||
with self._lock:
|
with self._lock:
|
||||||
return self._node_pending.get(wallet, 0.0)
|
return self._node_pending.get(wallet, 0.0)
|
||||||
|
|
||||||
|
def usage_for(self, api_keys: list[str], *, recent_limit: int = 20) -> dict:
|
||||||
|
"""Aggregate charge history for a set of API keys (dashboard view)."""
|
||||||
|
keys = set(api_keys)
|
||||||
|
requests = 0
|
||||||
|
total_tokens = 0
|
||||||
|
total_cost = 0.0
|
||||||
|
recent: list[dict] = []
|
||||||
|
with self._lock:
|
||||||
|
for event in self._event_log:
|
||||||
|
if event.get("type") != "charge" or event.get("api_key") not in keys:
|
||||||
|
continue
|
||||||
|
requests += 1
|
||||||
|
total_tokens += int(event.get("total_tokens", 0))
|
||||||
|
total_cost += float(event.get("cost", 0.0))
|
||||||
|
recent.append({
|
||||||
|
"api_key": event["api_key"],
|
||||||
|
"model": event.get("model"),
|
||||||
|
"total_tokens": event.get("total_tokens", 0),
|
||||||
|
"cost": event.get("cost", 0.0),
|
||||||
|
"ts": event.get("ts", 0.0),
|
||||||
|
})
|
||||||
|
return {
|
||||||
|
"requests": requests,
|
||||||
|
"total_tokens": total_tokens,
|
||||||
|
"total_cost": total_cost,
|
||||||
|
"recent": recent[-recent_limit:],
|
||||||
|
}
|
||||||
|
|
||||||
def snapshot(self) -> dict:
|
def snapshot(self) -> dict:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import argparse
|
|||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
|
|
||||||
|
from .accounts import DEFAULT_ACCOUNTS_DB_PATH
|
||||||
|
from .billing import DEFAULT_BILLING_DB_PATH
|
||||||
from .server import TrackerServer, derive_relay_url_from_public_tracker_url
|
from .server import TrackerServer, derive_relay_url_from_public_tracker_url
|
||||||
|
|
||||||
|
|
||||||
@@ -40,9 +42,31 @@ def main() -> None:
|
|||||||
)
|
)
|
||||||
common.add_argument(
|
common.add_argument(
|
||||||
"--billing-db",
|
"--billing-db",
|
||||||
default=None,
|
default=DEFAULT_BILLING_DB_PATH,
|
||||||
metavar="PATH",
|
metavar="PATH",
|
||||||
help="SQLite database path for the USDT billing ledger (enables billing; ADR-0015)",
|
help=(
|
||||||
|
"SQLite database path for the USDT billing ledger "
|
||||||
|
f"(default: {DEFAULT_BILLING_DB_PATH}; ADR-0015)"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
common.add_argument(
|
||||||
|
"--no-billing",
|
||||||
|
action="store_true",
|
||||||
|
help="Disable the USDT billing ledger",
|
||||||
|
)
|
||||||
|
common.add_argument(
|
||||||
|
"--accounts-db",
|
||||||
|
default=DEFAULT_ACCOUNTS_DB_PATH,
|
||||||
|
metavar="PATH",
|
||||||
|
help=(
|
||||||
|
"SQLite database path for dashboard user accounts "
|
||||||
|
f"(default: {DEFAULT_ACCOUNTS_DB_PATH})"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
common.add_argument(
|
||||||
|
"--no-accounts",
|
||||||
|
action="store_true",
|
||||||
|
help="Disable dashboard user accounts (registration/login)",
|
||||||
)
|
)
|
||||||
common.add_argument(
|
common.add_argument(
|
||||||
"--solana-rpc-url",
|
"--solana-rpc-url",
|
||||||
@@ -108,7 +132,9 @@ def main() -> None:
|
|||||||
cluster_self_url=args.self_url,
|
cluster_self_url=args.self_url,
|
||||||
stats_db=getattr(args, "stats_db", None),
|
stats_db=getattr(args, "stats_db", None),
|
||||||
relay_url=relay_url,
|
relay_url=relay_url,
|
||||||
billing_db=getattr(args, "billing_db", None),
|
enable_billing=not args.no_billing,
|
||||||
|
billing_db=None if args.no_billing else args.billing_db,
|
||||||
|
accounts_db=None if args.no_accounts else args.accounts_db,
|
||||||
treasury=treasury,
|
treasury=treasury,
|
||||||
settle_period=args.settle_period,
|
settle_period=args.settle_period,
|
||||||
payout_threshold=args.payout_threshold,
|
payout_threshold=args.payout_threshold,
|
||||||
|
|||||||
@@ -30,6 +30,20 @@
|
|||||||
.empty { color:var(--dim); font-style:italic; }
|
.empty { color:var(--dim); font-style:italic; }
|
||||||
.pill { display:inline-block; padding:0 7px; border-radius:9px;
|
.pill { display:inline-block; padding:0 7px; border-radius:9px;
|
||||||
border:1px solid var(--border); font-size:11px; }
|
border:1px solid var(--border); font-size:11px; }
|
||||||
|
input, button { font:inherit; color:var(--fg); background:var(--bg);
|
||||||
|
border:1px solid var(--border); border-radius:6px; padding:5px 8px; }
|
||||||
|
input { width:100%; margin-bottom:6px; }
|
||||||
|
button { cursor:pointer; color:var(--accent); }
|
||||||
|
button:hover { border-color:var(--accent); }
|
||||||
|
button.small { font-size:11px; padding:1px 7px; }
|
||||||
|
.form-row { display:flex; gap:8px; }
|
||||||
|
.form-row button { white-space:nowrap; }
|
||||||
|
.error-msg { color:var(--bad); font-size:12px; min-height:16px; }
|
||||||
|
.keybox { word-break:break-all; background:var(--bg); border:1px solid var(--border);
|
||||||
|
border-radius:6px; padding:4px 8px; margin:4px 0; font-size:11px; }
|
||||||
|
.tabs { display:flex; gap:10px; margin-bottom:8px; }
|
||||||
|
.tabs a { color:var(--dim); cursor:pointer; }
|
||||||
|
.tabs a.active { color:var(--accent); border-bottom:1px solid var(--accent); }
|
||||||
</style>
|
</style>
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
@@ -39,6 +53,8 @@
|
|||||||
<span class="meta" id="refreshed"></span>
|
<span class="meta" id="refreshed"></span>
|
||||||
</header>
|
</header>
|
||||||
<main>
|
<main>
|
||||||
|
<section id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
|
||||||
|
<section id="admin-section" style="display:none"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
|
||||||
<section><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
|
<section><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
|
||||||
<section><h2>Nodes & coverage</h2><div id="nodes" class="empty">loading…</div></section>
|
<section><h2>Nodes & coverage</h2><div id="nodes" class="empty">loading…</div></section>
|
||||||
<section><h2>Client balances</h2><div id="clients" class="empty">loading…</div></section>
|
<section><h2>Client balances</h2><div id="clients" class="empty">loading…</div></section>
|
||||||
@@ -193,6 +209,149 @@ function renderThroughput(stats) {
|
|||||||
$("throughput").innerHTML = table(["node", "model", "tps (1h)", "samples"], rows);
|
$("throughput").innerHTML = table(["node", "model", "tps (1h)", "samples"], rows);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- account panel (registration / login / balance / usage / API keys) ----
|
||||||
|
|
||||||
|
let sessionToken = localStorage.getItem("meshnet_session") || null;
|
||||||
|
let authTab = "login";
|
||||||
|
|
||||||
|
async function apiCall(path, method, body) {
|
||||||
|
const headers = { "Content-Type": "application/json" };
|
||||||
|
if (sessionToken) headers["Authorization"] = "Bearer " + sessionToken;
|
||||||
|
try {
|
||||||
|
const r = await fetch(path, {
|
||||||
|
method: method || "GET",
|
||||||
|
headers,
|
||||||
|
body: body ? JSON.stringify(body) : undefined,
|
||||||
|
});
|
||||||
|
const data = await r.json().catch(() => ({}));
|
||||||
|
return { ok: r.ok, status: r.status, data };
|
||||||
|
} catch {
|
||||||
|
return { ok: false, status: 0, data: {} };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function setSession(token) {
|
||||||
|
sessionToken = token;
|
||||||
|
if (token) localStorage.setItem("meshnet_session", token);
|
||||||
|
else localStorage.removeItem("meshnet_session");
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderAuthForms(errorMsg) {
|
||||||
|
const tab = (name, label) =>
|
||||||
|
`<a class="${authTab === name ? "active" : ""}" onclick="switchAuthTab('${name}')">${label}</a>`;
|
||||||
|
const identityFields =
|
||||||
|
'<input id="auth-email" type="email" placeholder="email (or leave blank)">' +
|
||||||
|
'<input id="auth-wallet" type="text" placeholder="wallet address (or leave blank)">';
|
||||||
|
const form = authTab === "login"
|
||||||
|
? '<input id="auth-identifier" type="text" placeholder="email or wallet address">' +
|
||||||
|
'<input id="auth-password" type="password" placeholder="password">' +
|
||||||
|
'<div class="form-row"><button onclick="doLogin()">Log in</button></div>'
|
||||||
|
: identityFields +
|
||||||
|
'<input id="auth-password" type="password" placeholder="password (min 8 chars)">' +
|
||||||
|
'<div class="form-row"><button onclick="doRegister()">Create account</button></div>';
|
||||||
|
$("account").innerHTML =
|
||||||
|
`<div class="tabs">${tab("login", "Log in")}${tab("register", "Register")}</div>` +
|
||||||
|
form + `<div class="error-msg">${errorMsg ? esc(errorMsg) : ""}</div>`;
|
||||||
|
$("admin-section").style.display = "none";
|
||||||
|
}
|
||||||
|
|
||||||
|
function switchAuthTab(name) { authTab = name; renderAuthForms(); }
|
||||||
|
|
||||||
|
async function doRegister() {
|
||||||
|
const email = $("auth-email").value.trim();
|
||||||
|
const wallet = $("auth-wallet").value.trim();
|
||||||
|
const password = $("auth-password").value;
|
||||||
|
const r = await apiCall("/v1/auth/register", "POST",
|
||||||
|
{ email: email || null, wallet: wallet || null, password });
|
||||||
|
if (!r.ok) { renderAuthForms(r.data.error || "registration failed"); return; }
|
||||||
|
setSession(r.data.session_token);
|
||||||
|
await renderAccountPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function doLogin() {
|
||||||
|
const identifier = $("auth-identifier").value.trim();
|
||||||
|
const password = $("auth-password").value;
|
||||||
|
const r = await apiCall("/v1/auth/login", "POST", { identifier, password });
|
||||||
|
if (!r.ok) { renderAuthForms(r.data.error || "login failed"); return; }
|
||||||
|
setSession(r.data.session_token);
|
||||||
|
await renderAccountPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function doLogout() {
|
||||||
|
await apiCall("/v1/auth/logout", "POST", {});
|
||||||
|
setSession(null);
|
||||||
|
renderAuthForms();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function createKey() {
|
||||||
|
const r = await apiCall("/v1/account/keys", "POST", {});
|
||||||
|
if (r.ok) await renderAccountPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function revokeKey(key) {
|
||||||
|
if (!confirm("Revoke this API key? Clients using it will stop working.")) return;
|
||||||
|
await apiCall("/v1/account/keys/revoke", "POST", { api_key: key });
|
||||||
|
await renderAccountPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function renderAccountPanel() {
|
||||||
|
const r = await apiCall("/v1/account");
|
||||||
|
if (r.status === 404) { // accounts disabled on this tracker
|
||||||
|
$("account-section").style.display = "none";
|
||||||
|
$("admin-section").style.display = "none";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!r.ok) { setSession(null); renderAuthForms(); return; }
|
||||||
|
const { account, api_keys, balances, total_balance, usage } = r.data;
|
||||||
|
const who = account.email || account.wallet || account.account_id;
|
||||||
|
let html =
|
||||||
|
`<div><b>${esc(who)}</b> <span class="pill">${esc(account.role)}</span> ` +
|
||||||
|
`<button class="small" style="float:right" onclick="doLogout()">log out</button></div>` +
|
||||||
|
`<div style="margin:6px 0">balance: <b class="${total_balance > 0 ? "ok" : "bad"}">` +
|
||||||
|
`${usdt(total_balance)}</b> USDT · requests: <b>${usage.requests}</b> · ` +
|
||||||
|
`tokens: <b>${usage.total_tokens}</b> · spent: <b>${usdt(usage.total_cost)}</b> USDT</div>`;
|
||||||
|
html += '<div style="margin:6px 0"><b class="dim">API keys</b> ' +
|
||||||
|
'<button class="small" onclick="createKey()">+ new key</button></div>';
|
||||||
|
if (api_keys.length) {
|
||||||
|
for (const key of api_keys) {
|
||||||
|
html += `<div class="keybox">${esc(key)}` +
|
||||||
|
` <span class="dim">(${usdt(balances[key] ?? 0)} USDT)</span>` +
|
||||||
|
` <button class="small" onclick="revokeKey('${esc(key)}')">revoke</button></div>`;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
html += '<div class="empty">no active keys</div>';
|
||||||
|
}
|
||||||
|
if (usage.recent && usage.recent.length) {
|
||||||
|
html += '<div style="margin-top:6px"><b class="dim">recent usage</b></div>' +
|
||||||
|
table(["time", "model", "tokens", "cost"], usage.recent.slice().reverse().map(u => [
|
||||||
|
new Date(u.ts * 1000).toLocaleTimeString(),
|
||||||
|
esc(short(u.model || "?", 24)),
|
||||||
|
`<span class="num">${esc(String(u.total_tokens))}</span>`,
|
||||||
|
`<span class="num">${usdt(u.cost)}</span>`,
|
||||||
|
]));
|
||||||
|
}
|
||||||
|
$("account").innerHTML = html;
|
||||||
|
if (account.role === "admin") await renderAdminPanel();
|
||||||
|
else $("admin-section").style.display = "none";
|
||||||
|
}
|
||||||
|
|
||||||
|
async function renderAdminPanel() {
|
||||||
|
const r = await apiCall("/v1/admin/accounts");
|
||||||
|
if (!r.ok) { $("admin-section").style.display = "none"; return; }
|
||||||
|
$("admin-section").style.display = "";
|
||||||
|
const rows = (r.data.accounts || []).map(a => {
|
||||||
|
const balance = Object.values(a.balances || {}).reduce((x, y) => x + y, 0);
|
||||||
|
return [
|
||||||
|
esc(short(a.email || a.wallet || a.account_id, 24)),
|
||||||
|
`<span class="pill">${esc(a.role)}</span>`,
|
||||||
|
String((a.api_keys || []).length),
|
||||||
|
`<span class="num ${balance > 0 ? "ok" : "bad"}">${usdt(balance)}</span>`,
|
||||||
|
new Date(a.created_ts * 1000).toLocaleDateString(),
|
||||||
|
];
|
||||||
|
});
|
||||||
|
$("admin").innerHTML = table(["account", "role", "keys", "balance (USDT)", "created"], rows);
|
||||||
|
}
|
||||||
|
|
||||||
async function refresh() {
|
async function refresh() {
|
||||||
$("self-url").textContent = location.host;
|
$("self-url").textContent = location.host;
|
||||||
const [raft, map, summary, settlements, wallets, stats] = await Promise.all([
|
const [raft, map, summary, settlements, wallets, stats] = await Promise.all([
|
||||||
@@ -213,7 +372,9 @@ async function refresh() {
|
|||||||
$("refreshed").textContent = "refreshed " + new Date().toLocaleTimeString();
|
$("refreshed").textContent = "refreshed " + new Date().toLocaleTimeString();
|
||||||
}
|
}
|
||||||
refresh();
|
refresh();
|
||||||
|
renderAccountPanel();
|
||||||
setInterval(refresh, 4000);
|
setInterval(refresh, 4000);
|
||||||
|
setInterval(() => { if (sessionToken) renderAccountPanel(); }, 8000);
|
||||||
</script>
|
</script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
@@ -36,7 +36,8 @@ import uuid
|
|||||||
from importlib.resources import files
|
from importlib.resources import files
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from .billing import BillingLedger
|
from .accounts import DEFAULT_ACCOUNTS_DB_PATH, AccountStore
|
||||||
|
from .billing import DEFAULT_BILLING_DB_PATH, BillingLedger
|
||||||
from .gossip import NodeGossip
|
from .gossip import NodeGossip
|
||||||
from .raft import RaftNode
|
from .raft import RaftNode
|
||||||
|
|
||||||
@@ -1261,6 +1262,7 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
|||||||
gossip: "NodeGossip | None" = None,
|
gossip: "NodeGossip | None" = None,
|
||||||
stats: "_StatsCollector | None" = None,
|
stats: "_StatsCollector | None" = None,
|
||||||
billing: "BillingLedger | None" = None,
|
billing: "BillingLedger | None" = None,
|
||||||
|
accounts: "AccountStore | None" = None,
|
||||||
benchmark_results_path: str | None = None,
|
benchmark_results_path: str | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
super().__init__(addr, handler)
|
super().__init__(addr, handler)
|
||||||
@@ -1275,6 +1277,7 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
|||||||
self.gossip = gossip
|
self.gossip = gossip
|
||||||
self.stats: _StatsCollector | None = stats
|
self.stats: _StatsCollector | None = stats
|
||||||
self.billing: BillingLedger | None = billing
|
self.billing: BillingLedger | None = billing
|
||||||
|
self.accounts: AccountStore | None = accounts
|
||||||
self.benchmark_results_path = benchmark_results_path or os.path.join(
|
self.benchmark_results_path = benchmark_results_path or os.path.join(
|
||||||
os.getcwd(), "benchmark_results.json"
|
os.getcwd(), "benchmark_results.json"
|
||||||
)
|
)
|
||||||
@@ -1337,6 +1340,24 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
if self.path == "/v1/billing/forfeit":
|
if self.path == "/v1/billing/forfeit":
|
||||||
self._handle_billing_forfeit()
|
self._handle_billing_forfeit()
|
||||||
return
|
return
|
||||||
|
if self.path == "/v1/auth/register":
|
||||||
|
self._handle_auth_register()
|
||||||
|
return
|
||||||
|
if self.path == "/v1/auth/login":
|
||||||
|
self._handle_auth_login()
|
||||||
|
return
|
||||||
|
if self.path == "/v1/auth/logout":
|
||||||
|
self._handle_auth_logout()
|
||||||
|
return
|
||||||
|
if self.path == "/v1/account/keys":
|
||||||
|
self._handle_account_key_create()
|
||||||
|
return
|
||||||
|
if self.path == "/v1/account/keys/revoke":
|
||||||
|
self._handle_account_key_revoke()
|
||||||
|
return
|
||||||
|
if self.path == "/v1/accounts/gossip":
|
||||||
|
self._handle_accounts_gossip()
|
||||||
|
return
|
||||||
if self.path == "/v1/benchmark/hop-penalty":
|
if self.path == "/v1/benchmark/hop-penalty":
|
||||||
self._handle_benchmark_hop_penalty()
|
self._handle_benchmark_hop_penalty()
|
||||||
return
|
return
|
||||||
@@ -1382,6 +1403,10 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
self._handle_billing_summary()
|
self._handle_billing_summary()
|
||||||
elif parsed.path == "/v1/billing/settlements":
|
elif parsed.path == "/v1/billing/settlements":
|
||||||
self._handle_billing_settlements()
|
self._handle_billing_settlements()
|
||||||
|
elif parsed.path == "/v1/account":
|
||||||
|
self._handle_account_me()
|
||||||
|
elif parsed.path == "/v1/admin/accounts":
|
||||||
|
self._handle_admin_accounts()
|
||||||
elif parsed.path == "/v1/benchmark/results":
|
elif parsed.path == "/v1/benchmark/results":
|
||||||
self._handle_benchmark_results()
|
self._handle_benchmark_results()
|
||||||
elif parsed.path == "/v1/registry/wallets":
|
elif parsed.path == "/v1/registry/wallets":
|
||||||
@@ -1649,6 +1674,13 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
"code": "missing_api_key",
|
"code": "missing_api_key",
|
||||||
}})
|
}})
|
||||||
return
|
return
|
||||||
|
if server.accounts is not None and server.accounts.is_key_revoked(api_key):
|
||||||
|
self._send_json(401, {"error": {
|
||||||
|
"message": "API key has been revoked",
|
||||||
|
"type": "invalid_request_error",
|
||||||
|
"code": "invalid_api_key",
|
||||||
|
}})
|
||||||
|
return
|
||||||
if not server.billing.has_funds(api_key):
|
if not server.billing.has_funds(api_key):
|
||||||
self._send_json(402, {"error": {
|
self._send_json(402, {"error": {
|
||||||
"message": "insufficient balance: deposit USDT to continue",
|
"message": "insufficient balance: deposit USDT to continue",
|
||||||
@@ -2431,6 +2463,165 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
)
|
)
|
||||||
self._send_json(200, {"forfeited": event["amount"], **strike_state})
|
self._send_json(200, {"forfeited": event["amount"], **strike_state})
|
||||||
|
|
||||||
|
# ---- user accounts (registration, login, API keys) ----
|
||||||
|
|
||||||
|
def _session_account(self) -> dict | None:
|
||||||
|
"""Resolve the session token in the Authorization header, or None."""
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if server.accounts is None:
|
||||||
|
return None
|
||||||
|
return server.accounts.session_account(_api_key_from_headers(self.headers))
|
||||||
|
|
||||||
|
def _require_accounts(self) -> "AccountStore | None":
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if server.accounts is None:
|
||||||
|
self._send_json(404, {"error": "accounts are not enabled on this tracker"})
|
||||||
|
return None
|
||||||
|
return server.accounts
|
||||||
|
|
||||||
|
def _handle_auth_register(self):
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
body = self._read_json_body()
|
||||||
|
if body is None:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
account = accounts.register(
|
||||||
|
email=body.get("email"),
|
||||||
|
wallet=body.get("wallet"),
|
||||||
|
password=str(body.get("password") or ""),
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
self._send_json(400, {"error": str(exc)})
|
||||||
|
return
|
||||||
|
token = accounts.create_session(account["account_id"])
|
||||||
|
api_key = accounts.create_api_key(account["account_id"])
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if server.billing is not None:
|
||||||
|
server.billing.ensure_client(api_key) # grant Caller Credit up front
|
||||||
|
print(
|
||||||
|
f"[tracker] account registered: {account.get('email') or account.get('wallet')} "
|
||||||
|
f"role={account['role']}", flush=True,
|
||||||
|
)
|
||||||
|
self._send_json(200, {"account": account, "session_token": token, "api_key": api_key})
|
||||||
|
|
||||||
|
def _handle_auth_login(self):
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
body = self._read_json_body()
|
||||||
|
if body is None:
|
||||||
|
return
|
||||||
|
identifier = body.get("identifier") or body.get("email") or body.get("wallet") or ""
|
||||||
|
account = accounts.verify_login(str(identifier), str(body.get("password") or ""))
|
||||||
|
if account is None:
|
||||||
|
self._send_json(401, {"error": "invalid credentials"})
|
||||||
|
return
|
||||||
|
token = accounts.create_session(account["account_id"])
|
||||||
|
self._send_json(200, {"account": account, "session_token": token})
|
||||||
|
|
||||||
|
def _handle_auth_logout(self):
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
accounts.destroy_session(_api_key_from_headers(self.headers))
|
||||||
|
self._send_json(200, {"ok": True})
|
||||||
|
|
||||||
|
def _handle_account_me(self):
|
||||||
|
"""Balance, usage, and API keys for the logged-in account."""
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if self._require_accounts() is None:
|
||||||
|
return
|
||||||
|
account = self._session_account()
|
||||||
|
if account is None:
|
||||||
|
self._send_json(401, {"error": "login required"})
|
||||||
|
return
|
||||||
|
keys = server.accounts.keys_for(account["account_id"]) # type: ignore[union-attr]
|
||||||
|
balances = {}
|
||||||
|
usage: dict = {"requests": 0, "total_tokens": 0, "total_cost": 0.0, "recent": []}
|
||||||
|
if server.billing is not None:
|
||||||
|
balances = {key: server.billing.get_client_balance(key) for key in keys}
|
||||||
|
usage = server.billing.usage_for(keys)
|
||||||
|
self._send_json(200, {
|
||||||
|
"account": account,
|
||||||
|
"api_keys": keys,
|
||||||
|
"balances": balances,
|
||||||
|
"total_balance": sum(balances.values()),
|
||||||
|
"usage": usage,
|
||||||
|
})
|
||||||
|
|
||||||
|
def _handle_account_key_create(self):
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
account = self._session_account()
|
||||||
|
if account is None:
|
||||||
|
self._send_json(401, {"error": "login required"})
|
||||||
|
return
|
||||||
|
api_key = accounts.create_api_key(account["account_id"])
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if server.billing is not None:
|
||||||
|
server.billing.ensure_client(api_key)
|
||||||
|
self._send_json(200, {"api_key": api_key})
|
||||||
|
|
||||||
|
def _handle_account_key_revoke(self):
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
account = self._session_account()
|
||||||
|
if account is None:
|
||||||
|
self._send_json(401, {"error": "login required"})
|
||||||
|
return
|
||||||
|
body = self._read_json_body()
|
||||||
|
if body is None:
|
||||||
|
return
|
||||||
|
api_key = body.get("api_key")
|
||||||
|
if not api_key or not isinstance(api_key, str):
|
||||||
|
self._send_json(400, {"error": "api_key is required"})
|
||||||
|
return
|
||||||
|
if not accounts.revoke_api_key(account["account_id"], api_key):
|
||||||
|
self._send_json(404, {"error": "key not found on this account"})
|
||||||
|
return
|
||||||
|
self._send_json(200, {"revoked": api_key})
|
||||||
|
|
||||||
|
def _handle_admin_accounts(self):
|
||||||
|
"""Admin-only: all accounts with their keys and balances."""
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
accounts = self._require_accounts()
|
||||||
|
if accounts is None:
|
||||||
|
return
|
||||||
|
account = self._session_account()
|
||||||
|
if account is None:
|
||||||
|
self._send_json(401, {"error": "login required"})
|
||||||
|
return
|
||||||
|
if account["role"] != "admin":
|
||||||
|
self._send_json(403, {"error": "admin role required"})
|
||||||
|
return
|
||||||
|
listing = accounts.list_accounts()
|
||||||
|
if server.billing is not None:
|
||||||
|
for entry in listing:
|
||||||
|
entry["balances"] = {
|
||||||
|
key: server.billing.get_client_balance(key)
|
||||||
|
for key in entry.get("api_keys", [])
|
||||||
|
}
|
||||||
|
self._send_json(200, {"accounts": listing})
|
||||||
|
|
||||||
|
def _handle_accounts_gossip(self):
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
body = self._read_json_body()
|
||||||
|
if body is None:
|
||||||
|
return
|
||||||
|
if server.accounts is None:
|
||||||
|
self._send_json(200, {"applied": 0})
|
||||||
|
return
|
||||||
|
events = body.get("events")
|
||||||
|
if not isinstance(events, list):
|
||||||
|
self._send_json(400, {"error": "events must be a list"})
|
||||||
|
return
|
||||||
|
applied = server.accounts.apply_events([e for e in events if isinstance(e, dict)])
|
||||||
|
self._send_json(200, {"applied": applied})
|
||||||
|
|
||||||
def _handle_wallet_register(self):
|
def _handle_wallet_register(self):
|
||||||
"""Bind the caller's client wallet pubkey to their API key (US-032).
|
"""Bind the caller's client wallet pubkey to their API key (US-032).
|
||||||
|
|
||||||
@@ -2996,7 +3187,10 @@ class TrackerServer:
|
|||||||
stats_db: str | None = None,
|
stats_db: str | None = None,
|
||||||
relay_url: str | None = None,
|
relay_url: str | None = None,
|
||||||
billing: BillingLedger | None = None,
|
billing: BillingLedger | None = None,
|
||||||
|
enable_billing: bool = False,
|
||||||
billing_db: str | None = None,
|
billing_db: str | None = None,
|
||||||
|
accounts: AccountStore | None = None,
|
||||||
|
accounts_db: str | None = None,
|
||||||
benchmark_results_path: str | None = None,
|
benchmark_results_path: str | None = None,
|
||||||
treasury: Any | None = None,
|
treasury: Any | None = None,
|
||||||
deposit_poll_interval: float = 15.0,
|
deposit_poll_interval: float = 15.0,
|
||||||
@@ -3028,15 +3222,27 @@ class TrackerServer:
|
|||||||
self._stats: _StatsCollector | None = _StatsCollector(db_path=stats_db) if stats_db else _StatsCollector()
|
self._stats: _StatsCollector | None = _StatsCollector(db_path=stats_db) if stats_db else _StatsCollector()
|
||||||
self._stats_stop = threading.Event()
|
self._stats_stop = threading.Event()
|
||||||
self._stats_thread: threading.Thread | None = None
|
self._stats_thread: threading.Thread | None = None
|
||||||
if billing is None and billing_db:
|
if billing is None:
|
||||||
preset_prices = {
|
db_path = billing_db
|
||||||
name: float(preset["price_per_1k_tokens"])
|
if db_path is None and enable_billing:
|
||||||
for name, preset in self._model_presets.items()
|
db_path = DEFAULT_BILLING_DB_PATH
|
||||||
if isinstance(preset, dict) and "price_per_1k_tokens" in preset
|
if db_path:
|
||||||
}
|
preset_prices = {
|
||||||
billing = BillingLedger(db_path=billing_db, prices=preset_prices)
|
name: float(preset["price_per_1k_tokens"])
|
||||||
|
for name, preset in self._model_presets.items()
|
||||||
|
if isinstance(preset, dict) and "price_per_1k_tokens" in preset
|
||||||
|
}
|
||||||
|
billing = BillingLedger(db_path=db_path, prices=preset_prices)
|
||||||
self._billing: BillingLedger | None = billing
|
self._billing: BillingLedger | None = billing
|
||||||
self._billing_gossip_cursor = 0
|
self._billing_gossip_cursor = 0
|
||||||
|
if accounts is None:
|
||||||
|
accounts_path = accounts_db
|
||||||
|
if accounts_path is None and enable_billing:
|
||||||
|
accounts_path = DEFAULT_ACCOUNTS_DB_PATH
|
||||||
|
if accounts_path:
|
||||||
|
accounts = AccountStore(db_path=accounts_path)
|
||||||
|
self._accounts: AccountStore | None = accounts
|
||||||
|
self._accounts_gossip_cursor = 0
|
||||||
self._benchmark_results_path = benchmark_results_path
|
self._benchmark_results_path = benchmark_results_path
|
||||||
self._treasury = treasury
|
self._treasury = treasury
|
||||||
self._deposit_poll_interval = deposit_poll_interval
|
self._deposit_poll_interval = deposit_poll_interval
|
||||||
@@ -3074,6 +3280,7 @@ class TrackerServer:
|
|||||||
relay_url=effective_relay_url,
|
relay_url=effective_relay_url,
|
||||||
stats=self._stats,
|
stats=self._stats,
|
||||||
billing=self._billing,
|
billing=self._billing,
|
||||||
|
accounts=self._accounts,
|
||||||
benchmark_results_path=self._benchmark_results_path,
|
benchmark_results_path=self._benchmark_results_path,
|
||||||
)
|
)
|
||||||
self.port = self._server.server_address[1]
|
self.port = self._server.server_address[1]
|
||||||
@@ -3250,6 +3457,8 @@ class TrackerServer:
|
|||||||
self._stats.save_to_db()
|
self._stats.save_to_db()
|
||||||
if self._billing is not None:
|
if self._billing is not None:
|
||||||
self._billing.save_to_db()
|
self._billing.save_to_db()
|
||||||
|
if self._accounts is not None:
|
||||||
|
self._accounts.save_to_db()
|
||||||
if self._stats is not None and self._cluster_peers:
|
if self._stats is not None and self._cluster_peers:
|
||||||
self_url = self._cluster_self_url or f"http://{self._host}:{self.port}"
|
self_url = self._cluster_self_url or f"http://{self._host}:{self.port}"
|
||||||
local_rpms = self._stats.get_local_rpms()
|
local_rpms = self._stats.get_local_rpms()
|
||||||
@@ -3288,6 +3497,25 @@ class TrackerServer:
|
|||||||
# via event-id dedupe on the receiving side).
|
# via event-id dedupe on the receiving side).
|
||||||
if delivered_all:
|
if delivered_all:
|
||||||
self._billing_gossip_cursor = cursor
|
self._billing_gossip_cursor = cursor
|
||||||
|
if self._accounts is not None and self._cluster_peers:
|
||||||
|
events, cursor = self._accounts.events_since(self._accounts_gossip_cursor)
|
||||||
|
if events:
|
||||||
|
delivered_all = True
|
||||||
|
body = json.dumps({"events": events}).encode()
|
||||||
|
for peer in self._cluster_peers:
|
||||||
|
try:
|
||||||
|
req = urllib.request.Request(
|
||||||
|
f"{peer}/v1/accounts/gossip",
|
||||||
|
data=body,
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=2.0) as r:
|
||||||
|
r.read()
|
||||||
|
except Exception:
|
||||||
|
delivered_all = False
|
||||||
|
if delivered_all:
|
||||||
|
self._accounts_gossip_cursor = cursor
|
||||||
|
|
||||||
def stop(self) -> None:
|
def stop(self) -> None:
|
||||||
if self._raft is not None:
|
if self._raft is not None:
|
||||||
@@ -3304,6 +3532,8 @@ class TrackerServer:
|
|||||||
self._stats.save_to_db()
|
self._stats.save_to_db()
|
||||||
if self._billing is not None:
|
if self._billing is not None:
|
||||||
self._billing.save_to_db()
|
self._billing.save_to_db()
|
||||||
|
if self._accounts is not None:
|
||||||
|
self._accounts.save_to_db()
|
||||||
self._server.shutdown()
|
self._server.shutdown()
|
||||||
self._server.server_close()
|
self._server.server_close()
|
||||||
if self._thread is not None:
|
if self._thread is not None:
|
||||||
|
|||||||
221
tests/test_accounts.py
Normal file
221
tests/test_accounts.py
Normal file
@@ -0,0 +1,221 @@
|
|||||||
|
"""Dashboard user accounts: registration, login, roles, API keys, usage.
|
||||||
|
|
||||||
|
Unit tests for AccountStore plus HTTP integration on the tracker:
|
||||||
|
register/login/logout, per-account balance and usage, API-key lifecycle
|
||||||
|
(revoked keys rejected by the OpenAI proxy), and the admin listing.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from meshnet_tracker.accounts import AccountStore
|
||||||
|
from meshnet_tracker.billing import BillingLedger
|
||||||
|
from meshnet_tracker.server import TrackerServer
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- unit tests
|
||||||
|
|
||||||
|
|
||||||
|
def test_first_account_is_admin_then_users():
|
||||||
|
store = AccountStore()
|
||||||
|
first = store.register(email="admin@example.com", password="secret-123")
|
||||||
|
second = store.register(email="user@example.com", password="secret-123")
|
||||||
|
assert first["role"] == "admin"
|
||||||
|
assert second["role"] == "user"
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_requires_email_or_wallet_and_password_length():
|
||||||
|
store = AccountStore()
|
||||||
|
with pytest.raises(ValueError, match="email or a wallet"):
|
||||||
|
store.register(password="secret-123")
|
||||||
|
with pytest.raises(ValueError, match="invalid email"):
|
||||||
|
store.register(email="not-an-email", password="secret-123")
|
||||||
|
with pytest.raises(ValueError, match="at least 8"):
|
||||||
|
store.register(email="a@b.co", password="short")
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_rejects_duplicate_identifiers():
|
||||||
|
store = AccountStore()
|
||||||
|
store.register(email="dup@example.com", password="secret-123")
|
||||||
|
with pytest.raises(ValueError, match="already exists"):
|
||||||
|
store.register(email="DUP@example.com", password="other-secret")
|
||||||
|
|
||||||
|
|
||||||
|
def test_login_by_email_or_wallet():
|
||||||
|
store = AccountStore()
|
||||||
|
account = store.register(
|
||||||
|
email="both@example.com", wallet="WalletXYZ", password="secret-123"
|
||||||
|
)
|
||||||
|
assert store.verify_login("both@example.com", "secret-123")["account_id"] == account["account_id"]
|
||||||
|
assert store.verify_login("WalletXYZ", "secret-123")["account_id"] == account["account_id"]
|
||||||
|
assert store.verify_login("both@example.com", "wrong-password") is None
|
||||||
|
assert store.verify_login("nobody@example.com", "secret-123") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_sessions_resolve_and_destroy():
|
||||||
|
store = AccountStore()
|
||||||
|
account = store.register(email="s@example.com", password="secret-123")
|
||||||
|
token = store.create_session(account["account_id"])
|
||||||
|
assert store.session_account(token)["account_id"] == account["account_id"]
|
||||||
|
store.destroy_session(token)
|
||||||
|
assert store.session_account(token) is None
|
||||||
|
assert store.session_account("bogus") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_api_key_lifecycle():
|
||||||
|
store = AccountStore()
|
||||||
|
account = store.register(email="k@example.com", password="secret-123")
|
||||||
|
other = store.register(email="other@example.com", password="secret-123")
|
||||||
|
key = store.create_api_key(account["account_id"])
|
||||||
|
assert key.startswith("sk-mesh-")
|
||||||
|
assert store.keys_for(account["account_id"]) == [key]
|
||||||
|
# someone else's account cannot revoke it
|
||||||
|
assert store.revoke_api_key(other["account_id"], key) is False
|
||||||
|
assert store.revoke_api_key(account["account_id"], key) is True
|
||||||
|
assert store.keys_for(account["account_id"]) == []
|
||||||
|
assert store.is_key_revoked(key)
|
||||||
|
|
||||||
|
|
||||||
|
def test_accounts_persist_across_restart(tmp_path):
|
||||||
|
db = str(tmp_path / "accounts.db")
|
||||||
|
store = AccountStore(db_path=db)
|
||||||
|
account = store.register(email="p@example.com", password="secret-123")
|
||||||
|
key = store.create_api_key(account["account_id"])
|
||||||
|
store.save_to_db()
|
||||||
|
|
||||||
|
reloaded = AccountStore(db_path=db)
|
||||||
|
assert reloaded.verify_login("p@example.com", "secret-123") is not None
|
||||||
|
assert reloaded.keys_for(account["account_id"]) == [key]
|
||||||
|
|
||||||
|
|
||||||
|
def test_account_events_replicate_and_dedupe():
|
||||||
|
leader = AccountStore()
|
||||||
|
follower = AccountStore()
|
||||||
|
account = leader.register(email="r@example.com", password="secret-123")
|
||||||
|
key = leader.create_api_key(account["account_id"])
|
||||||
|
leader.revoke_api_key(account["account_id"], key)
|
||||||
|
|
||||||
|
events, cursor = leader.events_since(0)
|
||||||
|
assert follower.apply_events(events) == len(events)
|
||||||
|
assert follower.apply_events(events) == 0 # replay is a no-op
|
||||||
|
assert follower.verify_login("r@example.com", "secret-123") is not None
|
||||||
|
assert follower.is_key_revoked(key)
|
||||||
|
more, _ = leader.events_since(cursor)
|
||||||
|
assert more == []
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------- HTTP integration
|
||||||
|
|
||||||
|
|
||||||
|
def _call(url, method="GET", body=None, token=None):
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
if token:
|
||||||
|
headers["Authorization"] = f"Bearer {token}"
|
||||||
|
data = json.dumps(body).encode() if body is not None else None
|
||||||
|
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||||
|
with urllib.request.urlopen(req) as r:
|
||||||
|
return json.loads(r.read())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def account_tracker():
|
||||||
|
ledger = BillingLedger(starting_credit=0.5, default_price_per_1k=0.02)
|
||||||
|
tracker = TrackerServer(billing=ledger, accounts=AccountStore())
|
||||||
|
port = tracker.start()
|
||||||
|
yield f"http://127.0.0.1:{port}", ledger
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_login_and_account_view(account_tracker):
|
||||||
|
url, _ = account_tracker
|
||||||
|
reg = _call(f"{url}/v1/auth/register", "POST",
|
||||||
|
{"email": "admin@example.com", "password": "secret-123"})
|
||||||
|
assert reg["account"]["role"] == "admin"
|
||||||
|
assert reg["api_key"].startswith("sk-mesh-")
|
||||||
|
assert reg["session_token"]
|
||||||
|
|
||||||
|
login = _call(f"{url}/v1/auth/login", "POST",
|
||||||
|
{"identifier": "admin@example.com", "password": "secret-123"})
|
||||||
|
me = _call(f"{url}/v1/account", token=login["session_token"])
|
||||||
|
assert me["account"]["email"] == "admin@example.com"
|
||||||
|
assert me["api_keys"] == [reg["api_key"]]
|
||||||
|
# registration granted Caller Credit on the ledger
|
||||||
|
assert me["total_balance"] == pytest.approx(0.5)
|
||||||
|
assert me["usage"]["requests"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_bad_credentials_and_missing_session_are_401(account_tracker):
|
||||||
|
url, _ = account_tracker
|
||||||
|
_call(f"{url}/v1/auth/register", "POST",
|
||||||
|
{"email": "a@example.com", "password": "secret-123"})
|
||||||
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
_call(f"{url}/v1/auth/login", "POST",
|
||||||
|
{"identifier": "a@example.com", "password": "wrong-pass"})
|
||||||
|
assert exc_info.value.code == 401
|
||||||
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
_call(f"{url}/v1/account")
|
||||||
|
assert exc_info.value.code == 401
|
||||||
|
|
||||||
|
|
||||||
|
def test_key_create_revoke_and_revoked_key_rejected_by_proxy(account_tracker):
|
||||||
|
url, _ = account_tracker
|
||||||
|
reg = _call(f"{url}/v1/auth/register", "POST",
|
||||||
|
{"email": "k@example.com", "password": "secret-123"})
|
||||||
|
token = reg["session_token"]
|
||||||
|
|
||||||
|
new_key = _call(f"{url}/v1/account/keys", "POST", {}, token=token)["api_key"]
|
||||||
|
me = _call(f"{url}/v1/account", token=token)
|
||||||
|
assert sorted(me["api_keys"]) == sorted([reg["api_key"], new_key])
|
||||||
|
|
||||||
|
_call(f"{url}/v1/account/keys/revoke", "POST", {"api_key": new_key}, token=token)
|
||||||
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
_call(f"{url}/v1/chat/completions", "POST",
|
||||||
|
{"model": "any", "messages": []}, token=new_key)
|
||||||
|
assert exc_info.value.code == 401
|
||||||
|
assert "revoked" in exc_info.value.read().decode()
|
||||||
|
|
||||||
|
|
||||||
|
def test_admin_listing_requires_admin_role(account_tracker):
|
||||||
|
url, _ = account_tracker
|
||||||
|
admin = _call(f"{url}/v1/auth/register", "POST",
|
||||||
|
{"email": "admin@example.com", "password": "secret-123"})
|
||||||
|
user = _call(f"{url}/v1/auth/register", "POST",
|
||||||
|
{"wallet": "WalletUser1", "password": "secret-123"})
|
||||||
|
|
||||||
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
_call(f"{url}/v1/admin/accounts", token=user["session_token"])
|
||||||
|
assert exc_info.value.code == 403
|
||||||
|
|
||||||
|
listing = _call(f"{url}/v1/admin/accounts", token=admin["session_token"])
|
||||||
|
accounts = listing["accounts"]
|
||||||
|
assert len(accounts) == 2
|
||||||
|
assert accounts[0]["role"] == "admin"
|
||||||
|
assert accounts[1]["wallet"] == "WalletUser1"
|
||||||
|
assert "balances" in accounts[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_accounts_gossip_endpoint_applies_events(account_tracker):
|
||||||
|
url, _ = account_tracker
|
||||||
|
peer = AccountStore()
|
||||||
|
peer.register(email="remote@example.com", password="secret-123")
|
||||||
|
events, _ = peer.events_since(0)
|
||||||
|
result = _call(f"{url}/v1/accounts/gossip", "POST", {"events": events})
|
||||||
|
assert result["applied"] == len(events)
|
||||||
|
login = _call(f"{url}/v1/auth/login", "POST",
|
||||||
|
{"identifier": "remote@example.com", "password": "secret-123"})
|
||||||
|
assert login["account"]["email"] == "remote@example.com"
|
||||||
|
|
||||||
|
|
||||||
|
def test_accounts_endpoints_404_when_disabled():
|
||||||
|
tracker = TrackerServer() # no accounts, no billing
|
||||||
|
port = tracker.start()
|
||||||
|
try:
|
||||||
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
_call(f"http://127.0.0.1:{port}/v1/auth/register", "POST",
|
||||||
|
{"email": "x@example.com", "password": "secret-123"})
|
||||||
|
assert exc_info.value.code == 404
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
@@ -103,6 +103,22 @@ def test_restart_persistence(tmp_path):
|
|||||||
assert reloaded.snapshot()["protocol_cut"] == pytest.approx(0.02 * 0.10)
|
assert reloaded.snapshot()["protocol_cut"] == pytest.approx(0.02 * 0.10)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tracker_enables_billing_with_default_db_when_requested(tmp_path, monkeypatch):
|
||||||
|
from meshnet_tracker.billing import DEFAULT_BILLING_DB_PATH
|
||||||
|
|
||||||
|
monkeypatch.chdir(tmp_path)
|
||||||
|
tracker = TrackerServer(enable_billing=True)
|
||||||
|
port = tracker.start()
|
||||||
|
try:
|
||||||
|
summary = json.loads(
|
||||||
|
urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/billing/summary").read()
|
||||||
|
)
|
||||||
|
assert "protocol_cut" in summary
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
assert (tmp_path / DEFAULT_BILLING_DB_PATH).exists()
|
||||||
|
|
||||||
|
|
||||||
def test_event_replication_converges_and_dedupes():
|
def test_event_replication_converges_and_dedupes():
|
||||||
leader = BillingLedger(starting_credit=1.0, default_price_per_1k=0.02)
|
leader = BillingLedger(starting_credit=1.0, default_price_per_1k=0.02)
|
||||||
follower = BillingLedger(starting_credit=1.0, default_price_per_1k=0.02)
|
follower = BillingLedger(starting_credit=1.0, default_price_per_1k=0.02)
|
||||||
|
|||||||
Reference in New Issue
Block a user