532 lines
71 KiB
JSON
532 lines
71 KiB
JSON
{
|
||
"name": "Alpha Hardening",
|
||
"description": "Pre-release alpha hardening of neuron-tai money/trust path. Generated from .scratch/alpha-hardening/issues for Ralph unattended sequential execution.",
|
||
"branchName": "ralph/alpha-hardening",
|
||
"userStories": [
|
||
{
|
||
"id": "AH-001",
|
||
"title": "01 — C1: Authenticate hive gossip endpoints",
|
||
"description": "# 01 — C1: Authenticate hive gossip endpoints\n\n## What to build\n\nAdd authenticated peer identity to all tracker gossip mutation endpoints. Today any caller can push billing, account, and stats events without verification.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_gossip` (~2414–2427)\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_accounts_gossip` (~2610–2623)\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_stats_gossip` (~2355–2364)\n- `packages/tracker/meshnet_tracker/billing.py` — `apply_events` (~301–311)\n- `packages/tracker/meshnet_tracker/accounts.py` — `apply_events` (~220–226)\n\nImplement per ADR-0017 §3 using the auth helper/config from issue 02: shared hive HMAC (body + timestamp) or mutual TLS between configured tracker peers. Reject unauthenticated gossip with 401.\n\n**Note:** `/v1/gossip` (node throughput fan-out, `server.py` ~1331) is **not** in scope for this issue — see ADR-0017 §3 out-of-scope note.\n\n## Test-first\n\n1. Red: unauthenticated POST to `/v1/billing/gossip` applies a credit event today — test must fail after fix.\n2. Red: authenticated peer with valid HMAC applies events; invalid/missing auth returns 401 and `applied: 0`.\n3. Green: wire the issue-02 verifier/config (`--hive-secret` or peer cert paths) into the three hive mutation endpoints.\n\n## Acceptance criteria\n\n- [ ] `/v1/billing/gossip`, `/v1/accounts/gossip`, `/v1/stats/gossip` reject requests without valid hive auth\n- [ ] Authenticated peers replicate events as today (id-dedup preserved)\n- [ ] Config documented for multi-tracker dev setups\n- [ ] Tests cover reject + accept paths without live network\n\n## ADR links\n\n- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md` — owns shared auth middleware/config. Implement in the same PR if simpler.",
|
||
"acceptanceCriteria": [
|
||
"`/v1/billing/gossip`, `/v1/accounts/gossip`, `/v1/stats/gossip` reject requests without valid hive auth",
|
||
"Authenticated peers replicate events as today (id-dedup preserved)",
|
||
"Config documented for multi-tracker dev setups",
|
||
"Tests cover reject + accept paths without live network",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 1,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/01-c1-gossip-auth.md",
|
||
"dependsOn": [
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Marked done in source alpha-hardening issue file before PRD generation."
|
||
},
|
||
{
|
||
"id": "AH-002",
|
||
"title": "02 — A2: Unified auth boundary for privileged and financial reads",
|
||
"description": "# 02 — A2: Unified auth boundary for privileged and financial reads\n\n## What to build\n\nReplace header-presence stubs with a single auth middleware that resolves API keys, admin sessions, validator service tokens, and hive peer identity. Close leaks on financial and operator endpoints. This is the auth foundation issue; issue 01 should only apply hive auth to gossip endpoints once the helper exists.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_forfeit` (~2429–2464) — H3: non-empty `Authorization` only\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_benchmark_hop_penalty` (~2650–2658), `_handle_benchmark_results` (~2745–2748) — H3\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_summary` (~2366–2371) — H4\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_settlements` (~2407–2412) — H4\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_registry_wallets` (~2391–2405) — H4\n- `packages/tracker/meshnet_tracker/server.py` — `_session_account` (~2468+), `_handle_admin_accounts` (~2588–2608) — H4\n- `packages/tracker/meshnet_tracker/accounts.py` — `session_account()`, `create_session()` only (session store; not handler wiring)\n\nPer ADR-0017 §4: forfeit → validator or admin; benchmark → admin; billing summary/settlements/registry wallets → admin session. Include the validator service token shape from `20-validator-service-token.md` in the same implementation if practical.\n\n## Test-first\n\n1. Red: POST `/v1/billing/forfeit` with `Authorization: Bearer garbage` succeeds today — must require validator/admin identity.\n2. Red: GET `/v1/billing/summary` without admin session returns 401/403.\n3. Green: middleware + role checks; existing inference API-key path unchanged.\n\n## Acceptance criteria\n\n- [ ] Single `_require_auth(role=...)` (or equivalent) used by all privileged handlers\n- [ ] Shared auth config supports admin sessions, validator service token, and hive peer HMAC/mTLS\n- [ ] Forfeit accepts only validator service token or admin session — not arbitrary Bearer strings\n- [ ] Financial read endpoints require admin session (alpha posture)\n- [ ] Benchmark write/read require admin or service token\n- [ ] Integration tests for each endpoint class (reject unauth, accept valid)\n\n## ADR links\n\n- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)\n\n## Related\n\n- `20-validator-service-token.md` — checklist for validator service token format, rotation, forfeit auth\n\n## Blocked by\n\nNone. This issue should land before `01-c1-gossip-auth.md`.",
|
||
"acceptanceCriteria": [
|
||
"Single `_require_auth(role=...)` (or equivalent) used by all privileged handlers",
|
||
"Shared auth config supports admin sessions, validator service token, and hive peer HMAC/mTLS",
|
||
"Forfeit accepts only validator service token or admin session — not arbitrary Bearer strings",
|
||
"Financial read endpoints require admin session (alpha posture)",
|
||
"Benchmark write/read require admin or service token",
|
||
"Integration tests for each endpoint class (reject unauth, accept valid)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 2,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/02-a2-unified-auth-boundary.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Marked done in source alpha-hardening issue file before PRD generation."
|
||
},
|
||
{
|
||
"id": "AH-003",
|
||
"title": "03 — C5 + M1: Starting credit 0, funded-account gate, spend cap",
|
||
"description": "# 03 — C5 + M1: Starting credit 0, funded-account gate, spend cap\n\n## What to build\n\nClose the free-credit faucet. New API keys start at **0 USDT**; inference requires a real deposit or admin credit. Add a configurable per-request spend cap (M1) to limit runaway charges on compromised keys.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/billing.py` — `DEFAULT_STARTING_CREDIT = 1.0` (~22), `ensure_client` (~73–85), `has_funds` (~87–88), duplicate credit on charge (~130–138)\n- `packages/tracker/meshnet_tracker/server.py` — billing gate before routing (~1667–1690)\n\nPer ADR-0017 §2 and ADR-0016 §3.\n\n## Test-first\n\n1. Red: new API key gets 1.0 USDT implicit credit — test expects 0 balance until deposit.\n2. Red: first inference without deposit returns 402.\n3. Green: `DEFAULT_STARTING_CREDIT = 0.0`; optional `--max-charge-per-request` config.\n\n## Acceptance criteria\n\n- [ ] `DEFAULT_STARTING_CREDIT` is 0.0; no automatic caller credit on first touch\n- [ ] `has_funds` false for fresh keys; 402 before routing (server.py ~1684)\n- [ ] Admin `credit_client` or bound-wallet deposit still funds accounts\n- [ ] Configurable max charge per request (M1) rejects oversize completions with clear error\n- [ ] Tests: fresh key blocked; after credit/deposit, inference proceeds\n\n## ADR links\n\n- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md)\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md` (admin credit path secured)",
|
||
"acceptanceCriteria": [
|
||
"`DEFAULT_STARTING_CREDIT` is 0.0; no automatic caller credit on first touch",
|
||
"`has_funds` false for fresh keys; 402 before routing (server.py ~1684)",
|
||
"Admin `credit_client` or bound-wallet deposit still funds accounts",
|
||
"Configurable max charge per request (M1) rejects oversize completions with clear error",
|
||
"Tests: fresh key blocked; after credit/deposit, inference proceeds",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 3,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/03-c5-starting-credit-zero.md",
|
||
"dependsOn": [
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-004",
|
||
"title": "04 — H2: Tracker-authoritative token and work-unit accounting",
|
||
"description": "# 04 — H2: Tracker-authoritative token and work-unit accounting\n\n## What to build\n\nStop trusting node-reported usage for billing. The tracker already proxies responses — use tracker-observed response data and request limits to cap billable tokens, and compute work units from the **route it constructed**, not node declarations.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — `node_work` from route construction (~1776–1782, ~1781–1782)\n- `packages/tracker/meshnet_tracker/server.py` — streaming token/chunk billing (~1890–1921)\n- `packages/tracker/meshnet_tracker/server.py` — non-streaming `_usage_total_tokens` (~1938–1943)\n- `packages/tracker/meshnet_tracker/billing.py` — `charge_request` node_work split (~104–151)\n\nAccounting fraud = inflating tokens or shard span. Per ADR-0018 §5.\n\n## Test-first\n\n1. Red: mock upstream returns inflated `usage.total_tokens` in body but tracker bills that value — test expects the tracker to cap billable tokens from observed stream chunks or request bounds.\n2. Red: node registers false `shard_end`; billing uses tracker route span, not registration field alone.\n3. Green: authoritative counters; ignore node-reported work units on charge path.\n\n## Acceptance criteria\n\n- [ ] Streaming token count uses tracker-observed chunks/tokens; upstream `usage.total_tokens` can only lower or match that observed count, never inflate it\n- [ ] Non-streaming token count caps upstream `usage.total_tokens` by tracker-known request bounds (`max_tokens`, and prompt estimate if available); exact tokenizer-backed counts are deferred unless already available locally\n- [ ] Work units = tracker-computed layer span per hop at route build time (~1781–1782)\n- [ ] Nodes cannot increase payout by lying about shard range mid-request\n- [ ] Integration test: malicious node metadata does not inflate `charge_request` shares\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §5\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md`",
|
||
"acceptanceCriteria": [
|
||
"Streaming token count uses tracker-observed chunks/tokens; upstream `usage.total_tokens` can only lower or match that observed count, never inflate it",
|
||
"Non-streaming token count caps upstream `usage.total_tokens` by tracker-known request bounds (`max_tokens`, and prompt estimate if available); exact tokenizer-backed counts are deferred unless already available locally",
|
||
"Work units = tracker-computed layer span per hop at route build time (~1781–1782)",
|
||
"Nodes cannot increase payout by lying about shard range mid-request",
|
||
"Integration test: malicious node metadata does not inflate `charge_request` shares",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 4,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/04-h2-tracker-authoritative-accounting.md",
|
||
"dependsOn": [
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-005",
|
||
"title": "05 — A1/A5: Persist strike, ban, and reputation state",
|
||
"description": "# 05 — A1/A5: Persist strike, ban, and reputation state\n\n## What to build\n\nRegistry strike/ban/reputation state today lives in RAM-only `_LocalContractState` — tracker restart wipes penalties. Persist to SQLite (same pattern as `BillingLedger` and `AccountStore`) so reputation carries forward per ADR-0016 §4.\n\n**Code refs:**\n\n- `packages/contracts/meshnet_contracts/__init__.py` — `RegistryContract`, `RegistryWallet`, in-memory `_state.registry` (~103–206)\n- `packages/tracker/meshnet_tracker/billing.py` — SQLite persistence pattern (~60, event log)\n- `packages/tracker/meshnet_tracker/accounts.py` — SQLite + event replication (~40–56)\n\nInclude fields for: `strike_count`, `banned`, `completed_job_count`, graduated **reputation score** (float, default 1.0), `last_audit_ts`, probation tracking.\n\n**Scope split:** this issue owns **schema + persistence + load/reload** only. Reputation **scoring deltas** (audit pass/fail adjustments, decay rules) belong in issue 08.\n\n## Test-first\n\n1. Red: record strike, restart tracker process, strike count is 0 — must fail.\n2. Green: persist + reload; gossip replicates strike events if multi-tracker.\n3. Red: banned wallet registers node — must reject (wire to routing).\n\n## Acceptance criteria\n\n- [ ] Strike/ban/reputation survive tracker restart (SQLite or equivalent)\n- [ ] `RegistryContract.list_wallets` reflects persisted state\n- [ ] Banned wallet rejected at registration and excluded from routes\n- [ ] Reputation score field present for routing/audit issues (08–09)\n- [ ] Event-sourced mutations compatible with future Raft (ADR-0019)\n\n## ADR links\n\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md) §4\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §6\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md`",
|
||
"acceptanceCriteria": [
|
||
"Strike/ban/reputation survive tracker restart (SQLite or equivalent)",
|
||
"`RegistryContract.list_wallets` reflects persisted state",
|
||
"Banned wallet rejected at registration and excluded from routes",
|
||
"Reputation score field present for routing/audit issues (08–09)",
|
||
"Event-sourced mutations compatible with future Raft (ADR-0019)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 5,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/05-a1-a5-persist-strike-ban-reputation.md",
|
||
"dependsOn": [
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-006",
|
||
"title": "06 — FRAUD: TOPLOC integration (teacher-forced audit primitive)",
|
||
"description": "# 06 — FRAUD: TOPLOC integration (teacher-forced audit primitive)\n\n## What to build\n\nAdopt [TOPLOC](https://github.com/PrimeIntellect-ai/toploc) (MIT, `pip install toploc`) for activation fingerprint commit and verify. Replace string-equality validator checks with teacher-forced prefill + TOPLOC tolerance matching.\n\n**Estimated effort:** 2+ sessions. First landing should be the validator-only TOPLOC primitive and docs; node runtime commitments/on-demand capture can follow in issue 07 if this grows.\n\n| Subtask | Owner package | Deliverable |\n|---|---|---|\n| Validator audit primitive | `packages/validator/` | Teacher-forced prefill, TOPLOC verify, unit tests with stub tensors |\n| Node runtime commitments | `packages/node/` (if prover-side) | On-demand activation fingerprint generation on audit-selected requests; move to issue 07 if it blocks the validator primitive |\n\n**Code refs:**\n\n- `packages/validator/meshnet_validator/__init__.py` — `_run_reference`, `_outputs_match` (~92–148)\n- `packages/validator/README.md` — deterrence math (update for 19× at p=0.05)\n- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layers 1–2, build-vs-adopt table\n\nPin one canonical precision/quantization per model preset. Add `toploc` to validator (and node if prover-side) dependencies.\n\n## Test-first\n\n1. Red: validator compares final text strings — fails on cross-GPU honest divergence (document expected).\n2. Green: stub activation tensors + TOPLOC proofs round-trip in unit test.\n3. Integration: reference node teacher-forces tokens; verify accepts honest proof, rejects swapped precision.\n\n## Acceptance criteria\n\n- [ ] `toploc` dependency declared; `build_proofs_*` / `verify_proofs_*` wired\n- [ ] Validator re-runs claimed token sequence as prefill, not free generation\n- [ ] Model preset documents canonical dtype/quantization\n- [ ] README updated: 19× deterrence at 5% audit (research §1.1)\n- [ ] Tests with deterministic stub tensors (no GPU required in CI)\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §2\n- Research: [research-verifiable-inference.md](../research-verifiable-inference.md) §8, §9 build-vs-adopt\n\n## Blocked by\n\n- `05-a1-a5-persist-strike-ban-reputation.md`\n\n**Prod gate:** do not enable production audit thresholds until `21-honest-noise-calibration-corpus.md` completes (see README Phase 2 note).",
|
||
"acceptanceCriteria": [
|
||
"`toploc` dependency declared; `build_proofs_*` / `verify_proofs_*` wired",
|
||
"Validator re-runs claimed token sequence as prefill, not free generation",
|
||
"Model preset documents canonical dtype/quantization",
|
||
"README updated: 19× deterrence at 5% audit (research §1.1)",
|
||
"Tests with deterministic stub tensors (no GPU required in CI)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 6,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/06-fraud-toploc-integration.md",
|
||
"dependsOn": [
|
||
"AH-005"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-007",
|
||
"title": "07 — FRAUD: On-demand commitment + hop bisection blame",
|
||
"description": "# 07 — FRAUD: On-demand commitment + hop bisection blame\n\n## What to build\n\nOn audit selection, require nodes to supply TOPLOC-style fingerprints of **output boundary activations** per hop (on-demand, brief retention). On verify failure, referee identifies the **first divergent hop** — not always the last text node.\n\n**Code refs:**\n\n- `packages/validator/meshnet_validator/__init__.py` — `_slash_route`, `_final_text_node` bug (~102–140) — blames `max(shard_end)` only\n- `packages/tracker/meshnet_tracker/server.py` — route hop construction (~1774–1783) — cut-points for bisection\n- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §1.2, §8 layer 3 (Verde **pattern**, not on-chain game)\n\n## Test-first\n\n1. Red: two-hop route, corrupt hop-0 activations — `_final_text_node` blames hop-1 — test must fail.\n2. Green: bisection selects hop-0; forfeit targets hop-0 wallet.\n3. On-demand: commitment requested only when audit flag set on proxied request.\n\n## Acceptance criteria\n\n- [ ] Audit requests carry tracker RNG/VRF flag indistinguishable from normal traffic (research §6)\n- [ ] Nodes retain recent boundary activations for on-demand commit window (configurable TTL)\n- [ ] Validator/tracker compares fingerprints at each hop cut-point; first mismatch = culprit\n- [ ] `_final_text_node` removed or limited to text-only fallback\n- [ ] Integration test: multi-hop pipeline, fault injected at known hop\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §3–4\n\n## Blocked by\n\n- `06-fraud-toploc-integration.md`",
|
||
"acceptanceCriteria": [
|
||
"Audit requests carry tracker RNG/VRF flag indistinguishable from normal traffic (research §6)",
|
||
"Nodes retain recent boundary activations for on-demand commit window (configurable TTL)",
|
||
"Validator/tracker compares fingerprints at each hop cut-point; first mismatch = culprit",
|
||
"`_final_text_node` removed or limited to text-only fallback",
|
||
"Integration test: multi-hop pipeline, fault injected at known hop",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 7,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/07-fraud-commitment-bisection-blame.md",
|
||
"dependsOn": [
|
||
"AH-006"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-008",
|
||
"title": "08 — FRAUD: Reputation model + persistence",
|
||
"description": "# 08 — FRAUD: Reputation model + persistence\n\n## What to build\n\nImplement graduated reputation per ADR-0018 §6: score derives only from tracker audit outcomes + uptime/latency. Slow build, instant loss, inactivity decay. ×0.8 routing multiplier per strike (not whole penalty — forfeiture stays full pending).\n\n**Scope split:** issue 05 owns **schema + SQLite persistence**; this issue owns **scoring rules** (deltas, decay, strike→multiplier wiring) on top of persisted fields.\n\n**Code refs:**\n\n- `packages/contracts/meshnet_contracts/__init__.py` — extend `RegistryWallet` / persistence from issue 05\n- `packages/validator/meshnet_validator/__init__.py` — `_slash_route` forfeiture path (~125–133)\n- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292)\n- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §6\n\n## Test-first\n\n1. Red: persisted reputation/strike fields from issue 05 are ignored by scoring/routing today.\n2. Green: clean audit +0.05 (tunable); failed audit −0.3 and strike; three strikes → ban persisted via issue-05 fields.\n3. Inactivity decay after N days without completed jobs.\n\n## Acceptance criteria\n\n- [ ] Uses `reputation_score` and strike/ban fields persisted by issue 05; does not introduce a second schema path\n- [ ] Audit pass/fail updates score with documented deltas\n- [ ] Strike applies ×0.8 multiplier to routing weight (separate from forfeiture amount)\n- [ ] Ban at 3 strikes; probation job count still enforced\n- [ ] No peer-to-peer reputation inputs\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §6\n\n## Blocked by\n\n- `05-a1-a5-persist-strike-ban-reputation.md`\n- `07-fraud-commitment-bisection-blame.md` (audit outcomes feed reputation)",
|
||
"acceptanceCriteria": [
|
||
"Uses `reputation_score` and strike/ban fields persisted by issue 05; does not introduce a second schema path",
|
||
"Audit pass/fail updates score with documented deltas",
|
||
"Strike applies ×0.8 multiplier to routing weight (separate from forfeiture amount)",
|
||
"Ban at 3 strikes; probation job count still enforced",
|
||
"No peer-to-peer reputation inputs",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 8,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/08-fraud-reputation-model-persistence.md",
|
||
"dependsOn": [
|
||
"AH-005",
|
||
"AH-007"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-009",
|
||
"title": "09 — FRAUD: Reputation-weighted routing + adaptive audit rate",
|
||
"description": "# 09 — FRAUD: Reputation-weighted routing + adaptive audit rate\n\n## What to build\n\nWire reputation into route selection and audit sampling. Default network audit budget ≈5% — **not a cap**. New/low-reputation nodes: 20–30% audit rate; veterans: 2–3% floor ≥2%. Tripwires escalate rate without direct punishment.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — route selection `_select_route`, `_effective_throughput` (~1747, routing helpers)\n- `packages/validator/meshnet_validator/__init__.py` — `sample_rate=0.05`\n- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §1.1, §6, §8 layers 2–4\n\nAudit selection must be unpredictable at request time (tracker RNG after commitment window opens).\n\n## Test-first\n\n1. Red: uniform 5% sample regardless of reputation — test expects higher rate for low-reputation wallet.\n2. Green: budget balancer keeps fleet-wide average ≈ configured target.\n3. Routing prefers higher reputation among equal throughput candidates.\n\n## Acceptance criteria\n\n- [ ] Per-wallet audit probability function of reputation (newcomer high, veteran low, floor ≥2%)\n- [ ] Fleet-wide audit budget configurable (~5% default target); over ≥1000 requests with fixed seed, measured fleet audit rate within **±1.0 percentage point** of configured target (e.g. 4.0–6.0% at 5% default)\n- [ ] Route scoring includes reputation multiplier (earnings scale with tenure)\n- [ ] Passive tripwire flags (perplexity/repetition) bump audit rate only\n- [ ] Tests: deterministic seed for sampling distribution checks\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §1, §6–7\n- [ADR-0013](../../docs/adr/0013-rolling-stats-smart-routing.md)\n\n## Blocked by\n\n- `08-fraud-reputation-model-persistence.md`",
|
||
"acceptanceCriteria": [
|
||
"Per-wallet audit probability function of reputation (newcomer high, veteran low, floor ≥2%)",
|
||
"Fleet-wide audit budget configurable (~5% default target); over ≥1000 requests with fixed seed, measured fleet audit rate within **±1.0 percentage point** of configured target (e.g. 4.0–6.0% at 5% default)",
|
||
"Route scoring includes reputation multiplier (earnings scale with tenure)",
|
||
"Passive tripwire flags (perplexity/repetition) bump audit rate only",
|
||
"Tests: deterministic seed for sampling distribution checks",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 9,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/09-fraud-reputation-routing-adaptive-audit.md",
|
||
"dependsOn": [
|
||
"AH-008"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-010",
|
||
"title": "10 — FRAUD: Penalty calibration wiring (forfeit + strike + ban)",
|
||
"description": "# 10 — FRAUD: Penalty calibration wiring (forfeit + strike + ban)\n\n## What to build\n\nEnd-to-end wiring: confirmed audit failure → atomic pending forfeiture + strike + reputation decay + audit-rate snap to max. Ensure payout cannot race penalty (ADR-0015). Document 19× deterrence math in validator README.\n\n**Code refs:**\n\n- `packages/validator/meshnet_validator/__init__.py` — `_slash_route` (~102–134)\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_billing_forfeit` (~2429–2464)\n- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292), payout exclusion for banned (~3337–3344 in settlement loop)\n- `packages/validator/README.md` — update 20× → 19× at p=0.05\n\nPer ADR-0018: **full pending forfeiture** is primary penalty; ×0.8 is routing decay per strike, not partial forfeit.\n\n## Test-first\n\n1. Red: integration from issue 34 — extend with multi-hop blame wallet from issue 07.\n2. Green: node with pending balance → audit fail → pending zero, strike++, banned on 3rd, excluded from next settlement.\n3. Settlement loop skips banned wallets (~3337–3344).\n\n## Acceptance criteria\n\n- [ ] Audit failure triggers forfeiture + strike in one tracker transaction\n- [ ] Banned nodes excluded from `payables` / settlement\n- [ ] Validator uses authenticated forfeit endpoint (issue 02)\n- [ ] README: `L > 19× g` at p=0.05; pending balance = collateral\n- [ ] Integration test: 60-request fraud scenario → ban within threshold\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md)\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)\n- Research: [research-verifiable-inference.md](../research-verifiable-inference.md) §1.1\n\n## Blocked by\n\n- `07-fraud-commitment-bisection-blame.md`\n- `08-fraud-reputation-model-persistence.md`\n- `02-a2-unified-auth-boundary.md`",
|
||
"acceptanceCriteria": [
|
||
"Audit failure triggers forfeiture + strike in one tracker transaction",
|
||
"Banned nodes excluded from `payables` / settlement",
|
||
"Validator uses authenticated forfeit endpoint (issue 02)",
|
||
"README: `L > 19× g` at p=0.05; pending balance = collateral",
|
||
"Integration test: 60-request fraud scenario → ban within threshold",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 10,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/10-fraud-penalty-calibration-wiring.md",
|
||
"dependsOn": [
|
||
"AH-007",
|
||
"AH-008",
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-011",
|
||
"title": "11 — C6: Wallet binding ownership proof + binding overwrite safety",
|
||
"description": "# 11 — C6: Wallet binding ownership proof + binding overwrite safety\n\n## What to build\n\n`POST /v1/wallet/register` binds a client Solana wallet to an API key for deposit attribution. Today any Bearer key can bind any wallet string without proving ownership. Prevent hijack and accidental overwrite.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — `_handle_wallet_register` (~2625–2648)\n- `packages/tracker/meshnet_tracker/billing.py` — `bind_wallet` (~153+), `_wallet_bindings` / direct overwrite on apply (~351)\n\nRequire signed message from wallet pubkey (ed25519 via `cryptography` / solders). Reject rebinding without admin or signed release. Use explicit overwrite policy — today `~351` overwrites binding directly; gossip apply must reject conflicting binds instead of silently clobbering.\n\n## Test-first\n\n1. Red: bind wallet A with only API key, no signature — must fail after fix.\n2. Red: wallet already bound to key1; key2 cannot steal without proof.\n3. Green: valid signature binds; deposit watcher credits correct API key.\n\n## Acceptance criteria\n\n- [ ] Wallet binding requires cryptographic proof of pubkey ownership\n- [ ] One wallet → one API key (or documented admin override)\n- [ ] Gossip `bind` events cannot overwrite existing binding via direct overwrite at `~351`\n- [ ] Tests with deterministic keypairs (local adapter)\n\n## ADR links\n\n- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md) §5\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md`\n- `03-c5-starting-credit-zero.md`",
|
||
"acceptanceCriteria": [
|
||
"Wallet binding requires cryptographic proof of pubkey ownership",
|
||
"One wallet → one API key (or documented admin override)",
|
||
"Gossip `bind` events cannot overwrite existing binding via direct overwrite at `~351`",
|
||
"Tests with deterministic keypairs (local adapter)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 11,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/11-c6-wallet-binding-proof.md",
|
||
"dependsOn": [
|
||
"AH-002",
|
||
"AH-003"
|
||
],
|
||
"completionNotes": "Completed by agent; source issue status set to done, wallet binding proof implemented and tested."
|
||
},
|
||
{
|
||
"id": "AH-012",
|
||
"title": "12 — C2: On-chain settlement idempotency (deferred)",
|
||
"description": "# 12 — C2: On-chain settlement idempotency (deferred)\n\n## What to build\n\nHarden payout idempotency so Solana transaction retries never double-pay. Design accepted in ADR-0019 §1; **implementation deferred post-alpha**.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — `_settlement_loop` resend (~3331–3356), `_send_settlement` (~3358–3376)\n- `packages/contracts/meshnet_contracts/solana_adapter.py` — `send_payouts` (~186–213)\n\nToday: pending debited before broadcast with stable `settlement_id`; unconfirmed batches resent. Gap: on-chain confirmation vs ledger state if tx succeeds but confirm fails.\n\n## Acceptance criteria\n\n- [ ] `confirm_settlement` only after RPC finalized confirmation\n- [ ] Retry path reuses same `settlement_id` and detects already-confirmed signature\n- [ ] Property test: N retries → single on-chain transfer per wallet per settlement_id\n- [ ] Document recovery procedure for stuck unconfirmed batches\n\n## ADR links\n\n- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §1\n\n## Blocked by\n\nAlpha release (ADR-0016 single settlement tracker)",
|
||
"acceptanceCriteria": [
|
||
"`confirm_settlement` only after RPC finalized confirmation",
|
||
"Retry path reuses same `settlement_id` and detects already-confirmed signature",
|
||
"Property test: N retries → single on-chain transfer per wallet per settlement_id",
|
||
"Document recovery procedure for stuck unconfirmed batches",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 12,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/12-c2-on-chain-idempotency.md\nRalph skip: Source issue is ready-for-human/deferred; skipped by unattended Ralph auto.",
|
||
"dependsOn": [],
|
||
"completionNotes": "Source issue is ready-for-human/deferred; skipped by unattended Ralph auto."
|
||
},
|
||
{
|
||
"id": "AH-013",
|
||
"title": "13 — C3/C4: Consensus-gated money mutations (deferred)",
|
||
"description": "# 13 — C3/C4: Consensus-gated money mutations (deferred)\n\n## What to build\n\nRoute money-affecting ledger events through Raft commit, not gossip-only apply. Extend `raft.py` command set beyond register/deregister. Settlement remains leader-only with treasury key.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/server.py` — settlement leader gate (~3331–3332), payout batch (~3353–3356)\n- `packages/tracker/meshnet_tracker/raft.py` — log entry types (~26–27)\n- `packages/tracker/meshnet_tracker/billing.py` — `apply_events` (~301–311)\n\nDesign: ADR-0019 §2. **Deferred post-alpha** while single operator holds settlement.\n\n## Acceptance criteria\n\n- [ ] `charge`, `payout`, `forfeit`, `credit`, `settlement`, `bind` commit via Raft log\n- [ ] Followers reject direct gossip money mutations\n- [ ] Leader-only `_settlement_loop` unchanged in semantics\n- [ ] Migration plan from gossip-only billing to Raft-backed log\n\n## ADR links\n\n- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §2\n\n## Blocked by\n\n- `12-c2-on-chain-idempotency.md`\n- `14-a3-raft-durable-term-vote.md`",
|
||
"acceptanceCriteria": [
|
||
"`charge`, `payout`, `forfeit`, `credit`, `settlement`, `bind` commit via Raft log",
|
||
"Followers reject direct gossip money mutations",
|
||
"Leader-only `_settlement_loop` unchanged in semantics",
|
||
"Migration plan from gossip-only billing to Raft-backed log",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 13,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/13-c3-c4-consensus-gated-settlement.md\nRalph skip: Source issue is ready-for-human/deferred; skipped by unattended Ralph auto.",
|
||
"dependsOn": [
|
||
"AH-012",
|
||
"AH-014"
|
||
],
|
||
"completionNotes": "Source issue is ready-for-human/deferred; skipped by unattended Ralph auto."
|
||
},
|
||
{
|
||
"id": "AH-014",
|
||
"title": "14 — A3: Durable Raft term and vote state (deferred)",
|
||
"description": "# 14 — A3: Durable Raft term and vote state (deferred)\n\n## What to build\n\nPersist Raft `currentTerm`, `votedFor`, and log metadata to disk. In-memory-only term (~26) risks split leadership after tracker restart → duplicate settlement epochs.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/raft.py` — `LogEntry.term` (~25–27), election state in `RaftNode`\n\n## Acceptance criteria\n\n- [ ] Term/vote persisted alongside tracker data dir\n- [ ] Restart resumes as follower/candidate with monotonic term\n- [ ] Test: kill leader mid-settlement, restart, no duplicate payout batch\n\n## ADR links\n\n- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §3\n\n## Blocked by\n\nAlpha single-settlement posture",
|
||
"acceptanceCriteria": [
|
||
"Term/vote persisted alongside tracker data dir",
|
||
"Restart resumes as follower/candidate with monotonic term",
|
||
"Test: kill leader mid-settlement, restart, no duplicate payout batch",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 14,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/14-a3-raft-durable-term-vote.md\nRalph skip: Source issue is ready-for-human/deferred; skipped by unattended Ralph auto.",
|
||
"dependsOn": [],
|
||
"completionNotes": "Source issue is ready-for-human/deferred; skipped by unattended Ralph auto."
|
||
},
|
||
{
|
||
"id": "AH-015",
|
||
"title": "15 — H1: Commutative forfeit event ordering (deferred)",
|
||
"description": "# 15 — H1: Commutative forfeit event ordering (deferred)\n\n## What to build\n\nDefine deterministic ordering when `forfeit`, `charge`, and `payout` events replicate concurrently. Forfeit snapshots amount at creation (~287) but apply order can desync pending balances under gossip.\n\n**Code refs:**\n\n- `packages/tracker/meshnet_tracker/billing.py` — `forfeit_pending` (~280–292), `_apply_locked` forfeit branch (~345–349)\n- `packages/tracker/meshnet_tracker/billing.py` — `_pending_since.setdefault` (~324), wallet bind direct overwrite (~351)\n\n## Acceptance criteria\n\n- [ ] Documented commit order: charges before forfeit before payout for same wallet epoch\n- [ ] Forfeit events carry pending snapshot or `(term, index)` for tie-break\n- [ ] `setdefault` replaced with explicit merge rules on out-of-order apply\n- [ ] Property tests under shuffled event delivery\n\n## ADR links\n\n- [ADR-0019](../../docs/adr/0019-money-path-consistency-multi-tracker.md) §4\n\n## Blocked by\n\n- `13-c3-c4-consensus-gated-settlement.md`",
|
||
"acceptanceCriteria": [
|
||
"Documented commit order: charges before forfeit before payout for same wallet epoch",
|
||
"Forfeit events carry pending snapshot or `(term, index)` for tie-break",
|
||
"`setdefault` replaced with explicit merge rules on out-of-order apply",
|
||
"Property tests under shuffled event delivery",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 15,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/15-h1-commutative-forfeit.md\nRalph skip: Source issue is ready-for-human/deferred; skipped by unattended Ralph auto.",
|
||
"dependsOn": [
|
||
"AH-013"
|
||
],
|
||
"completionNotes": "Source issue is ready-for-human/deferred; skipped by unattended Ralph auto."
|
||
},
|
||
{
|
||
"id": "AH-016",
|
||
"title": "16 — DOC: US-006 reconciliation note",
|
||
"description": "# 16 — DOC: US-006 reconciliation note\n\n## What to build\n\nReconcile stale US-006 (Solana testnet stake contracts) with ADR-0015/0016 devnet custodial settlement. Issue `docs/issues/06-solana-stake-and-settlement.md` says \"never devnet\"; ADR-0015 explicitly targets devnet mock-USDT.\n\nAlso reconcile legacy fraud issues with the alpha-hardening fraud arc:\n\n- `docs/issues/07-fraud-detection-slash.md` — on-chain stake slash model superseded by pending-balance forfeiture + TOPLOC (ADR-0018)\n- `docs/issues/34-forfeiture-penalty.md` — partially implemented; remaining fraud work lives in `.scratch/alpha-hardening/issues/06-fraud-toploc-integration.md` through `10-fraud-penalty-calibration-wiring.md`\n\n## Acceptance criteria\n\n- [ ] Add reconciliation comment atop `docs/issues/06-solana-stake-and-settlement.md` (Status: superseded for alpha — see ADR-0015, issue 33/34)\n- [ ] Add **superseded** banner atop `docs/issues/07-fraud-detection-slash.md` → ADR-0018 + issues 06–10\n- [ ] Add **superseded for remaining scope** banner atop `docs/issues/34-forfeiture-penalty.md` → ADR-0018 + issues 06–10 (note done items: basic forfeiture wired)\n- [ ] Update `docs/prd.json` US-006 description footnote if present\n- [ ] Cross-link ADR-0015 devnet decision\n- [ ] No production code changes\n\n## ADR links\n\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)\n\n## Blocked by\n\nNone",
|
||
"acceptanceCriteria": [
|
||
"Add reconciliation comment atop `docs/issues/06-solana-stake-and-settlement.md` (Status: superseded for alpha — see ADR-0015, issue 33/34)",
|
||
"Add **superseded** banner atop `docs/issues/07-fraud-detection-slash.md` → ADR-0018 + issues 06–10",
|
||
"Add **superseded for remaining scope** banner atop `docs/issues/34-forfeiture-penalty.md` → ADR-0018 + issues 06–10 (note done items: basic forfeiture wired)",
|
||
"Update `docs/prd.json` US-006 description footnote if present",
|
||
"Cross-link ADR-0015 devnet decision",
|
||
"No production code changes",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 16,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/16-doc-us006-reconciliation.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-017",
|
||
"title": "17 — DOC: Duplicate US-020 issue dedup",
|
||
"description": "# 17 — DOC: Duplicate US-020 issue dedup\n\n## What to build\n\nTwo files share the US-020 number with different slugs:\n\n- `docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md` (ready-for-agent)\n- `docs/issues/20-tracker-node-hardening.md` (done)\n\nResolve numbering collision without losing history.\n\n## Acceptance criteria\n\n- [ ] Document canonical mapping in this issue's Comments or a short `docs/issues/README.md` note\n- [ ] Renumber or prefix disambiguation (e.g. keep done item as US-020a, renumber memory-budget to next slot) — **human approval before git mv**\n- [ ] Update any prd.json / cross-links that reference US-020 ambiguously\n- [ ] No production code changes\n\n## Blocked by\n\nHuman approval for renumbering. An agent may prepare the mapping note, but must not run `git mv` or rewrite cross-links until the canonical number is approved.",
|
||
"acceptanceCriteria": [
|
||
"Document canonical mapping in this issue's Comments or a short `docs/issues/README.md` note",
|
||
"Renumber or prefix disambiguation (e.g. keep done item as US-020a, renumber memory-budget to next slot) — **human approval before git mv**",
|
||
"Update any prd.json / cross-links that reference US-020 ambiguously",
|
||
"No production code changes",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 17,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/17-doc-duplicate-us020-dedup.md\nRalph skip: Source issue is ready-for-human/deferred; skipped by unattended Ralph auto.",
|
||
"dependsOn": [],
|
||
"completionNotes": "Source issue is ready-for-human/deferred; skipped by unattended Ralph auto."
|
||
},
|
||
{
|
||
"id": "AH-018",
|
||
"title": "18 — DOC: Operational runbooks (stubs)",
|
||
"description": "# 18 — DOC: Operational runbooks (stubs)\n\n## What to build\n\nAdd operational runbook stubs for alpha operators under `docs/runbooks/` (or `.scratch/alpha-hardening/runbooks/` until close-feature):\n\n1. **Ledger backup** — billing SQLite, accounts SQLite, registry DB paths; gossip pause procedure\n2. **Treasury key rotation** — devnet mock-USDT mint + treasury keypair rotation without double-credit\n3. **Upgrade path** — tracker rolling restart with persisted strike/reputation (post issue 05)\n\n## Acceptance criteria\n\n- [ ] Three markdown runbook stubs with prerequisites, steps, rollback\n- [ ] Reference ADR-0015 settlement loop and ADR-0016 trust assumptions\n- [ ] Secrets handling: never commit `.env.devnet`, keypairs\n- [ ] No production code changes\n\n## ADR links\n\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md)\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)\n\n## Blocked by\n\nNone (stubs can land before issue 05; update after persistence ships)",
|
||
"acceptanceCriteria": [
|
||
"Three markdown runbook stubs with prerequisites, steps, rollback",
|
||
"Reference ADR-0015 settlement loop and ADR-0016 trust assumptions",
|
||
"Secrets handling: never commit `.env.devnet`, keypairs",
|
||
"No production code changes",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 18,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/18-doc-operational-runbooks.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-019",
|
||
"title": "19 — DOC: Cryptography dependency + test environment note",
|
||
"description": "# 19 — DOC: Cryptography dependency + test environment note\n\n## What to build\n\nDocument and verify test/dev environment setup for wallet crypto paths. `packages/node/meshnet_node/wallet.py` uses `cryptography`; failures occur when `.venv` lacks deps. `cryptography>=41` is already declared in `packages/node/pyproject.toml`, so this issue should focus on documenting the editable-install path and only add root/dev extras if tests still import the node wallet without installing the node package.\n\n**Code refs:**\n\n- `packages/node/pyproject.toml` — `cryptography>=41` (verify declared)\n- `packages/node/meshnet_node/wallet.py`\n- Handoff: tests fail without `cryptography`, `openai`, `langchain` in `.venv`\n\n## Acceptance criteria\n\n- [ ] Confirm `cryptography>=41` remains in node package deps; add to root/dev extras only if tests import wallet without node install\n- [ ] Add short **Test environment** section to `docs/dev/test-env.md` (or `CONTRIBUTING.md` if created): use `.venv/Scripts/python.exe`, `pip install -e packages/node ...`, optional dep skips\n- [ ] Note which tests require optional deps (`--ignore=test_openai_gateway,...`)\n- [ ] No unrelated production code changes\n\n## Blocked by\n\nNone",
|
||
"acceptanceCriteria": [
|
||
"Confirm `cryptography>=41` remains in node package deps; add to root/dev extras only if tests import wallet without node install",
|
||
"Add short **Test environment** section to `docs/dev/test-env.md` (or `CONTRIBUTING.md` if created): use `.venv/Scripts/python.exe`, `pip install -e packages/node ...`, optional dep skips",
|
||
"Note which tests require optional deps (`--ignore=test_openai_gateway,...`)",
|
||
"No unrelated production code changes",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 19,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/19-doc-cryptography-test-env.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-020",
|
||
"title": "20 — Validator service token for `/v1/billing/forfeit`",
|
||
"description": "# 20 — Validator service token for `/v1/billing/forfeit`\n\n## What to build\n\nDefine and implement a **validator service token** distinct from client API keys and admin sessions. The validator process must authenticate when calling `POST /v1/billing/forfeit`; arbitrary Bearer strings and client API keys must be rejected. This is a checklist subtask for issue 02 and should normally land in the same PR as the unified auth middleware.\n\nPer [ADR-0017 §4](../../docs/adr/0017-tracker-authentication-and-authorization.md): forfeit accepts **validator service identity or admin session** only.\n\n## Configuration\n\n| Item | Alpha default |\n|---|---|\n| Env var | `MESHNET_VALIDATOR_SERVICE_TOKEN` (tracker + validator) |\n| Config flag | `--validator-service-token` / tracker config file equivalent |\n| Header format | `Authorization: Bearer <service-token>` with a dedicated prefix or separate header scheme documented in runbooks (e.g. `Authorization: Service <token>` — pick one and test consistently) |\n| Rotation | Manual: set new token on tracker + validator, restart both; document zero-downtime rotation as post-alpha |\n\n## Rejection rules\n\n- Client API keys (`sk-mesh-…`) → **403** on forfeit (even if valid for inference)\n- Non-empty garbage Bearer → **401/403**\n- Missing auth → **401**\n- Valid validator service token → **200** (existing forfeit semantics)\n- Admin session → **200** (operator override)\n\n## Test-first\n\n1. Red: validator (or test client) posts forfeit with a valid API key — must fail after fix.\n2. Red: `Authorization: Bearer garbage` — must fail (covered by issue 02; this issue defines the accepted token).\n3. Green: configured service token succeeds; wrong token fails.\n\n## Acceptance criteria\n\n- [ ] Service token configurable via env/flag on tracker and validator\n- [ ] Unified auth middleware resolves service token → `validator` role (issue 02)\n- [ ] API keys explicitly rejected on forfeit path\n- [ ] Integration test: validator client with service token forfeit succeeds; API key forfeit fails\n- [ ] Runbook stub: rotation procedure (manual alpha)\n\n## ADR links\n\n- [ADR-0017](../../docs/adr/0017-tracker-authentication-and-authorization.md) §4\n\n## Related\n\n- `02-a2-unified-auth-boundary.md` — middleware + role checks\n\n## Blocked by\n\n- `02-a2-unified-auth-boundary.md`",
|
||
"acceptanceCriteria": [
|
||
"Service token configurable via env/flag on tracker and validator",
|
||
"Unified auth middleware resolves service token → `validator` role (issue 02)",
|
||
"API keys explicitly rejected on forfeit path",
|
||
"Integration test: validator client with service token forfeit succeeds; API key forfeit fails",
|
||
"Runbook stub: rotation procedure (manual alpha)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 20,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/20-validator-service-token.md",
|
||
"dependsOn": [
|
||
"AH-002"
|
||
],
|
||
"completionNotes": "Marked done in source alpha-hardening issue file before PRD generation."
|
||
},
|
||
{
|
||
"id": "AH-021",
|
||
"title": "21 — Honest-noise TOPLOC calibration corpus",
|
||
"description": "# 21 — Honest-noise TOPLOC calibration corpus\n\n## What to build\n\nBefore enabling production TOPLOC audit thresholds, collect an **honest-noise baseline** across the heterogeneous volunteer fleet. Run identical inference jobs on every active node/GPU combo; measure the divergence envelope (TOPLOC exponent/mantissa deltas, logprob-rank spread) under real hardware variance.\n\nPer [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and-reputation.md): threshold calibration requires an honest-noise corpus across the fleet before production thresholds.\n\nResearch anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — \"collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds.\"\n\n## Deliverables\n\n- [ ] Scripted benchmark job (fixed prompt, model preset, seed policy) runnable on all nodes\n- [ ] Aggregated corpus artifact (per node: GPU model, dtype, TOPLOC deltas vs reference)\n- [ ] Recommended tolerance thresholds documented (p99 honest envelope + safety margin)\n- [ ] Gate checklist: production audit enable blocked until corpus covers ≥N distinct hardware profiles (define N in runbook, suggest ≥3)\n\n## Acceptance criteria\n\n- [ ] Corpus collected from current fleet (or documented subset + extrapolation note)\n- [ ] Threshold constants in validator config derived from corpus, not guessed\n- [ ] False-positive rate estimate documented at chosen thresholds\n- [ ] README / runbook cross-link: **do not enable production audits** until this issue closes\n\n## ADR links\n\n- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) — Consequences (honest-noise corpus)\n\n## Blocked by\n\n- `06-fraud-toploc-integration.md` (TOPLOC wired; calibration uses same primitive)\n\n## Blocks (prod gate)\n\n- Production enable of adaptive audit thresholds (issues 09–10 in prod)",
|
||
"acceptanceCriteria": [
|
||
"Corpus collected from current fleet (or documented subset + extrapolation note)",
|
||
"Threshold constants in validator config derived from corpus, not guessed",
|
||
"False-positive rate estimate documented at chosen thresholds",
|
||
"README / runbook cross-link: **do not enable production audits** until this issue closes",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 21,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
|
||
"dependsOn": [
|
||
"AH-006"
|
||
],
|
||
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
|
||
},
|
||
{
|
||
"id": "AH-022",
|
||
"title": "22 — DOC: MEMORY.md + project-status alpha-hardening index",
|
||
"description": "# 22 — DOC: MEMORY.md + project-status alpha-hardening index\n\n## What to build\n\nUpdate persistent memory files so agents and humans find the alpha-hardening feature without stale handoff paths.\n\n## Acceptance criteria\n\n- [x] `.claude/memory/MEMORY.md` — index entry for alpha-hardening (`.scratch/alpha-hardening/`, ADRs 0016–0019, issue count)\n- [x] `.claude/memory/project-status.md` — brief alpha-hardening section: planning complete, Bucket 1 blockers next, link README\n- [x] Cross-link `.scratch/alpha-hardening/handoff.md` from README (not temp path)\n\n## ADR links\n\n- [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md)\n\n## Blocked by\n\nNone — completed\n\n## Comments\n\n2026-07-04 triage: already satisfied by `.claude/memory/MEMORY.md`, `.claude/memory/project-status.md`, and `.scratch/alpha-hardening/README.md`.",
|
||
"acceptanceCriteria": [
|
||
"[x] `.claude/memory/MEMORY.md` — index entry for alpha-hardening (`.scratch/alpha-hardening/`, ADRs 0016–0019, issue count)",
|
||
"[x] `.claude/memory/project-status.md` — brief alpha-hardening section: planning complete, Bucket 1 blockers next, link README",
|
||
"[x] Cross-link `.scratch/alpha-hardening/handoff.md` from README (not temp path)",
|
||
"Keep the implementation scoped to this issue; do not refactor unrelated server.py areas",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 22,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/22-doc-memory-project-status.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Marked done in source alpha-hardening issue file before PRD generation."
|
||
},
|
||
{
|
||
"id": "AH-023",
|
||
"title": "23 — Dynamic per-model pricing benchmarked against HuggingFace inference rates",
|
||
"description": "Status: ready-for-agent\n\nScoped 2026-07-06 during alpha-launch-readiness grilling session. High priority, ship-soon for launch — **not** an alpha-release blocker (unlike issue 21): a stale/static price is a revenue/business-model risk, not a safety risk, so the friends + hired-VPS launch may proceed on the current static default while this lands in parallel.\n\n# 23 — Dynamic per-model pricing benchmarked against HuggingFace inference rates\n\n## What to build\n\nClient-facing price per model should track the market: **80% of the cheapest comparable provider rate on HuggingFace's inference marketplace** (`https://huggingface.co/inference/models`), refreshed daily, auto-adjusting so served models stay competitively priced as the market moves. Nodes are unaffected by this loop (per launch design: clients are the only party spending real money; node payouts come from the 90/10 split of whatever price is charged, per ADR-0015/`packages/validator/README.md`).\n\n**Current state (confirmed by code read 2026-07-06):** pricing is 100% static today. `DEFAULT_PRICE_PER_1K_TOKENS = 0.02` (`packages/tracker/meshnet_tracker/billing.py:21`) is the fallback nearly every model hits, since `model_presets.json` currently has no `price_per_1k_tokens` key for any preset. `BillingLedger.set_price(model, price)` (`billing.py:67-69`) is the only write path and already exists — no CLI/admin route calls it yet. No external HTTP/market-data integration exists anywhere in the tracker.\n\n**Data source:** `https://huggingface.co/inference/models` aggregates multiple providers (novita, together, fireworks-ai, deepinfra, etc.) with per-model, per-provider $/1M input and output token pricing; the \"cheapest\" badge already identifies the lowest-cost provider per model on the page itself. It supports a GET query param for filtering, e.g. `?search=GLM`. **No confirmed public JSON API was found** during this session's fetch — the page reads as a rendered table. Owner's suggestion: try a plain `requests` + BeautifulSoup scrape first; if the pricing table turns out to be client-rendered (not present in the initial HTML), that's the fallback signal to escalate to a headless-browser fetch (e.g. Playwright) — confirm which is needed during implementation before building the full pipeline around it. Another data source is acceptable if more convenient/stable, owner is not wedded to this specific page.\n\n## Deliverables\n\n- [x] Live-fetch attempt (requests + BeautifulSoup against the HF page with `?search=<model-family>`, or an equivalent stable source) as the primary path — confirm during implementation whether the pricing table is present in the raw HTML or requires a headless-browser fetch, and note which in the PR\n- [x] Extend `model_presets.json` per model with: `hf_aliases` (curated list of comparable HF model+provider IDs — **human-verified, not auto-discovered**), `hf_verified_match_note` (free text: params count + quantization confirmation, so a human signs off once per alias that it is a fair comparable before it's used for auto-pricing), `hf_last_price_per_1k` (derived from the $/1M rate), `hf_last_updated` (ISO date)\n- [x] Daily refresh job reusing the tracker's existing daemon-thread pattern (`_settlement_loop`/`_deposit_loop` in `server.py`, `threading.Event().wait(interval)` loop) — for each preset with a non-empty `hf_aliases` list, fetch current pricing for those aliases, compute `0.8 × cheapest matched alias price`, call `set_price()`, and update `hf_last_price_per_1k`/`hf_last_updated`\n- [x] Every price change logged (old price, new price, source alias, timestamp) — needed for dispute auditability if a client questions a charge\n- [x] Fallback behavior: empty/missing `hf_aliases`, fetch failure, or no verified match → silently keep the existing static default price. Never error the pricing path, never zero-price a model\n\n## Acceptance criteria\n\n- [x] At least one model preset has a working end-to-end refresh (alias → live fetch → 80% computed price → `set_price()` called → metadata updated) demonstrated in a test\n- [x] Models without a curated/verified alias continue to use the static default, unaffected by this feature\n- [x] Fetch failures (network error, page structure change, no match found) degrade gracefully — logged, not raised to the request path\n- [x] Price-change log is queryable/inspectable (doesn't need a UI yet — a log line or table row is sufficient for alpha)\n- [x] Note in the runbook/issue on which fetch mechanism (plain HTTP scrape vs. headless browser) was actually required, so the next person doesn't have to rediscover it\n\n## ADR links\n\n- [ADR-0015](../../docs/adr/0015-usdt-custodial-settlement.md) — settlement/pricing this touches (90/10 split, per-model pricing)\n\n## Blocked by\n\nNone — independent of the alpha-hardening trust-boundary work; touches `billing.py`/`server.py` pricing paths only.\n\n## Blocks\n\nNone — ship-soon for launch quality, not a release gate (see status note above).\n",
|
||
"acceptanceCriteria": [
|
||
"End-to-end refresh demonstrated for at least one model preset (alias -> live fetch -> 80% computed price -> set_price() -> metadata updated)",
|
||
"Models without curated/verified alias continue using static default, unaffected",
|
||
"Fetch failures degrade gracefully (logged, not raised to request path)",
|
||
"Price-change log is queryable/inspectable (log line or table row sufficient for alpha)",
|
||
"Note in issue which fetch mechanism (plain scrape vs headless browser) was actually required",
|
||
"Update the source issue file Status header to done when complete",
|
||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||
],
|
||
"priority": 23,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md. High priority, ship-soon for launch, NOT an alpha-release blocker (unlike AH-021).",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by agent"
|
||
},
|
||
{
|
||
"id": "AH-024",
|
||
"title": "24 - Finish learned-routing telemetry and live-progress cleanup",
|
||
"description": "Status: ready-for-agent\n\nScoped 2026-07-07 from an interrupted Claude session. The learned-routing feature is already committed at 518c259 (`routing improvements - dynamic (wip)`): routing_stats.py, tracker route enumeration and bandit selection, CLI routing flags, `/v1/routing`, dashboard Routing (learned), ADR-0021, and tests/test_dynamic_routing.py including the GPU(0-21)+CPU(0-39) hybrid topology. The dirty working tree contains follow-up live-progress/current-request telemetry in torch_server.py, startup.py, tracker server/dashboard, tests, and QUICKSTART. Known blocker found during resume: importing meshnet_tracker.server currently crashes at `server.py:1490` because `ws_lock: threading.Lock | None = None` evaluates `threading.Lock` as a factory function, not a type. Fix that first, then verify and commit the telemetry cleanup separately from the already-committed dynamic-routing work. Leave `.claude/settings.local.json` uncommitted unless explicitly approved.\n\nSource issue has exact file list, commands, and the reported pre-existing unrelated failure (`test_proxy_chat_splits_payout_by_tracker_assigned_route_span`).",
|
||
"acceptanceCriteria": [
|
||
"Importing `meshnet_tracker.server` no longer crashes on the lock annotation",
|
||
"Current-request heartbeat payloads are sanitized and surfaced in `/v1/network/map`",
|
||
"Node-side in-flight chat snapshots report request id, model, token count, elapsed seconds, tokens/sec, and routing completion",
|
||
"Dashboard call wall can show active requests from heartbeat data, not only tracker console terminal events",
|
||
"Targeted telemetry tests pass",
|
||
"Dynamic routing tests still pass, including GPU(0-21)+CPU(0-39) hybrid-route enumeration and traffic split behavior",
|
||
"Full or non-integration suite result is recorded; unrelated pre-existing failures are named explicitly",
|
||
"`.claude/settings.local.json` remains uncommitted unless intentionally approved"
|
||
],
|
||
"priority": 24,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/24-routing-telemetry-resume.md. Resume task for interrupted 2026-07-07 Claude session; first known fix is server.py:1490 annotation crash.",
|
||
"dependsOn": [],
|
||
"completionNotes": ""
|
||
},
|
||
{
|
||
"id": "AH-025",
|
||
"title": "25 — Sharded per-node KV cache for distributed generation (MoE/hybrid-attention aware)",
|
||
"description": "Status: ready-for-agent\n\nScoped 2026-07-08 from a live two-machine distributed-inference debugging session. The ADR-0020 mixed-topology start_layer bug is fixed (518c259, e44abc9, 1ecc599); this is the next performance blocker in the same path. The distributed generation loop has NO KV cache at all: model_backend.py passes use_cache: False in every layer-forward call, and each autoregressive step re-encodes the entire prompt-so-far from scratch, re-running every layer on every node in the route for every generated token. Observed on a live 2-node Qwen2.5-0.5B GPU pipeline: tps decayed from 22.3 (at 235 output tokens) to 12.6 (at 449 tokens) within a single generation, the expected quadratic-cost signature. X-Meshnet-Session already exists on the wire but is minted fresh per token and only labels one activation transfer for chunk reassembly/logging, not keyed to any cached state. Build: (1) stable per-request session lifecycle instead of per-token, (2) per-node sharded cache keyed by session scoped to that node's own layer range only, (3) prefill-vs-decode split so post-prefill steps send only the newest token's activation, (4) cache abstraction that holds whatever use_cache=True returns per layer range (not K/V-shaped-only) because Qwen3.6's hybrid linear-attention layers cache recurrent conv/delta state, not standard K/V, (5) TTL+LRU eviction with an explicit cache-miss fallback to full re-prefill so restarts/route-changes degrade gracefully instead of corrupting output. MoE expert routing itself is layer-local and was already ruled out as the cause of the earlier Qwen3.6 garbage-output bug (that was the start_layer double-execution); the MoE angle that matters here is architecture-awareness so the cache design does not hardcode a K/V shape assumption that breaks on Qwen3.6's hybrid attention layers.\n\nSource issue has full subtask table and code refs.",
|
||
"acceptanceCriteria": [
|
||
"A session ID is stable across all steps of one chat generation (not re-minted per token)",
|
||
"Steps after the first prefill send only the new token's activation, not the full sequence, over the wire between nodes",
|
||
"Each node caches past_key_values/recurrent state only for its own shard's layer range; no node ever holds another node's cache",
|
||
"Cache works correctly for both standard-attention shards and Qwen3.6-style hybrid linear-attention/recurrent shards",
|
||
"Bounded memory: TTL + LRU eviction; eviction/restart triggers a documented cache-miss response, not silent corruption",
|
||
"Golden-output regression test proves cached and uncached distributed generation produce equivalent output for a fixed prompt",
|
||
"Measured tps improvement recorded on the same 2-node Qwen2.5-0.5B topology used to observe the regression (target: flat tps across generation length)",
|
||
"tests/test_two_node_pipeline.py and tests/test_dynamic_routing.py still pass",
|
||
"Design captured in a new ADR (or an amendment to ADR-0020/0021) covering the cache-miss/route-change interaction"
|
||
],
|
||
"priority": 25,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/alpha-hardening/issues/25-per-node-kv-cache-distributed.md. Perf follow-up to the ADR-0020 routing fix; no prior story covered KV caching or MoE-specific caching needs.",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by agent"
|
||
}
|
||
],
|
||
"metadata": {
|
||
"updatedAt": "2026-07-08T20:09:33.742Z"
|
||
}
|
||
} |