feat(alpha): complete hardening backlog
Complete the alpha-hardening Ralph task set, including tracker billing/accounting guards, validator fraud-audit primitives, wallet binding proof support, documentation runbooks, and updated tests. Verification: .venv/bin/python -m compileall -q packages tests; .venv/bin/python -m pytest -q --tb=short (313 passed, 3 skipped, 1 failed: tests/test_mining_cli.py::test_legacy_start_without_port_uses_next_available_port because meshnet-node pid 1263451 is already listening on port 7000).
This commit is contained in:
88
.scratch/alpha-hardening/runbooks/01-ledger-backup.md
Normal file
88
.scratch/alpha-hardening/runbooks/01-ledger-backup.md
Normal file
@@ -0,0 +1,88 @@
|
||||
Status: stub
|
||||
|
||||
# Runbook: Ledger backup
|
||||
|
||||
Covers backing up the tracker's authoritative money/trust state — the billing
|
||||
ledger, dashboard accounts DB, and node registry (strike/ban/reputation) — and
|
||||
how to pause hive gossip during the backup window so peers don't replicate
|
||||
against a half-copied file.
|
||||
|
||||
## Trust assumptions (read first)
|
||||
|
||||
Per [ADR-0016](../../../docs/adr/0016-alpha-scope-and-known-limitations.md), one
|
||||
operator-designated tracker holds the treasury keypair and is the source of
|
||||
truth for settlement; other hive members only replicate. Back up **that**
|
||||
tracker's databases — a follower's copies are eventually consistent, not
|
||||
authoritative. See [ADR-0015](../../../docs/adr/0015-usdt-custodial-settlement.md)
|
||||
for the settlement loop these tables feed.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Shell access to the settlement-capable tracker host.
|
||||
- `sqlite3` CLI (or `.backup` support in the Python `sqlite3` module) available
|
||||
for online, consistent snapshots.
|
||||
- Know the tracker's configured DB paths — defaults, unless overridden by CLI
|
||||
flags:
|
||||
- Billing ledger: `billing.sqlite` (`--billing-db`, `DEFAULT_BILLING_DB_PATH`
|
||||
in `packages/tracker/meshnet_tracker/billing.py`)
|
||||
- Dashboard accounts: `accounts.sqlite` (`--accounts-db`,
|
||||
`DEFAULT_ACCOUNTS_DB_PATH` in `packages/tracker/meshnet_tracker/accounts.py`)
|
||||
- Node registry (strike/ban/reputation event log,
|
||||
`packages/contracts/meshnet_contracts/__init__.py::RegistryEventLog`): path
|
||||
is whatever was passed as `registry_db` when the tracker's
|
||||
`LocalSolanaContracts` was constructed. **As of this writing the tracker
|
||||
CLI (`meshnet_tracker/cli.py`) does not expose a `--registry-db` flag or
|
||||
wire a `contracts=` instance into `TrackerServer` by default** — confirm
|
||||
with whoever deployed this tracker whether registry persistence is
|
||||
actually enabled before assuming a file exists to back up. If it isn't
|
||||
wired up yet, strike/ban/reputation state is RAM-only and this step is
|
||||
moot until that gap closes (tracked loosely against issue 05).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Identify the actual DB paths in use (check the tracker's start command /
|
||||
systemd unit / process env for `--billing-db`, `--accounts-db`, and any
|
||||
registry DB argument).
|
||||
2. **Pause hive gossip** on this tracker so peers don't pull a partial/locked
|
||||
file mid-backup:
|
||||
- If the tracker is the sole settlement node with no `--cluster-peers`,
|
||||
gossip is already off — skip to step 3.
|
||||
- Otherwise, stop replication by restarting the process without
|
||||
`--cluster-peers` (or with an empty peer list) for the duration of the
|
||||
backup, or take the backup during a maintenance window with peers
|
||||
temporarily pointed away from this tracker at the load balancer/DNS
|
||||
level. There is currently no live "pause gossip" admin endpoint — this is
|
||||
a process-restart-level operation.
|
||||
- Confirm no in-flight `/v1/registry/gossip`, `/v1/billing/gossip`, or
|
||||
`/v1/accounts/gossip` traffic before proceeding (check access logs).
|
||||
3. Take an online, consistent copy of each SQLite file using the backup API
|
||||
rather than `cp` (WAL-mode files can be mid-write):
|
||||
```
|
||||
sqlite3 billing.sqlite ".backup '/backups/billing-$(date +%Y%m%dT%H%M%S).sqlite'"
|
||||
sqlite3 accounts.sqlite ".backup '/backups/accounts-$(date +%Y%m%dT%H%M%S).sqlite'"
|
||||
# registry DB, if configured:
|
||||
sqlite3 registry.sqlite ".backup '/backups/registry-$(date +%Y%m%dT%H%M%S).sqlite'"
|
||||
```
|
||||
4. Verify each backup opens and has rows in its expected tables
|
||||
(`billing_ledger`/event log tables, `accounts`, `registry_events`).
|
||||
5. Resume gossip (restore `--cluster-peers` / routing) once backups are
|
||||
confirmed good.
|
||||
6. Ship backups off-host per your normal retention policy. Do not store them
|
||||
alongside `.env.devnet` or keypair files (see secrets handling below).
|
||||
|
||||
## Rollback
|
||||
|
||||
- If a restore is needed, stop the tracker, replace the live `.sqlite` file(s)
|
||||
with the chosen backup, and restart. Because billing/accounts/registry each
|
||||
use append-only event logs, a stale restore under-counts recent activity
|
||||
rather than corrupting state — reconcile any gap against node/operator
|
||||
reports for the missing window before resuming payouts.
|
||||
- If gossip was paused via a peer-list restart, confirm peers re-sync
|
||||
(`events_since` catch-up) before considering the rollback complete.
|
||||
|
||||
## Secrets handling
|
||||
|
||||
- Never commit `.env.devnet`, treasury keypair JSON files, `--hive-secret`, or
|
||||
`--validator-service-token` values to a repo or ship them inside a DB backup
|
||||
archive. Back these up separately, encrypted, per your existing secrets
|
||||
process.
|
||||
Reference in New Issue
Block a user