Resolve conflicts in dashboard.html (Call wall + live TPS/queue from remote) and server.py (proxy progress logging, request id forwarding, current_requests on node entries).
Co-authored-by: Cursor <cursoragent@cursor.com>
Record completed tracker requests into the validation log with ordinary route metadata, then have the validator fetch missing hop TOPLOC commitments on demand only after sampling. This closes the remaining alpha review gap where bisection worked only for synthetic events.
Verification: .venv/bin/python -m compileall -q packages tests; pytest tests/test_hop_bisection.py tests/test_billing_ledger.py tests/test_toploc_audit.py -q (31 passed); full pytest: 316 passed, 3 skipped, 1 env failure from meshnet-node pid 1263451 occupying port 7000.
Address post-review blockers from the alpha hardening diff: charge non-stream completions that omit usage, include prompt tokens in max_charge_per_request enforcement, and wire the CLI to a persisted local registry contract by default.
Verification: .venv/bin/python -m compileall -q packages tests; focused billing/CLI checks passed; full pytest: 314 passed, 3 skipped, 1 env failure from existing meshnet-node on port 7000.
Wire server.py handlers to the auth helper: forfeit requires validator
service token or admin session (client API keys rejected); billing summary/
settlements/registry-wallets and benchmark endpoints require admin/service;
the three gossip mutation endpoints require a fresh hive HMAC signature and
outgoing gossip pushes are signed. Dashboard sends its session token on
panel fetches. Existing tests updated for the new gates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nodes behind NAT (5G, WSL2, home routers) can now participate in
distributed pipeline inference over the internet via the relay server.
- torch_server: add module-level _relay_hop() that opens a WebSocket
to relay.../rpc/{peer_id}, sends the binary activation with
body_base64 encoding, and returns (status, headers, body)
- torch_server: _get_remaining_route returns list[dict] (was list[tuple])
preserving relay_addr from injected X-Meshnet-Route header and
from /v1/route slow-path node info
- torch_server: _run_downstream_pipeline dispatches via _relay_hop
when hop has relay_addr; falls back to direct HTTP on relay error
- tracker server: downstream_hops dicts include relay_addr when node
has one registered, so head node knows how to reach each peer
- relay_bridge: binary bodies (bfloat16 activations) use body_base64;
response preserves all X-Meshnet-* headers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_select_route now prefers nodes with higher effective throughput
(benchmark_tokens_per_sec / (queue_depth + 1)) when multiple nodes
cover the same interval, breaking the tie after max-reach selection.
TorchNodeServer.route_timeout property added for external inspection.
Tests added:
- _select_route with overlapping shards (A:0-22, B:20-24) → correct X-Meshnet-Start-Layer protocol
- _select_route with gap → clear error message
- Fast node beats slow node when shards are equal
- Busy fast node vs idle slow node (queue_depth factor)
- Two-StubNodeServer integration via TrackerServer
- route_timeout property exposed on TorchNodeServer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/v1/network/assign now scores models by (demand_rpm + 1) × (coverage_deficit + 0.01)
so high-traffic, under-covered models are preferred when assigning new nodes.
Response includes price_per_token: 0.0 (reserved for future pricing protocol).
--memory MB flag added to node CLI to override autodetected VRAM budget for
shard assignment without changing hardware detection for inference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _RollingCounter and _StatsCollector to the tracker: three rolling windows
(hour=60×1min, day=24×1hr, month=30×1day) track request RPMs per model.
GET /v1/stats returns combined local + peer stats. POST /v1/stats/gossip lets
trackers push their local slice for additive merge in hive mode.
SQLite persistence via --stats-db flag; stats survive tracker restarts.
Records a stat for each proxied /v1/chat/completions request.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Coverage endpoint now skips node purge so dead nodes remain visible.
Each band in /v1/coverage/<model> includes a `nodes` array with per-node
health: alive status, last_seen_seconds_ago, heartbeat_success_rate,
inference_success_rate, queue_depth, uptime_seconds, and endpoint.
Updated tests to assert new band structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Node now sends cumulative stats in heartbeat body:
total_requests, failed_requests, queue_depth, uptime_seconds, status
Stats are tracked thread-safely in _TorchHTTPServer; buffered locally during
outage streak and flushed on next successful heartbeat.
Tracker stores stats on _NodeEntry (total_requests, failed_requests,
queue_depth, uptime_seconds, status, heartbeats_expected/received) and
returns new_assignment in heartbeat response when pending_new_assignment is set.
Node logs received new_assignment (hot-reload wired in US-026).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When _select_route picks two nodes with overlapping registrations (e.g.
A:0-22 and B:20-24), the tracker now injects start_layer per hop so B
executes only layers 23-24, not 20-24.
- model_backend: forward_bytes + _run_layers accept start_layer offset;
clamped to shard_start to prevent out-of-bounds indexing
- torch_server: _handle_binary_forward reads X-Meshnet-Start-Layer header;
_run_downstream_pipeline sends it per hop; route is now list[tuple[str,int]]
- server: proxy injects {endpoint, start_layer} objects in X-Meshnet-Route;
/v1/route response includes start_layer per node in the nodes list
- test: fake backends accept start_layer=None kwarg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
