Record completed tracker requests into the validation log with ordinary route metadata, then have the validator fetch missing hop TOPLOC commitments on demand only after sampling. This closes the remaining alpha review gap where bisection worked only for synthetic events.
Verification: .venv/bin/python -m compileall -q packages tests; pytest tests/test_hop_bisection.py tests/test_billing_ledger.py tests/test_toploc_audit.py -q (31 passed); full pytest: 316 passed, 3 skipped, 1 env failure from meshnet-node pid 1263451 occupying port 7000.
Address post-review blockers from the alpha hardening diff: charge non-stream completions that omit usage, include prompt tokens in max_charge_per_request enforcement, and wire the CLI to a persisted local registry contract by default.
Verification: .venv/bin/python -m compileall -q packages tests; focused billing/CLI checks passed; full pytest: 314 passed, 3 skipped, 1 env failure from existing meshnet-node on port 7000.
Wire server.py handlers to the auth helper: forfeit requires validator
service token or admin session (client API keys rejected); billing summary/
settlements/registry-wallets and benchmark endpoints require admin/service;
the three gossip mutation endpoints require a fresh hive HMAC signature and
outgoing gossip pushes are signed. Dashboard sends its session token on
panel fetches. Existing tests updated for the new gates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nodes behind NAT (5G, WSL2, home routers) can now participate in
distributed pipeline inference over the internet via the relay server.
- torch_server: add module-level _relay_hop() that opens a WebSocket
to relay.../rpc/{peer_id}, sends the binary activation with
body_base64 encoding, and returns (status, headers, body)
- torch_server: _get_remaining_route returns list[dict] (was list[tuple])
preserving relay_addr from injected X-Meshnet-Route header and
from /v1/route slow-path node info
- torch_server: _run_downstream_pipeline dispatches via _relay_hop
when hop has relay_addr; falls back to direct HTTP on relay error
- tracker server: downstream_hops dicts include relay_addr when node
has one registered, so head node knows how to reach each peer
- relay_bridge: binary bodies (bfloat16 activations) use body_base64;
response preserves all X-Meshnet-* headers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_select_route now prefers nodes with higher effective throughput
(benchmark_tokens_per_sec / (queue_depth + 1)) when multiple nodes
cover the same interval, breaking the tie after max-reach selection.
TorchNodeServer.route_timeout property added for external inspection.
Tests added:
- _select_route with overlapping shards (A:0-22, B:20-24) → correct X-Meshnet-Start-Layer protocol
- _select_route with gap → clear error message
- Fast node beats slow node when shards are equal
- Busy fast node vs idle slow node (queue_depth factor)
- Two-StubNodeServer integration via TrackerServer
- route_timeout property exposed on TorchNodeServer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/v1/network/assign now scores models by (demand_rpm + 1) × (coverage_deficit + 0.01)
so high-traffic, under-covered models are preferred when assigning new nodes.
Response includes price_per_token: 0.0 (reserved for future pricing protocol).
--memory MB flag added to node CLI to override autodetected VRAM budget for
shard assignment without changing hardware detection for inference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _RollingCounter and _StatsCollector to the tracker: three rolling windows
(hour=60×1min, day=24×1hr, month=30×1day) track request RPMs per model.
GET /v1/stats returns combined local + peer stats. POST /v1/stats/gossip lets
trackers push their local slice for additive merge in hive mode.
SQLite persistence via --stats-db flag; stats survive tracker restarts.
Records a stat for each proxied /v1/chat/completions request.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Coverage endpoint now skips node purge so dead nodes remain visible.
Each band in /v1/coverage/<model> includes a `nodes` array with per-node
health: alive status, last_seen_seconds_ago, heartbeat_success_rate,
inference_success_rate, queue_depth, uptime_seconds, and endpoint.
Updated tests to assert new band structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Node now sends cumulative stats in heartbeat body:
total_requests, failed_requests, queue_depth, uptime_seconds, status
Stats are tracked thread-safely in _TorchHTTPServer; buffered locally during
outage streak and flushed on next successful heartbeat.
Tracker stores stats on _NodeEntry (total_requests, failed_requests,
queue_depth, uptime_seconds, status, heartbeats_expected/received) and
returns new_assignment in heartbeat response when pending_new_assignment is set.
Node logs received new_assignment (hot-reload wired in US-026).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When _select_route picks two nodes with overlapping registrations (e.g.
A:0-22 and B:20-24), the tracker now injects start_layer per hop so B
executes only layers 23-24, not 20-24.
- model_backend: forward_bytes + _run_layers accept start_layer offset;
clamped to shard_start to prevent out-of-bounds indexing
- torch_server: _handle_binary_forward reads X-Meshnet-Start-Layer header;
_run_downstream_pipeline sends it per hop; route is now list[tuple[str,int]]
- server: proxy injects {endpoint, start_layer} objects in X-Meshnet-Route;
/v1/route response includes start_layer per node in the nodes list
- test: fake backends accept start_layer=None kwarg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Merges worktree-feat+us-016 into master. Combined both sets of _NodeEntry
fields: hf_repo/num_layers (HEAD) and relay_addr/cert_fingerprint/peer_id
(US-017). Kept HEAD's auto-shard tracker query and shard_label formatting.
US-016: mining-style startup CLI with live ASCII dashboard, hardware
detection, auto-shard range detection from model layer count, tracker
network-assign integration for gap-filling.
US-017: P2P gossip protocol, NAT-traversal relay node, TLS peer
authentication via cert fingerprint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap wfile.write in _TrackerHandler._send_json with except BrokenPipeError
- Replace uuid4 node IDs with deterministic SHA-256 hash of endpoint+model+shards
so nodes keep the same ID on re-registration after tracker restart
- /v1/models now lists HF-repo models (not just preset models)
- /v1/coverage/{model} now resolves HF repos, not just preset names
- /v1/route response includes node_id alongside endpoint
- startup.py exposes tracker_node_id on node object and prints it in dashboard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the tracker proxies /v1/chat/completions to a first-shard node it
already holds the full registry picture. It now resolves the downstream
route inline via _select_route, strips the proxied node, and sends the
result as X-Meshnet-Route header alongside the request body.
The first-shard node reads this header in _get_remaining_route and
returns it directly, skipping the second tracker HTTP call entirely.
Falls back to the tracker query transparently when the header is absent
(direct node-to-node calls or older tracker versions).
Reduces per-inference tracker round-trips from 2 to 1.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The tracker is now the single entrypoint for inference. Clients POST to
the tracker's /v1/chat/completions and it forwards to a live tracker-mode
(first-shard) node for the requested model, applying round-robin load
balancing across multiple first-shard nodes when available.
Streaming (text/event-stream) is relayed chunk-by-chunk. Non-streaming
responses are buffered and forwarded. BrokenPipeError on client disconnect
is silenced. Upstream errors relay the original HTTP status and body.
Also adds GET /v1/health → {"status": "ok"} on the tracker.
Usage:
curl -s http://tracker:8081/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[...]}'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/route now resolves HF model nodes (by hf_repo or short name)
in addition to preset models, using the same greedy interval-cover logic.
This allows distributed inference routing across two nodes each holding
half the model.
Endpoint dedup: re-registering the same endpoint atomically replaces the
old entry so stale registrations don't accumulate across node restarts.
Purge logging: tracker now prints when a node expires due to missed
heartbeats so operators can see dead nodes being removed.
Timing fix: heartbeat timeout raised from 30s to 90s (3 missed beats);
node heartbeat interval lowered from 30s to 20s to maintain margin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/network/assign now accepts an optional `hf_repo` query param
to restrict assignment to a specific model, and returns `gap_found: bool`
so callers know whether they received a real gap vs a redundancy slot.
Node startup with --model-id (no explicit shard args) now queries the
tracker first for an uncovered gap for that model before defaulting to
full coverage (0..n-1). This means a second node with --model-id will
serve only the missing layers, not the whole model again.
Auto-join fallback (no --model-id) now prints why it fell through
instead of silently switching to stub-model.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
