Record completed tracker requests into the validation log with ordinary route metadata, then have the validator fetch missing hop TOPLOC commitments on demand only after sampling. This closes the remaining alpha review gap where bisection worked only for synthetic events.
Verification: .venv/bin/python -m compileall -q packages tests; pytest tests/test_hop_bisection.py tests/test_billing_ledger.py tests/test_toploc_audit.py -q (31 passed); full pytest: 316 passed, 3 skipped, 1 env failure from meshnet-node pid 1263451 occupying port 7000.
Address post-review blockers from the alpha hardening diff: charge non-stream completions that omit usage, include prompt tokens in max_charge_per_request enforcement, and wire the CLI to a persisted local registry contract by default.
Verification: .venv/bin/python -m compileall -q packages tests; focused billing/CLI checks passed; full pytest: 314 passed, 3 skipped, 1 env failure from existing meshnet-node on port 7000.
Wire server.py handlers to the auth helper: forfeit requires validator
service token or admin session (client API keys rejected); billing summary/
settlements/registry-wallets and benchmark endpoints require admin/service;
the three gossip mutation endpoints require a fresh hive HMAC signature and
outgoing gossip pushes are signed. Dashboard sends its session token on
panel fetches. Existing tests updated for the new gates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nodes behind NAT (5G, WSL2, home routers) can now participate in
distributed pipeline inference over the internet via the relay server.
- torch_server: add module-level _relay_hop() that opens a WebSocket
to relay.../rpc/{peer_id}, sends the binary activation with
body_base64 encoding, and returns (status, headers, body)
- torch_server: _get_remaining_route returns list[dict] (was list[tuple])
preserving relay_addr from injected X-Meshnet-Route header and
from /v1/route slow-path node info
- torch_server: _run_downstream_pipeline dispatches via _relay_hop
when hop has relay_addr; falls back to direct HTTP on relay error
- tracker server: downstream_hops dicts include relay_addr when node
has one registered, so head node knows how to reach each peer
- relay_bridge: binary bodies (bfloat16 activations) use body_base64;
response preserves all X-Meshnet-* headers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_select_route now prefers nodes with higher effective throughput
(benchmark_tokens_per_sec / (queue_depth + 1)) when multiple nodes
cover the same interval, breaking the tie after max-reach selection.
TorchNodeServer.route_timeout property added for external inspection.
Tests added:
- _select_route with overlapping shards (A:0-22, B:20-24) → correct X-Meshnet-Start-Layer protocol
- _select_route with gap → clear error message
- Fast node beats slow node when shards are equal
- Busy fast node vs idle slow node (queue_depth factor)
- Two-StubNodeServer integration via TrackerServer
- route_timeout property exposed on TorchNodeServer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/v1/network/assign now scores models by (demand_rpm + 1) × (coverage_deficit + 0.01)
so high-traffic, under-covered models are preferred when assigning new nodes.
Response includes price_per_token: 0.0 (reserved for future pricing protocol).
--memory MB flag added to node CLI to override autodetected VRAM budget for
shard assignment without changing hardware detection for inference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _RollingCounter and _StatsCollector to the tracker: three rolling windows
(hour=60×1min, day=24×1hr, month=30×1day) track request RPMs per model.
GET /v1/stats returns combined local + peer stats. POST /v1/stats/gossip lets
trackers push their local slice for additive merge in hive mode.
SQLite persistence via --stats-db flag; stats survive tracker restarts.
Records a stat for each proxied /v1/chat/completions request.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Coverage endpoint now skips node purge so dead nodes remain visible.
Each band in /v1/coverage/<model> includes a `nodes` array with per-node
health: alive status, last_seen_seconds_ago, heartbeat_success_rate,
inference_success_rate, queue_depth, uptime_seconds, and endpoint.
Updated tests to assert new band structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Node now sends cumulative stats in heartbeat body:
total_requests, failed_requests, queue_depth, uptime_seconds, status
Stats are tracked thread-safely in _TorchHTTPServer; buffered locally during
outage streak and flushed on next successful heartbeat.
Tracker stores stats on _NodeEntry (total_requests, failed_requests,
queue_depth, uptime_seconds, status, heartbeats_expected/received) and
returns new_assignment in heartbeat response when pending_new_assignment is set.
Node logs received new_assignment (hot-reload wired in US-026).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When _select_route picks two nodes with overlapping registrations (e.g.
A:0-22 and B:20-24), the tracker now injects start_layer per hop so B
executes only layers 23-24, not 20-24.
- model_backend: forward_bytes + _run_layers accept start_layer offset;
clamped to shard_start to prevent out-of-bounds indexing
- torch_server: _handle_binary_forward reads X-Meshnet-Start-Layer header;
_run_downstream_pipeline sends it per hop; route is now list[tuple[str,int]]
- server: proxy injects {endpoint, start_layer} objects in X-Meshnet-Route;
/v1/route response includes start_layer per node in the nodes list
- test: fake backends accept start_layer=None kwarg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Merges worktree-feat+us-016 into master. Combined both sets of _NodeEntry
fields: hf_repo/num_layers (HEAD) and relay_addr/cert_fingerprint/peer_id
(US-017). Kept HEAD's auto-shard tracker query and shard_label formatting.
US-016: mining-style startup CLI with live ASCII dashboard, hardware
detection, auto-shard range detection from model layer count, tracker
network-assign integration for gap-filling.
US-017: P2P gossip protocol, NAT-traversal relay node, TLS peer
authentication via cert fingerprint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap wfile.write in _TrackerHandler._send_json with except BrokenPipeError
- Replace uuid4 node IDs with deterministic SHA-256 hash of endpoint+model+shards
so nodes keep the same ID on re-registration after tracker restart
- /v1/models now lists HF-repo models (not just preset models)
- /v1/coverage/{model} now resolves HF repos, not just preset names
- /v1/route response includes node_id alongside endpoint
- startup.py exposes tracker_node_id on node object and prints it in dashboard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the tracker proxies /v1/chat/completions to a first-shard node it
already holds the full registry picture. It now resolves the downstream
route inline via _select_route, strips the proxied node, and sends the
result as X-Meshnet-Route header alongside the request body.
The first-shard node reads this header in _get_remaining_route and
returns it directly, skipping the second tracker HTTP call entirely.
Falls back to the tracker query transparently when the header is absent
(direct node-to-node calls or older tracker versions).
Reduces per-inference tracker round-trips from 2 to 1.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The tracker is now the single entrypoint for inference. Clients POST to
the tracker's /v1/chat/completions and it forwards to a live tracker-mode
(first-shard) node for the requested model, applying round-robin load
balancing across multiple first-shard nodes when available.
Streaming (text/event-stream) is relayed chunk-by-chunk. Non-streaming
responses are buffered and forwarded. BrokenPipeError on client disconnect
is silenced. Upstream errors relay the original HTTP status and body.
Also adds GET /v1/health → {"status": "ok"} on the tracker.
Usage:
curl -s http://tracker:8081/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[...]}'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/route now resolves HF model nodes (by hf_repo or short name)
in addition to preset models, using the same greedy interval-cover logic.
This allows distributed inference routing across two nodes each holding
half the model.
Endpoint dedup: re-registering the same endpoint atomically replaces the
old entry so stale registrations don't accumulate across node restarts.
Purge logging: tracker now prints when a node expires due to missed
heartbeats so operators can see dead nodes being removed.
Timing fix: heartbeat timeout raised from 30s to 90s (3 missed beats);
node heartbeat interval lowered from 30s to 20s to maintain margin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/network/assign now accepts an optional `hf_repo` query param
to restrict assignment to a specific model, and returns `gap_found: bool`
so callers know whether they received a real gap vs a redundancy slot.
Node startup with --model-id (no explicit shard args) now queries the
tracker first for an uncovered gap for that model before defaulting to
full coverage (0..n-1). This means a second node with --model-id will
serve only the missing layers, not the whole model again.
Auto-join fallback (no --model-id) now prints why it fell through
instead of silently switching to stub-model.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker now prints a line when a node registers and on every heartbeat
received. Node prints its assigned node_id after successful registration
and starts a daemon heartbeat thread (30s interval) that logs each send.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker:
- _NodeEntry gains hf_repo + num_layers fields (parsed from register body)
- GET /v1/network/assign — finds the biggest uncovered shard gap across
registered HF-model nodes; returns {hf_repo, shard_start, shard_end, num_layers}
- Returns 503 when no HF-model nodes are registered yet
Node startup:
- When model_id is set: registers with tracker including hf_repo + num_layers
so other nodes can auto-join this model
- When model_id is empty/None: queries /v1/network/assign, gets assigned the
missing layers, loads TorchNodeServer with the assigned shard automatically
- Fixes empty-string model_id leaking from DEFAULTS (treats "" same as None)
Usage: `meshnet-node start --tracker http://localhost:8080 --quantization bfloat16`
Node discovers what to serve and joins the network without any model flags.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Tracker: add GET /v1/tracker-nodes/<model> returning nodes registered
with tracker_mode=true whose shard_start matches the model's first layer
- Node: StubNodeServer and TorchNodeServer accept tracker_mode/tracker_url;
when tracker_mode=True (or auto-detected via shard_start==0 for Torch),
/v1/chat/completions is served alongside /forward
- TorchNodeServer: full pipeline implementation — encode_prompt → route
selection via tracker → binary forward through remaining hops → decode
- Gateway: _handle_chat_completions checks _get_tracker_nodes() first and
proxies round-robin to tracker-nodes; falls back to existing direct
pipeline when none found (preserves all US-005 backward compat)
- CLI: --tracker-mode and --tracker-url flags added to meshnet-node start
- Test: two stub tracker-nodes + two mid-shard nodes for gpt2; 10 requests;
round-robin 5/5 split verified; all OpenAI-format responses validated
- All 78 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Recovered from interrupted Codex session (ended 13:02, changes uncommitted).
All 74 tests pass.
- Tracker accepts vram_bytes, ram_bytes, quantizations[], benchmark_tokens_per_sec
in node registration payload
- GET /v1/coverage/<model_preset> returns [{start_layer, end_layer, node_count}]
- Coverage-first bin-packing: fills gaps before adding redundancy
- Speed-weighted assignment: faster nodes get wider shard ranges
- LOAD_SHARD/DROP_SHARD rebalance directives delivered via heartbeat responses
- Model is unroutable when any layer range has node_count=0
- model_presets.json config for bytes_per_layer at each quantization level
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /v1/health, GET /v1/models, POST /v1/chat/completions (streaming + non-streaming)
- OpenAI SDK, LangChain ChatOpenAI, and SSE streaming integration tests
- Tracker-backed GET /v1/models endpoint
- OpenAI-format errors for unavailable model (503) and pipeline failures
- Malformed JSON body handled with 400 instead of crash
- Test deps (openai, langchain-openai) declared in root pyproject dev extras
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
