Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai

.scratch/distributed-inference-network/issues/20-tracker-node-hardening.md

@@ -0,0 +1,20 @@
+# US-020 — Tracker + node hardening: BrokenPipe fix, deterministic node IDs, HF coverage
+
+Status: done
+Priority: High
+Stage: Maintenance
+
+## Context
+
+First two-machine LAN test (US-018) exposed three reliability issues:
+
+1. `BrokenPipeError` crash in tracker `_send_json` when a slow-inference client disconnected mid-response
+2. Random UUID node IDs meant every re-registration (after tracker restart) created a phantom entry
+3. `GET /v1/coverage/<model>` returned no results when called with a short name (`Qwen2.5-0.5B`) instead of the full HF repo ID
+
+## Acceptance criteria
+
+- [ ] `BrokenPipeError` in tracker and node `_send_json` is silently swallowed
+- [ ] Node IDs are deterministic: `sha256(wallet_address + str(port))[:16]`
+- [ ] `GET /v1/coverage/<model>` accepts both short names and full `owner/repo` IDs
+- [ ] `python -m pytest` passes from repo root

.scratch/distributed-inference-network/issues/21-route-timeout-flag.md

@@ -0,0 +1,19 @@
+# US-021 — `--route-timeout` CLI flag for node tracker route lookup
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+The node's slow-path tracker route lookup (`/v1/route`) used a hard-coded 30-second HTTP timeout.
+On high-latency links (relay, satellite, 5G) or when the tracker is under load, legitimate route
+lookups were failing prematurely. The timeout is deployment-specific and should be tunable.
+
+## Acceptance criteria
+
+- [ ] `meshnet-node start` accepts `--route-timeout <seconds>` (float, default 30.0)
+- [ ] Value is passed through to `TorchNodeServer` and used in the `/v1/route` HTTP call
+- [ ] `TorchNodeServer` exposes `route_timeout` as a readable property
+- [ ] Test: setting `--route-timeout 45` is reflected as `45.0` on the running server object
+- [ ] `python -m pytest` passes

.scratch/distributed-inference-network/issues/22-start-layer-protocol.md

@@ -0,0 +1,29 @@
+# US-022 — X-Meshnet-Start-Layer: overlapping shard execution protocol
+
+Status: done
+Priority: High
+Stage: Implemented
+
+## Context
+
+Two nodes may register overlapping shard ranges (node A: 0–15, node B: 12–23) to increase
+redundancy or to enable partial-model nodes to form a complete route. Without coordination,
+node B would re-run layers 12–15 on top of activations that already include them, producing
+wrong output.
+
+The `X-Meshnet-Start-Layer` header tells each downstream node which model layer the incoming
+activation tensor represents, so the node skips layers it has already been told to skip.
+
+## Decision
+
+Option A: tracker injects `start_layer` into `X-Meshnet-Route` hops at proxy time. The head
+node passes it per-hop as `X-Meshnet-Start-Layer`. No peer-to-peer negotiation needed.
+
+## Acceptance criteria
+
+- [ ] Tracker `_handle_proxy_chat` builds route hops with `start_layer` = `covered_up_to + 1`
+- [ ] `_handle_binary_forward` reads `X-Meshnet-Start-Layer` and passes it to `backend.forward_bytes`
+- [ ] `_get_remaining_route` parses `start_layer` from injected header and from `/v1/route` slow-path
+- [ ] `TorchModelShard.forward_bytes` accepts optional `start_layer` and skips layers below it
+- [ ] Test: overlapping two-node route produces correct output without double-computing layers
+- [ ] `python -m pytest` passes

.scratch/distributed-inference-network/issues/23-heartbeat-stats.md

@@ -0,0 +1,26 @@
+# US-023 — Heartbeat stats payload: request counters + dynamic reassignment response
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+Node heartbeats are currently empty POSTs. The tracker has no visibility into per-node load,
+making load balancing and assignment decisions blind. Heartbeats should carry cumulative stats.
+The heartbeat response channel is also the natural place for the tracker to deliver reassignment
+instructions without requiring a node restart.
+
+## Acceptance criteria
+
+- [ ] Heartbeat POST body includes: `total_requests`, `failed_requests`, `queue_depth`, `uptime_seconds`, `status`
+- [ ] `TorchNodeServer` tracks the three counters with a `threading.Lock`
+- [ ] Tracker stores the last heartbeat payload per node
+- [ ] Heartbeat response may include `new_assignment: {model, shard_start, shard_end}`; node logs it
+- [ ] Stats survive tracker outage: buffered locally, flushed on next successful heartbeat
+- [ ] `python -m pytest` passes
+
+## Notes
+
+Hot-reload (loading a new shard without restart) is deferred to a future story. The response
+field is wired so trackers can send the signal; nodes log it but don't act yet.

.scratch/distributed-inference-network/issues/24-availability-map.md

@@ -0,0 +1,24 @@
+# US-024 — Enhanced availability map with per-node health details
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+`GET /v1/coverage/<model>` returns band-level coverage (start_layer, end_layer, node_count)
+but gives no visibility into which specific nodes are in each band or whether they are alive.
+Operators debugging a split-model deployment need to see node-level health at a glance.
+
+## Format decision
+
+Both: band metadata + node list grouped under each band. Dead nodes included with `healthy: false`
+so operators can see them.
+
+## Acceptance criteria
+
+- [ ] Each band in the response includes `nodes: [{node_id, endpoint, healthy, queue_depth, last_seen_s}]`
+- [ ] `healthy` is `true` iff last heartbeat < `heartbeat_timeout` seconds ago
+- [ ] Existing band fields (`start_layer`, `end_layer`, `node_count`) preserved
+- [ ] Tests updated to check band fields individually (not exact dict comparison)
+- [ ] `python -m pytest` passes

.scratch/distributed-inference-network/issues/25-rolling-rpm-stats.md

@@ -0,0 +1,30 @@
+# US-025 — Model usage statistics: rolling RPM windows + SQLite persistence
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+Trackers need per-model request-rate data to drive smart shard assignment (US-026) and
+to give operators visibility into what the network is actually serving. Stats must survive
+tracker restarts and should be shareable across a tracker cluster.
+
+## Design
+
+- `_RollingCounter`: circular-bucket counter, epoch-indexed, stale buckets auto-reset
+- Three windows: 60×1-min buckets (last hour), 24×1-hr (last day), 30×1-day (last month)
+- `_StatsCollector`: `record_request()`, `get_local_rpms()`, `merge_peer_rpms()`, `get_combined_stats()`
+- SQLite persistence via `--stats-db PATH`
+- Gossip: each tracker keeps its own slice; merge is additive (not averaged)
+
+## Acceptance criteria
+
+- [ ] `_RollingCounter` passes unit tests (record, rpm, stale bucket reset)
+- [ ] `_StatsCollector` accumulates and merges peer slices additively
+- [ ] SQLite round-trip: buckets saved and restored across restart
+- [ ] `GET /v1/stats` returns combined stats JSON
+- [ ] `POST /v1/stats/gossip` accepts peer slice and merges
+- [ ] `_handle_proxy_chat` calls `stats.record_request()` after model is resolved
+- [ ] `--stats-db PATH` CLI flag added to tracker
+- [ ] 6 unit tests pass; `python -m pytest` passes

.scratch/distributed-inference-network/issues/26-smart-model-assignment.md

@@ -0,0 +1,32 @@
+# US-026 — Smart model assignment via demand×coverage scoring
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+`/v1/network/assign` currently picks the model with the largest uncovered shard gap,
+ignoring traffic. A model serving 1000 RPM at 60% coverage is far more valuable to fill
+than a zero-traffic model at 50% coverage.
+
+## Scoring formula
+
+```
+score = (demand_rpm + 1.0) × (coverage_deficit + 0.01)
+```
+
+- `demand_rpm`: combined RPM from `_StatsCollector.get_combined_stats()`
+- `coverage_deficit`: fraction of model layers with zero node coverage, in [0.0, 1.0]
+- `+1.0` floor: models with no traffic still compete by coverage
+- `+0.01` floor: fully-covered models still have a non-zero score if they have traffic
+
+`price_per_token: 0.0` reserved in the response for future billing integration.
+
+## Acceptance criteria
+
+- [ ] `_handle_network_assign` computes score per model and returns the highest
+- [ ] Demand uses combined stats (local + peer slices)
+- [ ] `price_per_token: 0.0` present in response
+- [ ] Test: high-demand low-coverage model beats low-demand high-coverage model
+- [ ] `python -m pytest` passes

.scratch/distributed-inference-network/issues/27-throughput-routing.md

@@ -0,0 +1,28 @@
+# US-027 — Throughput-optimized routing: effective throughput as tiebreak
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+The greedy max-reach route selection picks nodes by shard coverage but ignores node speed.
+When two nodes cover the same remaining layer range, we should prefer the faster one.
+This is a tiebreak only — coverage maximization remains the primary objective.
+
+## Effective throughput formula
+
+```
+effective_throughput = benchmark_tokens_per_sec / (queue_depth + 1)
+```
+
+`benchmark_tokens_per_sec` comes from the hardware profile at registration time.
+`queue_depth` comes from the last heartbeat.
+
+## Acceptance criteria
+
+- [ ] `_effective_throughput(node)` helper in `server.py`
+- [ ] `_select_route` uses throughput as tiebreak when `shard_end` is equal
+- [ ] Test: two nodes, same shard range, different throughput → faster node selected
+- [ ] Existing coverage tests still pass unchanged
+- [ ] `python -m pytest` passes

.scratch/distributed-inference-network/issues/28-routing-correctness-tests.md

@@ -0,0 +1,26 @@
+# US-028 — Routing correctness tests: three-node, overlap, and throughput scenarios
+
+Status: done
+Priority: Medium
+Stage: Implemented
+
+## Context
+
+Route selection logic (`_select_route`) is the core of the inference network. Without
+locked-down tests, routing regressions are silent. This story adds a comprehensive
+scenario suite that locks in the routing contract.
+
+## Test scenarios
+
+1. **No-overlap three nodes**: greedy picks in layer-start order (A→C→B for ranges 0–7, 8–15, 16–23)
+   Note: the algorithm picks by earliest uncovered layer, not by node label — so if C.shard_start < B.shard_start, C comes first.
+2. **Overlapping shards**: correct resolution without double-computing layers
+3. **Throughput tiebreak**: faster node wins when shard_end is equal
+4. **Gap detection**: partial coverage returns a 503 error with description
+
+## Acceptance criteria
+
+- [ ] 7 routing tests covering the above scenarios
+- [ ] Tests use in-process tracker (no mocking)
+- [ ] `_make_node` helper for concise test setup
+- [ ] All tests pass; `python -m pytest` passes

.scratch/distributed-inference-network/issues/29-relay-outbound-client.md

@@ -0,0 +1,44 @@
+# US-029 — Outbound relay client: NAT/internet pipeline hops
+
+Status: done
+Priority: Critical
+Stage: Implemented
+
+## Context
+
+Nodes behind NAT (WSL2 with 172.x.x.x addresses, 5G mobile, home routers) register with the
+tracker and include a `relay_addr` (`wss://relay/rpc/{peer_id}`). When the head node needs to
+forward activations to such a peer, it currently fails because the direct HTTP endpoint is
+unreachable.
+
+The relay server (US-017) is already running and the node already opens a persistent outbound
+WebSocket (`RelayHttpBridge`). What is missing is the *outbound caller side*: given a `relay_addr`,
+open a per-hop WebSocket to the relay's `/rpc/{peer_id}` endpoint and send the activation through it.
+
+## Protocol
+
+```
+Node A  →  WS connect wss://relay/rpc/{peer_id_B}
+       →  send JSON: {request_id, method, path, headers, body_base64}
+Relay   →  forward as relay-http-request envelope to Node B's persistent WS
+Node B  →  process /forward locally
+       →  send relay-http-response envelope back
+Relay   →  resolve future, send response JSON to Node A
+Node A  ←  {request_id, status, headers, body_base64}
+```
+
+Binary activations (bfloat16) are base64-encoded. No precision loss.
+
+## Acceptance criteria
+
+- [ ] `_relay_hop(relay_addr, path, body, headers, timeout)` in `torch_server.py` — opens WS, sends, receives, returns `(status, headers_lower, body_bytes)`
+- [ ] `_get_remaining_route` returns `list[dict]` with `relay_addr` field (was `list[tuple]`)
+- [ ] `_run_downstream_pipeline` dispatches via `_relay_hop` when hop has `relay_addr`; falls back to direct HTTP if relay connection fails
+- [ ] Tracker `_handle_proxy_chat` includes `relay_addr` in downstream hop dicts when node has one
+- [ ] `relay_bridge._handle_request` decodes `body_base64`; response uses `body_base64` for `octet-stream` content
+- [ ] All 157 tests pass (`python -m pytest`)
+- [ ] QUICKSTART.md updated with relay NAT/internet architecture and test scenario
+
+## WSL2 test scenario
+
+Start two nodes in WSL2 pointing at the public tracker. Both get `172.x.x.x` endpoints (unreachable from outside). Both connect to relay automatically. Send an inference request through the tracker — activations flow via relay. No `--advertise-host` needed.