Files
neuron-tai/.scratch/distributed-inference-network/issues/27-throughput-routing.md
Dobromir Popov 2b439e8a5f docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29
Issue files (.scratch/issues/20-29): retrospective specs for all work
done in the current sprint — hardening, route-timeout, start-layer
protocol, heartbeat stats, availability map, rolling RPM, smart
assignment, throughput routing, routing tests, relay outbound client.

ADRs (docs/adr/0011-0014):
  0011 — Auto-shard from memory budget and tracker network assignment
  0012 — X-Meshnet-Start-Layer overlapping shard execution protocol
  0013 — Rolling RPM statistics, smart assignment scoring, throughput routing
  0014 — Relay outbound client for NAT/internet pipeline hops

prd.json: US-020 through US-029 added, all marked done. ralph_progress.py
now shows 29/29 complete (100%).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 22:15:41 +03:00

963 B

US-027 — Throughput-optimized routing: effective throughput as tiebreak

Status: done Priority: Medium Stage: Implemented

Context

The greedy max-reach route selection picks nodes by shard coverage but ignores node speed. When two nodes cover the same remaining layer range, we should prefer the faster one. This is a tiebreak only — coverage maximization remains the primary objective.

Effective throughput formula

effective_throughput = benchmark_tokens_per_sec / (queue_depth + 1)

benchmark_tokens_per_sec comes from the hardware profile at registration time. queue_depth comes from the last heartbeat.

Acceptance criteria

  • _effective_throughput(node) helper in server.py
  • _select_route uses throughput as tiebreak when shard_end is equal
  • Test: two nodes, same shard range, different throughput → faster node selected
  • Existing coverage tests still pass unchanged
  • python -m pytest passes