Files
neuron-tai/.scratch/distributed-inference-network/issues/27-throughput-routing.md
Dobromir Popov 2b439e8a5f docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29
Issue files (.scratch/issues/20-29): retrospective specs for all work
done in the current sprint — hardening, route-timeout, start-layer
protocol, heartbeat stats, availability map, rolling RPM, smart
assignment, throughput routing, routing tests, relay outbound client.

ADRs (docs/adr/0011-0014):
  0011 — Auto-shard from memory budget and tracker network assignment
  0012 — X-Meshnet-Start-Layer overlapping shard execution protocol
  0013 — Rolling RPM statistics, smart assignment scoring, throughput routing
  0014 — Relay outbound client for NAT/internet pipeline hops

prd.json: US-020 through US-029 added, all marked done. ralph_progress.py
now shows 29/29 complete (100%).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 22:15:41 +03:00

29 lines
963 B
Markdown

# US-027 — Throughput-optimized routing: effective throughput as tiebreak
Status: done
Priority: Medium
Stage: Implemented
## Context
The greedy max-reach route selection picks nodes by shard coverage but ignores node speed.
When two nodes cover the same remaining layer range, we should prefer the faster one.
This is a tiebreak only — coverage maximization remains the primary objective.
## Effective throughput formula
```
effective_throughput = benchmark_tokens_per_sec / (queue_depth + 1)
```
`benchmark_tokens_per_sec` comes from the hardware profile at registration time.
`queue_depth` comes from the last heartbeat.
## Acceptance criteria
- [ ] `_effective_throughput(node)` helper in `server.py`
- [ ] `_select_route` uses throughput as tiebreak when `shard_end` is equal
- [ ] Test: two nodes, same shard range, different throughput → faster node selected
- [ ] Existing coverage tests still pass unchanged
- [ ] `python -m pytest` passes