d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
29 lines
963 B
Markdown
29 lines
963 B
Markdown
# US-027 — Throughput-optimized routing: effective throughput as tiebreak
|
|
|
|
Status: done
|
|
Priority: Medium
|
|
Stage: Implemented
|
|
|
|
## Context
|
|
|
|
The greedy max-reach route selection picks nodes by shard coverage but ignores node speed.
|
|
When two nodes cover the same remaining layer range, we should prefer the faster one.
|
|
This is a tiebreak only — coverage maximization remains the primary objective.
|
|
|
|
## Effective throughput formula
|
|
|
|
```
|
|
effective_throughput = benchmark_tokens_per_sec / (queue_depth + 1)
|
|
```
|
|
|
|
`benchmark_tokens_per_sec` comes from the hardware profile at registration time.
|
|
`queue_depth` comes from the last heartbeat.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `_effective_throughput(node)` helper in `server.py`
|
|
- [ ] `_select_route` uses throughput as tiebreak when `shard_end` is equal
|
|
- [ ] Test: two nodes, same shard range, different throughput → faster node selected
|
|
- [ ] Existing coverage tests still pass unchanged
|
|
- [ ] `python -m pytest` passes
|