popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Expose node shard capacity limits

Browse Source
This commit is contained in:
Dobromir Popov
2026-06-30 21:29:30 +02:00
parent 94d2216a28
commit b06b86a6f0
3 changed files with 84 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.scratch/distributed-inference-network/issues/20-memory-budget-shard-slots-and-dropout-relocation.md
View File

@@ -63,3 +63,4 @@ The current runtime still effectively has one active backend shard per node. A n
## Comments
- 2026-06-30: Created after implementing the initial registration plumbing in commit `f1e4ed6` (`--memory`, `--max-shards`, tracker validation). This issue captures the remaining end-to-end behavior so it does not conflict with US-013.
- 2026-06-30: Implementation decision: `max_loaded_shards` is currently a validated and exposed capacity field, but multi-range assignment remains reserved because `TorchNodeServer` serves one active backend shard. The tracker therefore emits at most one active range per node while exposing `vram_bytes`, `ram_bytes`, `max_loaded_shards`, quantization, throughput, and computed `max_assignable_layers` in inspection endpoints.