d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
67 lines
4.7 KiB
Markdown
67 lines
4.7 KiB
Markdown
Status: ready-for-agent
|
|
|
|
# US-020 - Memory budget, shard slots, and dropout relocation hardening
|
|
|
|
## Goal
|
|
|
|
Make node capacity limits explicit and enforce them consistently when the tracker assigns, rebalances, and relocates shards after a node dropout.
|
|
|
|
This is a follow-up to US-013, not a replacement. US-013 owns the coverage-first assignment and rebalance algorithm. This issue hardens the capacity contract around that algorithm: operator memory budget, maximum loaded shard slots, and relocation behavior when one node must absorb or split ranges after another node disappears.
|
|
|
|
## Context
|
|
|
|
Recent work added the first part of the contract:
|
|
|
|
- `meshnet-node --memory MB` is registered with the tracker as `vram_bytes` when explicitly set.
|
|
- CPU nodes without `--memory` keep the tracker default capacity, preserving old behavior.
|
|
- `meshnet-node --max-shards N` is accepted and registered as `max_loaded_shards`.
|
|
- Tracker registration validates `max_loaded_shards >= 1`.
|
|
|
|
The current runtime still effectively has one active backend shard per node. A node may advertise `max_loaded_shards`, but the tracker does not yet use multiple shard slots in bin-packing, and the node does not yet host multiple concurrently loaded shard ranges.
|
|
|
|
## Scope
|
|
|
|
- Make tracker rebalance logic account for `max_loaded_shards` as a capacity multiplier or explicit shard-slot list.
|
|
- Ensure a node is never assigned more total layers than its memory budget can support across all loaded shard slots.
|
|
- Decide and implement the runtime behavior for multiple loaded shards:
|
|
- either support multiple concurrently loaded shard backends on one node, or
|
|
- keep one backend active and treat `max_loaded_shards` as future metadata, with tracker enforcement preventing multi-range assignment for now.
|
|
- On heartbeat timeout, relocate the dropped node's uncovered layer range to eligible managed nodes while respecting both memory and shard-slot limits.
|
|
- Surface the effective memory budget and shard slot count in tracker/network inspection output so operators can diagnose why a node did or did not receive a range.
|
|
|
|
## Non-Goals
|
|
|
|
- Do not redesign the US-013 coverage-first algorithm from scratch.
|
|
- Do not change relay, `/ws`, or `/rpc` behavior.
|
|
- Do not change the token/reward model.
|
|
- Do not require public internet verification; all behavior must be locally testable.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- Tracker stores and exposes `max_loaded_shards` for registered nodes.
|
|
- Assignment/rebalance never exceeds:
|
|
- `assigned_layers_total <= floor((vram_bytes * 0.8) / bytes_per_layer_at_quant)`
|
|
- `assigned_range_count <= max_loaded_shards`
|
|
- A managed node with `max_loaded_shards=1` only receives one active shard range.
|
|
- A managed node with `max_loaded_shards=2` can absorb two non-contiguous uncovered ranges only if the node runtime supports serving both; otherwise tracker must keep assigning at most one range and document `max_loaded_shards` as reserved.
|
|
- Dropout test: register nodes covering a model, let a middle/tail node heartbeat-expire, and assert the tracker queues `LOAD_SHARD` directives that restore full coverage without violating memory or shard-slot limits.
|
|
- CLI test: `--memory` and `--max-shards` are reflected in the registration payload.
|
|
- `python -m pytest tests/test_tracker_routing.py tests/test_node_startup.py` passes in the project virtualenv, aside from any pre-existing platform-specific wallet permission assertion documented in the final notes.
|
|
|
|
## Implementation Notes
|
|
|
|
- Existing files likely involved:
|
|
- `packages/node/meshnet_node/cli.py`
|
|
- `packages/node/meshnet_node/startup.py`
|
|
- `packages/node/meshnet_node/torch_server.py`
|
|
- `packages/tracker/meshnet_tracker/server.py`
|
|
- `tests/test_tracker_routing.py`
|
|
- `tests/test_node_startup.py`
|
|
- Keep backward compatibility: nodes that omit `vram_bytes` default to tracker defaults; nodes that omit `max_loaded_shards` default to `1`.
|
|
- Prefer a small internal representation for assigned ranges if multiple ranges become real, for example `assigned_shards: list[tuple[int, int]]`, while preserving `shard_start`/`shard_end` in public responses for single-range nodes.
|
|
|
|
## Comments
|
|
|
|
- 2026-06-30: Created after implementing the initial registration plumbing in commit `f1e4ed6` (`--memory`, `--max-shards`, tracker validation). This issue captures the remaining end-to-end behavior so it does not conflict with US-013.
|
|
- 2026-06-30: Implementation decision: `max_loaded_shards` is currently a validated and exposed capacity field, but multi-range assignment remains reserved because `TorchNodeServer` serves one active backend shard. The tracker therefore emits at most one active range per node while exposing `vram_bytes`, `ram_bytes`, `max_loaded_shards`, quantization, throughput, and computed `max_assignable_layers` in inspection endpoints.
|