feat: harden node placement and partial model loading

This commit is contained in:
Dobromir Popov
2026-07-13 21:58:08 +02:00
parent a6bcc69288
commit 5d87e81bc9
21 changed files with 497 additions and 55 deletions

View File

@@ -46,13 +46,12 @@ model rather than waiting for an operator to request a load.
## Node ownership
- A startup-assigned `(model, shard range, quantization)` is pinned and never
changed by the tracker.
- Spare capacity on a pinned node, and all capacity on a model-less node, is
available for tracker-managed assignments.
- Tracker-added assignments are explicitly marked managed and may be moved or
removed by the tracker under the safety policy. Runtime UI controls are a
later feature.
Reconciled with [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md) and NCA (ADR-0023):
- A **startup-assigned** `(model, shard range, quantization)` from explicit `--model` or accepted bootstrap assign is **pinned** until the operator restarts.
- **Tracker-managed** assignments (this feature) use only **spare capacity** — model-less nodes or (future, US-048) unused shard slots — and are marked `managed: true`.
- The tracker may move or remove managed assignments under the safety policy below; it must not retarget a pinned serving assignment to satisfy demand.
- Every assignment, pinned or managed, must pass NCA `doctor` before becoming routable when admission is enabled.
## Pricing