feat: harden node placement and partial model loading
This commit is contained in:
@@ -46,13 +46,12 @@ model rather than waiting for an operator to request a load.
|
||||
|
||||
## Node ownership
|
||||
|
||||
- A startup-assigned `(model, shard range, quantization)` is pinned and never
|
||||
changed by the tracker.
|
||||
- Spare capacity on a pinned node, and all capacity on a model-less node, is
|
||||
available for tracker-managed assignments.
|
||||
- Tracker-added assignments are explicitly marked managed and may be moved or
|
||||
removed by the tracker under the safety policy. Runtime UI controls are a
|
||||
later feature.
|
||||
Reconciled with [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md) and NCA (ADR-0023):
|
||||
|
||||
- A **startup-assigned** `(model, shard range, quantization)` from explicit `--model` or accepted bootstrap assign is **pinned** until the operator restarts.
|
||||
- **Tracker-managed** assignments (this feature) use only **spare capacity** — model-less nodes or (future, US-048) unused shard slots — and are marked `managed: true`.
|
||||
- The tracker may move or remove managed assignments under the safety policy below; it must not retarget a pinned serving assignment to satisfy demand.
|
||||
- Every assignment, pinned or managed, must pass NCA `doctor` before becoming routable when admission is enabled.
|
||||
|
||||
## Pricing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user