52 lines
3.4 KiB
Markdown
52 lines
3.4 KiB
Markdown
# ADR-0026: Node assignment ownership — pinned startup vs managed demand placement
|
||
|
||
## Status: Accepted
|
||
|
||
## Context
|
||
|
||
Three features define how a node gets its `(model, shard range, recipe/quantization)`:
|
||
|
||
1. **ADR-0011 / US-013** — tracker suggests a gap from coverage map on startup or auto-join.
|
||
2. **Node capability admission (ADR-0023 / NCA)** — a node must pass `doctor` + real forward before becoming routable; startup-assigned work is validated, not blindly trusted.
|
||
3. **Qwen demand placement** (`.scratch/qwen3.6-27b-demand-placement/`) — tracker deploys a model when chat demand appears and spare capacity exists.
|
||
|
||
These looked contradictory: NCA and the Qwen PRD both say startup assignments are "pinned," while demand placement wants the tracker to assign models dynamically.
|
||
|
||
## Decision
|
||
|
||
### Three assignment tiers
|
||
|
||
| Tier | How it is created | Mutable by tracker? | Admission |
|
||
|---|---|---|---|
|
||
| **Operator-initiated** | Node starts with explicit `--model` / shard flags | **No** — pinned until operator restarts or explicitly reloads | Must pass NCA `doctor` before routable |
|
||
| **Network bootstrap** | `/v1/network/assign` or `/v1/nodes/assign` on first join (ADR-0011) | **No** for the active loaded shard — treated as operator-equivalent once accepted at startup | Must pass NCA before routable |
|
||
| **Tracker-managed** | Demand-driven placement (Qwen PRD) on spare capacity | **Yes** — marked `managed: true`; subject to cooldown / safety policy | Must pass NCA for the new assignment before routable |
|
||
|
||
### Spare capacity rule (unifies NCA + Qwen)
|
||
|
||
- A node’s **active** `(model, shard, recipe)` from startup is **pinned** — the tracker does not silently retarget a serving node to a different model.
|
||
- **Spare capacity** — memory/slots not holding the pinned assignment, or a node registered without a model — may receive **tracker-managed** assignments to satisfy demand.
|
||
- Until multi-shard runtime exists (US-048), “spare capacity” effectively means **model-less nodes** or nodes explicitly registered for managed placement; do not overload a single-shard node with a second assignment.
|
||
|
||
### Demand placement interaction
|
||
|
||
- First chat request for an unrouted model queues **demand**; leader tracker may assign **managed** nodes only when eligible spare capacity exists (Qwen PRD).
|
||
- Until complete coverage + validated recipes exist, return retryable `503 model_loading` with coverage metadata.
|
||
- Managed assignments must not evict pinned assignments on other nodes without the Qwen safety policy (≥3 copies, 1.5× demand multiplier, cooldown).
|
||
|
||
### NCA is not optional for any tier
|
||
|
||
Regardless of assignment source, registration carries **validated capability** only after `doctor` succeeds. The tracker excludes nodes with absent, stale, or failed capability reports (ADR-0023).
|
||
|
||
## Consequences
|
||
|
||
- NCA and Qwen demand placement are complementary: NCA gates *quality*; demand placement gates *where new coverage comes from*.
|
||
- US-048 (multi-shard slots) extends spare capacity — until then, demand placement primarily targets nodes that join without `--model`.
|
||
- Rebalance / dropout relocation (US-013, US-048) applies to **coverage gaps**, not retroactive retargeting of pinned nodes for demand convenience.
|
||
|
||
## Verification
|
||
|
||
- NCA tests: unvalidated nodes never routed.
|
||
- Demand-placement tests (when implemented): managed flag set; pinned nodes unchanged.
|
||
- Documented in Qwen scratch PRD and NCA README cross-links.
|