feat: harden node placement and partial model loading
This commit is contained in:
@@ -123,3 +123,11 @@ Rejected. gRPC/HTTP2 already provides mature streaming, flow control, deadlines,
|
||||
4. Real two-machine execution using both Shards.
|
||||
5. End-to-end performance/fit advantage over the current distributed route.
|
||||
6. Separate Qwen3-family architecture certification.
|
||||
|
||||
## Relationship to US-042 (whole-model GGUF shortcut)
|
||||
|
||||
[US-042](../issues/42-gguf-llamacpp-node-backend.md) **phase C** ships first: a node with enough RAM serves a **full** GGUF via llama.cpp on a single-hop Inference Route using the existing HTTP activation seam and PyTorch-era tracker integration. That is intentionally small and does not require this ADR's gRPC worker or llama.cpp patch stack.
|
||||
|
||||
This ADR's track starts only after **DGR-001** (controlled safetensors-vs-GGUF benchmark) shows a meaningful speed or fit benefit. Then implement the native worker (DGR-002+) — which subsumes US-042 direction A (layer-range GGUF + boundary tensors) if the benchmark warrants it.
|
||||
|
||||
Do not run US-042 phase C and DGR-008+ in parallel on the same node backend without an explicit integration plan; phase C uses llama-cpp-python (or equivalent) whole-model path; ADR-0024 uses the standalone C++ worker.
|
||||
|
||||
Reference in New Issue
Block a user