popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: harden node placement and partial model loading

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-13 21:58:08 +02:00
parent a6bcc69288
commit 5d87e81bc9
21 changed files with 497 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md Normal file
View File

@@ -0,0 +1,10 @@
Status: superseded
# Superseded — renumbered to US-048
This issue slot was a duplicate of tracker-node-hardening (US-020). Memory budget / shard slots / dropout relocation work lives at:
- **Issue:** [48-memory-budget-shard-slots-and-dropout-relocation.md](./48-memory-budget-shard-slots-and-dropout-relocation.md)
- **PRD:** `docs/prd.json` → `US-048`
Do not implement from this file.

12
docs/issues/42-gguf-llamacpp-node-backend.md
View File

@@ -35,7 +35,17 @@ to it (single-hop route). Smallest step, no cross-node activation work, and
already useful: Strix Halo 128 GB serves DeepSeek-V4-Flash IQ3_XXS (114 GB)
via llama.cpp Vulkan today.
Recommended sequencing: C first (small, real value), then A/B investigation.
Recommended sequencing: **C first** (US-042), then **ADR-0024 benchmark gate** (DGR-001), then distributed native worker (DGR-002+). Direction B (llama.cpp RPC) is rejected per ADR-0024.
## Runtime sequencing
| Stage | Track | Delivers |
|---|---|---|
| **C — Whole-model GGUF** | US-042 (this issue) | Single-hop llama.cpp, billing, relay streaming |
| **0 — Benchmark gate** | ADR-0024 DGR-001 | Safetensors vs GGUF measured contract |
| **1 — Distributed GGUF** | ADR-0024 `.scratch/distributed-gguf-runtime/` | gRPC C++ worker, layer-range GGUF |
Phase C uses the existing tracker hop path (whole model, one node). ADR-0024 direction A (layer-range GGUF + activations) merges into the native worker track after the benchmark gate — not in parallel with phase C on the same backend without an integration plan.
## Also in scope

15
docs/issues/44-tracker-shard-source-partial-download.md
View File

@@ -86,10 +86,10 @@ What exists already (build on it, don't duplicate):
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
layers 0–k; machine B joins with no model and receives **only** the files
for its assigned range from A — nothing fetched from HF
- [ ] Machine B's resident memory scales with its shard size, not model size
- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
- [x] Machine B's resident memory scales with its shard size, not model size
- [x] Checksums verified end-to-end; corrupted transfer falls back cleanly
- [x] Single-node/full-model flows unchanged
- [ ] `python -m pytest` passes from repo root
- [x] `python -m pytest` passes from repo root
## Implementation notes
@@ -98,6 +98,13 @@ What exists already (build on it, don't duplicate):
`full_url`; HuggingFace remains fallback-only, and when it is used the node
computes `allow_patterns` from the repo's remote SafeTensors index so it
stays layer-filtered even without tracker-cached files. Remaining hard half
is true partial model materialization: the backend can prefer a downloaded
is partial model materialization: the backend can prefer a downloaded
local model directory, but Transformers still needs a `meta`-device load
path that materializes only assigned layers.
- 2026-07-13: Partial LOAD implemented. `_load_partial_model_from_snapshot` builds
on `meta` via `init_empty_weights`, materializes only layer-scoped checkpoint
tensors, and finalizes device placement without copying unmaterialized meta
weights (`_finalize_active_shard_modules_on_device`). Tests cover memory
scaling (`test_partial_snapshot_resident_weight_numel_scales_with_shard`)
and real-torch meta-vs-materialized counts. Remaining: live two-machine LAN
verification.

2
docs/issues/49-mainnet-usdt-cutover-two-wallet-pilot.md
View File

@@ -75,7 +75,7 @@ meshnet-tracker start \
- [ ] `--starting-credit 0` — no free inference credit
- [ ] Treasury keypair not committed to git; file mode 600
- [ ] Plan multisig migration before large float ([alpha runbook](../../.scratch/alpha-hardening/runbooks/02-treasury-key-rotation.md) intent)
- [ ] Issue **21** (TOPLOC calibration) before production audit thresholds on untrusted nodes
- [ ] Issue **21** (TOPLOC calibration) before production audit thresholds on untrusted nodes — runbook: [04-toploc-calibration-run](../../.scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md)
## Cost estimate (this pilot)

15
docs/issues/50-qwen3.6-27b-demand-placement.md Normal file
View File

@@ -0,0 +1,15 @@
Status: in-design
# US-050 — Qwen3.6-27B demand-driven managed placement
> Full spec: [.scratch/qwen3.6-27b-demand-placement/PRD.md](../../.scratch/qwen3.6-27b-demand-placement/PRD.md)
> Assignment rules: [ADR-0026](../adr/0026-node-assignment-ownership-and-managed-placement.md)
> Admission: [ADR-0023](../adr/0023-model-agnostic-node-capability-admission.md)
## Summary
Deploy `Qwen/Qwen3.6-27B` when chat demand appears and **spare** fleet capacity exists. Startup `--model` assignments stay **pinned**; tracker-managed loads fill gaps on model-less or (future US-048) unused slot capacity only.
## Acceptance criteria
See scratch PRD and `docs/prd.json` US-050.