feat: add signed ROCm diagnostic lane

This commit is contained in:
Dobromir Popov
2026-07-13 21:24:43 +03:00
parent b1c9deeb01
commit ef2a9e67e8
16 changed files with 4443 additions and 50 deletions

View File

@@ -8,3 +8,4 @@
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.

View File

@@ -0,0 +1,29 @@
# DGR ROCm and llama.cpp setup
As of 2026-07-13:
- Project ROCm runtime: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm`
- ROCm/TheRock build: `7.13.0a20260513`, target `gfx1151`
- `rocm-sdk-devel` is installed. Its expanded SDK lives under the venv at
`site-packages/_rocm_sdk_devel`.
- The wheel's redundant packaged payload was relocated to
`/home/popov/.local/share/rocm-sdk/7.13.0a20260513/rocm_sdk_devel` and symlinked
back into the venv because installing both packaged and expanded forms filled
the mounted drive. Do not reinstall it blindly; the wheel expands beyond
20 GB.
- HIP llama.cpp source: `/run/media/popov/d/DEV/llamacpp/llama.cpp`, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25` (version 9991).
- HIP build: `/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip`
- HIP `llama-server` SHA-256:
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`
- Verified device: `ROCm0: Radeon 8060S Graphics`, `gfx1151`.
- Model artifacts remain under `/run/media/popov/DATA/llm`; none were put under
`/home`.
DGR-001's immutable contract remains CPU-only. GPU evidence uses the distinct
signed `gpu-diagnostic` profile because llama-server process VRAM is not yet
measurable by the benchmark driver. The profile must capture measured
llama-server startup evidence for `ROCm0` and the actual offloaded/total layer
count; configured `device` and `n_gpu_layers` values alone are not evidence.
The accepted signer fingerprint is anchored in
`.scratch/distributed-gguf-runtime/trusted-evidence-signers.json`.