neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-016/README.md

DGR-016 — Upstream llama.cpp collaboration package

Status: partial, blocked by DGR-010 Date: 2026-07-16

Summary

Assembled the upstream-facing collaboration package for llama.cpp without pulling Meshnet routing or control-plane logic into the upstream ask.

Durable outputs created for this story:

• api-note.md with the generic hook split and patch-per-concern proposal
• outreach.md with a maintainer-facing draft for Georgi/llama.cpp

The package is grounded in the existing research artifacts and the already implemented deterministic tests for:

• range-aware GGUF ownership and introspection
• architecture boundary input/output
• layer-filtered KV/session ownership
• reproducible pinned worker build wiring

The story itself remains blocked because DGR-010 is still marked passes: false and only has a blocked handoff, not a completed real-model acceptance README.

Files changed

• .scratch/distributed-gguf-runtime/evidence/DGR-016/README.md
• .scratch/distributed-gguf-runtime/evidence/DGR-016/api-note.md
• .scratch/distributed-gguf-runtime/evidence/DGR-016/outreach.md

Commands run and real results

Dependency and context review

sed -n '1,260p' .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/evidence/DGR-010/BLOCKED.md
sed -n '1,260p' docs/adr/0024-distributed-gguf-runtime.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/architecture.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/decision-framework.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/implementation-strategy.md
sed -n '1,260p' CONTEXT.md

Result:

• confirmed the runtime target is a small pinned llama.cpp worker with Meshnet kept outside upstream
• confirmed DGR-010 is still blocked because there is no certified dense-Llama artifact on mounted storage

Package-relevant targeted pytest

python -m pytest -q tests/test_llama_worker_build.py tests/test_gguf_backend.py tests/test_gguf_ownership.py tests/test_boundary_adapter.py tests/test_hot_kv_state.py

Result:

• 50 passed in 0.90s

Broader focused pytest slice

python -m pytest -q tests/test_llama_worker_build.py tests/test_native_shard_protocol.py tests/test_gguf_backend.py tests/test_boundary_adapter.py tests/test_gguf_ownership.py tests/test_hot_kv_state.py tests/test_kv_cache_distributed.py

Result:

• 58 passed, 1 skipped, 9 failed, 12 errors in 1.27s
• failures were pre-existing environment issues, not this documentation-only package:
  • tests/test_native_shard_protocol.py imported generated protobuf code built against gencode 7.35.0 while the active runtime is 6.33.6
  • tests/test_kv_cache_distributed.py hit sandbox socket PermissionError when trying to bind localhost servers

Research evidence review

sed -n '1,260p' docs/research/distributed-gguf-landscape.md
sed -n '1,260p' docs/research/distributed-gguf-github-followup.md
sed -n '1,220p' .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/evidence/DGR-007/README.md
sed -n '1,260p' .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md

Result:

• confirmed Nakshatra and prima.cpp are the right source/test donors for the upstream ask
• confirmed the generic API surface is range loading, boundary I/O, and KV ownership, not Meshnet policy

Package assembly

No code generation, downloads, or model execution were required for this story. The package is documentation-only and deterministic.

python -m compileall -q packages tests
git diff --check

Result:

• both commands exited 0

Correctness / performance / hardware classification

• Correctness evidence: research-only, no live model execution
• Performance evidence: none in this story
• Hardware evidence: none in this story

Known limitations and deferred work

• DGR-010 remains blocked, so this package cannot be treated as the final release-ready upstream handoff.
• The outreach draft is human-ready but not sent.
• The doc package does not change llama.cpp source code; it only prepares the upstream ask and test mapping.

Compatibility / migration notes

• Exact upstream pin for the eventual patch series: b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac
• The proposed patch split is:
  1. range-aware loading and ownership introspection
  2. boundary input/output and named tensor bundles
  3. layer-filtered KV and local sequence ownership
• Meshnet routing, billing, relay transport, and volunteer-network policy stay outside llama.cpp.
• The deterministic examples already exist in the tree and can be trimmed into upstream-facing MREs when the human maintainer sends the package.

Dependent-story handoff

• DGR-010 must clear before any real-model validation can be cited as the final end-to-end proof for this upstream package.
• Once DGR-010 has a completed evidence README, the package can be refreshed with the real-model context and sent to the llama.cpp maintainers as a smaller review bundle.