feat: checkpoint distributed gguf runtime stories

This commit is contained in:
Dobromir Popov
2026-07-15 23:42:58 +03:00
parent eaf00f6add
commit 1fe31ef38d
60 changed files with 8478 additions and 105 deletions

View File

@@ -0,0 +1,130 @@
# DGR-004 — reproducible pinned llama.cpp patch stack evidence
Status: done
Date: 2026-07-15
Evidence kind: **synthetic-build + repo checks**. No model download, no GPU,
no network fetch during validation, no API credits.
## Summary
Implemented the reproducible source-dependency boundary for llama.cpp and kept
the fork seam narrow and auditable:
- exact pinned upstream commit and repository metadata
- numbered patch stack isolated under `packages/node/native/llama/patches/`
- build script that verifies the pin, applies the patch stack, stages notices,
and compiles a standalone worker scaffold without manual source copying
- upstream file assumptions and fail-closed pin checking
- license/attribution preservation by staging upstream `LICENSE` and `AUTHORS`
- clean rebuild smoke test that only uses a fake local checkout and does not
download a model
The native smoke path is intentionally minimal in this story. It proves the
reproducible source dependency and build seam without pulling Meshnet protocol
code into llama.cpp.
## Files changed
- `packages/node/native/llama/UPSTREAM_COMMIT`
- `packages/node/native/llama/UPSTREAM_REPOSITORY`
- `packages/node/native/llama/UPSTREAM_ASSUMPTIONS.md`
- `packages/node/native/llama/README.md`
- `packages/node/native/llama/patches/0001-add-meshnet-worker-scaffold.patch`
- `packages/node/native/llama/templates/meshnet_worker.cpp`
- `packages/node/native/scripts/build_llama_worker.sh`
- `tests/test_llama_worker_build.py`
## Exact commands and real results
### Native smoke build against a fake pinned checkout
```bash
tmpdir=$(mktemp -d)
mkdir -p "$tmpdir/llama.cpp"
printf 'MIT\n' > "$tmpdir/llama.cpp/LICENSE"
printf 'AUTHORS\n' > "$tmpdir/llama.cpp/AUTHORS"
printf '# placeholder\n' > "$tmpdir/llama.cpp/CMakeLists.txt"
printf '%s\n' 'b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac' > "$tmpdir/llama.cpp/.meshnet-upstream-commit"
git init -q "$tmpdir/llama.cpp"
packages/node/native/scripts/build_llama_worker.sh \
--source-dir "$tmpdir/llama.cpp" \
--build-dir "$tmpdir/build"
```
Result:
- `meshnet worker scaffold ok`
- `upstream commit: b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac`
- `patchset version: 0001`
- `build ok: /tmp/.../build/meshnet_worker`
### Targeted pytest
```bash
python -m pytest -q tests/test_llama_worker_build.py
```
Result: `1 passed in 0.53s`
### Python compile check
```bash
python -m compileall -q packages tests
```
Result: exit 0
### Diff hygiene
```bash
git diff --check
```
Result: exit 0
### Full deterministic pytest
```bash
python -m pytest -q
```
Result: `424 passed, 13 skipped, 210 failed, 86 errors in 131.04s`
The failures are pre-existing sandbox socket failures in tracker/HTTP-backed
tests. Representative error:
- `PermissionError: [Errno 1] Operation not permitted` when the tracker tries
to bind a socket.
This matches the previously observed environment limitation in the DGR-002 and
DGR-003 evidence and is unrelated to the llama.cpp pin/build scaffold.
## Limitations
- The sandbox does not provide `cmake`, so the smoke build uses the available
direct C++ compiler path (`g++` here) instead of a CMake-generated target.
- The pinned upstream source was not fetched from GitHub during validation.
The script supports fetching the exact commit when network access is
available, but the validation run used a fake local checkout to keep the test
deterministic and model-free.
- The patch stack in this story is deliberately narrow and additive. It creates
a worker scaffold and build seam, not the final llama.cpp runtime patches.
## Compatibility notes
- The exact upstream pin is `b3c9d1b846cc80a6360adb6aeaa4fcd8c4c8dcac`.
- The build script fails closed if the checkout pin differs from that commit or
if the expected upstream files (`LICENSE`, `AUTHORS`, `CMakeLists.txt`) are
missing.
- The patch stack is isolated from Meshnet networking code and can be applied
to a clean pinned checkout before later worker stories extend the scaffold.
- Upstream attribution notices are preserved in the build output by copying the
staged `LICENSE` and `AUTHORS` files into `build/.../upstream-notices/`.
## Dependent-story handoff
- DGR-008 can replace the scaffold source with the real supervised C++ worker
while keeping the same pin metadata, patch stack, and build script boundary.
- DGR-005 and later native stories should keep using the same exact pin so the
worker seam remains reproducible while range-loading and session logic are
added.