[verified] feat: harden llama.cpp provenance workspace
This commit is contained in:
72
.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md
Normal file
72
.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# DGR-027 evidence — exact llama.cpp provenance manifest and fetch workspace
|
||||
|
||||
**Completed implementation:** 2026-07-17
|
||||
**Branch:** `ralph/dgr-small-terra`
|
||||
**Authority:** live Gitea issue #11. The controller fetched and claimed the issue
|
||||
through the Gitea API before launch; the isolated agent received that exact body.
|
||||
|
||||
## Changed files
|
||||
|
||||
- `packages/node/native/llama/UPSTREAM_LOCK.json`
|
||||
- `packages/node/native/llama/PATCH-STACK.md`
|
||||
- `scripts/llama_cpp_dependency.py`
|
||||
- `tests/test_llama_cpp_dependency.py`
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`
|
||||
|
||||
## Provenance and retrieval contract
|
||||
|
||||
`UPSTREAM_LOCK.json` records the upstream Git URL, immutable 40-character
|
||||
commit `e920c523e3b8a0163fe498af5bf90df35ff51d25`, expected Git tree
|
||||
`6c91a11407a3a3fb160f5dac705f9c59718f54f1`, MIT license, and the sole
|
||||
retrieval method: `git-clone-detached-commit` into `build/llama.cpp/source`.
|
||||
|
||||
`python3 scripts/llama_cpp_dependency.py fetch` has no branch, tag, ref, or
|
||||
repository override. On a first fetch it clones the manifest URL, checks out
|
||||
the detached commit, and verifies commit, tree, required upstream blobs,
|
||||
license, and cleanliness. If the workspace already exists, it makes no network
|
||||
request and accepts it only after the same verification. Dirty or mismatched
|
||||
caches fail closed. The build directory is already ignored by `.gitignore`.
|
||||
|
||||
## Verification
|
||||
|
||||
| Command | Result |
|
||||
| --- | --- |
|
||||
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py` | `7 passed in 0.22s` |
|
||||
| `python3 -m compileall packages tests` | passed |
|
||||
| `git diff --check` | passed (no output) |
|
||||
| `python3 scripts/llama_cpp_dependency.py inspect` | passed; reports exact commit/tree, retrieval workspace, MIT license, and two-patch stack |
|
||||
| `python3 scripts/llama_cpp_dependency.py fetch --workspace /tmp/not-llama-workspace` | failed closed with status 2: workspace outside the locked ignored build root |
|
||||
| symlinked workspace regression | passed; both a `build/` ancestor symlink and a final `source` symlink escaping the repository are refused |
|
||||
| attached-branch cache regression | passed; an exact commit on a local branch is refused until checked out as detached HEAD |
|
||||
| ignored/excluded injection regression | passed; a file hidden by `.git/info/exclude` is detected and refused |
|
||||
| tracked injection regression | passed; modified tracked content hidden by both `assume-unchanged` and `skip-worktree` is content-hashed and refused |
|
||||
| executable-mode regression | passed on the POSIX fixture for both index flags; the mounted project workspace has `core.filemode=false`, so its exact index tree is the canonical mode record and physical mode bits are not treated as meaningful |
|
||||
| `git check-ignore -v build/llama.cpp/source` | passed; `.gitignore:6:build/` |
|
||||
| `git diff --summary` and `git ls-files build packages/node/native/llama` | no source checkout or new submodule introduced; only manifest/docs/patches/native wrapper are tracked |
|
||||
| `python3 scripts/llama_cpp_dependency.py fetch` (controller network lane) | passed; fetched the exact detached commit and verified HEAD `e920c523e3b8a0163fe498af5bf90df35ff51d25` and tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1` in the ignored workspace |
|
||||
| `python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source` | failed on the pre-existing `0002-dense-llama-owned-range-loader.patch` as a corrupt patch at line 26; this is an explicit DGR-028 blocker and no native-build claim is made |
|
||||
|
||||
The targeted test suite creates a local Git fixture to prove offline cache reuse
|
||||
after full identity verification, then proves a dirty cache is rejected. It
|
||||
also proves the CLI rejects a repository/branch override and an arbitrary
|
||||
workspace.
|
||||
|
||||
## Limitations
|
||||
|
||||
- The controller successfully materialized and verified the exact upstream
|
||||
commit/tree, so the DGR-027 fetch and offline-cache boundary has real upstream
|
||||
evidence rather than fixture-only evidence.
|
||||
- The existing `0002-dense-llama-owned-range-loader.patch` is malformed and
|
||||
cannot pass `git apply --check` against the exact pin. DGR-027 changes no patch
|
||||
file; repairing and certifying the numbered patch stack belongs to DGR-028.
|
||||
Until that story closes, the repository must not claim patched-tree, native
|
||||
CMake/CTest, or reverse-apply certification.
|
||||
- No model, API credits, GPU, or model artifact storage was used.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
DGR-028, DGR-029, and DGR-044 must invoke the manifest-owned `fetch` command
|
||||
before touching llama.cpp source. They may use only the verified
|
||||
`build/llama.cpp/source` checkout and must record any native build, CTest, and
|
||||
patch apply/check/reverse evidence against the exact manifest pin. DGR-017's
|
||||
cleanup remains provenance only and grants no inherited completion credit.
|
||||
@@ -673,8 +673,8 @@
|
||||
"Offline reuse is supported only after the cached tree’s exact identity is verified.",
|
||||
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
|
||||
],
|
||||
"passes": false,
|
||||
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md; prd.json is authoritative.",
|
||||
"passes": true,
|
||||
"notes": "Completed from Gitea #11 via the Terra Ralph lane; independently reviewed after fail-closed cache identity hardening. Verified 7 focused tests, exact real-cache reuse, compileall, and diff-check. The pre-existing malformed 0002 patch is handed to DGR-028.",
|
||||
"blocks": [
|
||||
"DGR-028",
|
||||
"DGR-029",
|
||||
|
||||
@@ -15,11 +15,19 @@ loading, endpoint ownership, architecture-defined intermediate boundaries, and
|
||||
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
|
||||
billing, authentication, and telemetry must remain outside this directory.
|
||||
|
||||
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline
|
||||
blobs, validates every patch digest and context with `git apply --check`, then
|
||||
applies the series in `patches/series` order. It refuses a dirty source tree,
|
||||
wrong commit/tree/blob, changed patch digest, reordered series, or an existing
|
||||
destination/work directory.
|
||||
`scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out
|
||||
only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored
|
||||
build workspace. It verifies the exact commit/tree and baseline blobs before use. A
|
||||
later offline `fetch` may reuse that cache only after the same clean identity
|
||||
verification; an attached branch, tag/repository override, arbitrary destination,
|
||||
symlinked workspace, dirty checkout, ignored injected file, or tracked-file
|
||||
modification hidden by Git index flags is refused. The tool
|
||||
validates every patch digest and context
|
||||
with `git apply --check`, then applies the series in `patches/series` order.
|
||||
Tracked executable modes are checked physically when Git reports
|
||||
`core.filemode=true`; on mounted filesystems without POSIX mode fidelity, the
|
||||
locked index tree remains the canonical mode record while every working-file
|
||||
blob is independently re-hashed.
|
||||
|
||||
## Current semantic boundary
|
||||
|
||||
|
||||
@@ -3,6 +3,13 @@
|
||||
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||
"expected_source": {
|
||||
"git_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1"
|
||||
},
|
||||
"retrieval": {
|
||||
"method": "git-clone-detached-commit",
|
||||
"workspace": "build/llama.cpp"
|
||||
},
|
||||
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
|
||||
"upstream_license": "MIT",
|
||||
"patch_series": [
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Materialize, verify, build, and smoke-test DGR-004's llama.cpp pin.
|
||||
"""Fetch, verify, build, and smoke-test DGR-027's exact llama.cpp pin.
|
||||
|
||||
This tool deliberately owns only a source dependency boundary. It never
|
||||
downloads a model, invokes inference, or interprets generated text.
|
||||
@@ -12,6 +12,7 @@ import hashlib
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -67,14 +68,27 @@ def _load_lock() -> dict[str, Any]:
|
||||
raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error
|
||||
required = {
|
||||
"upstream", "commit", "commit_tree", "patched_tree", "patch_series",
|
||||
"required_upstream_blobs", "patched_paths", "build",
|
||||
"required_upstream_blobs", "patched_paths", "build", "upstream_license",
|
||||
"expected_source", "retrieval",
|
||||
}
|
||||
missing = sorted(required - lock.keys())
|
||||
if missing:
|
||||
raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}")
|
||||
commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
|
||||
if lock["commit"] != commit_file or len(commit_file) != 40:
|
||||
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on a full commit")
|
||||
object_ids = [commit_file, lock["commit"], lock["commit_tree"], lock["patched_tree"]]
|
||||
if lock["commit"] != commit_file or not all(
|
||||
isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value)
|
||||
for value in object_ids
|
||||
):
|
||||
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on full hexadecimal object IDs")
|
||||
if lock["expected_source"] != {"git_tree": lock["commit_tree"]}:
|
||||
raise DependencyError("expected_source must record the locked git tree")
|
||||
retrieval = lock["retrieval"]
|
||||
if retrieval != {
|
||||
"method": "git-clone-detached-commit",
|
||||
"workspace": "build/llama.cpp",
|
||||
}:
|
||||
raise DependencyError("retrieval must use the locked detached-commit build workspace")
|
||||
return lock
|
||||
|
||||
|
||||
@@ -103,6 +117,56 @@ def _git(source: pathlib.Path, *args: str) -> str:
|
||||
return _run("git", "-C", str(source), *args)
|
||||
|
||||
|
||||
def _verify_tracked_content(source: pathlib.Path, lock: dict[str, Any]) -> None:
|
||||
if _git(source, "write-tree") != lock["commit_tree"]:
|
||||
raise DependencyError("materialized checkout index differs from the locked tree")
|
||||
filemode_trusted = _git(source, "config", "--bool", "core.filemode") == "true"
|
||||
records = _git(source, "ls-files", "-s", "-z").split("\0")
|
||||
paths: list[str] = []
|
||||
expected: list[str] = []
|
||||
for record in records:
|
||||
if not record:
|
||||
continue
|
||||
metadata, path = record.split("\t", 1)
|
||||
mode, blob, stage = metadata.split()
|
||||
if stage != "0" or mode not in {"100644", "100755", "120000"}:
|
||||
raise DependencyError(f"unsupported tracked entry in materialized checkout: {record!r}")
|
||||
if "\n" in path:
|
||||
raise DependencyError(f"newline-bearing tracked path is unsupported: {path!r}")
|
||||
candidate = source / path
|
||||
cursor = source
|
||||
for part in pathlib.Path(path).parts[:-1]:
|
||||
cursor = cursor / part
|
||||
if cursor.is_symlink():
|
||||
raise DependencyError(f"tracked path traverses a symlink: {path!r}")
|
||||
if mode == "120000":
|
||||
if not candidate.is_symlink():
|
||||
raise DependencyError(f"tracked symlink type differs from the locked tree: {path!r}")
|
||||
else:
|
||||
if candidate.is_symlink() or not candidate.is_file():
|
||||
raise DependencyError(f"tracked file type differs from the locked tree: {path!r}")
|
||||
executable = bool(candidate.stat().st_mode & 0o111)
|
||||
if filemode_trusted and executable != (mode == "100755"):
|
||||
raise DependencyError(f"tracked executable mode differs from the locked tree: {path!r}")
|
||||
paths.append(path)
|
||||
expected.append(blob)
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
["git", "hash-object", "--stdin-paths"],
|
||||
cwd=source,
|
||||
input="".join(f"{path}\n" for path in paths),
|
||||
check=True,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
except subprocess.CalledProcessError as error:
|
||||
raise DependencyError(f"unable to hash tracked source content: {error.stderr.strip()}") from error
|
||||
actual = completed.stdout.splitlines()
|
||||
if len(actual) != len(expected) or actual != expected:
|
||||
raise DependencyError("tracked source content differs from the locked tree")
|
||||
|
||||
|
||||
def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None:
|
||||
if not (source / ".git").exists():
|
||||
raise DependencyError(f"not a materialized git checkout: {source}")
|
||||
@@ -110,8 +174,15 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
|
||||
raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit")
|
||||
if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]:
|
||||
raise DependencyError("upstream drift: checkout tree does not equal the locked tree")
|
||||
if require_clean and _git(source, "status", "--porcelain"):
|
||||
raise DependencyError("local edits detected in materialized llama.cpp checkout")
|
||||
if _git(source, "rev-parse", "--abbrev-ref", "HEAD") != "HEAD":
|
||||
raise DependencyError("materialized llama.cpp checkout must have a detached HEAD")
|
||||
if require_clean:
|
||||
_verify_tracked_content(source, lock)
|
||||
tracked = _git(source, "status", "--porcelain", "--untracked-files=no")
|
||||
untracked = _git(source, "ls-files", "--others", "--exclude-standard")
|
||||
ignored = _git(source, "ls-files", "--others", "--ignored", "--exclude-standard")
|
||||
if tracked or untracked or ignored:
|
||||
raise DependencyError("local edits or unmanifested files detected in materialized llama.cpp checkout")
|
||||
for relative, expected in lock["required_upstream_blobs"].items():
|
||||
actual = _git(source, "rev-parse", f"HEAD:{relative}")
|
||||
if actual != expected:
|
||||
@@ -122,15 +193,43 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
|
||||
raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution")
|
||||
|
||||
|
||||
def materialize(source: pathlib.Path, repository: str) -> None:
|
||||
def _workspace_source(workspace: pathlib.Path, lock: dict[str, Any]) -> pathlib.Path:
|
||||
relative = pathlib.Path(lock["retrieval"]["workspace"])
|
||||
expected = (ROOT / relative).absolute()
|
||||
supplied = workspace.absolute()
|
||||
if supplied != expected:
|
||||
raise DependencyError(f"--workspace must equal the locked ignored build root: {expected}")
|
||||
cursor = ROOT
|
||||
for part in relative.parts:
|
||||
cursor = cursor / part
|
||||
if cursor.is_symlink():
|
||||
raise DependencyError(f"locked build workspace may not traverse a symlink: {cursor}")
|
||||
resolved = workspace.resolve()
|
||||
try:
|
||||
resolved.relative_to(ROOT.resolve())
|
||||
except ValueError as error:
|
||||
raise DependencyError("locked build workspace escapes the repository root") from error
|
||||
source = resolved / "source"
|
||||
if source.is_symlink():
|
||||
raise DependencyError(f"locked source checkout may not be a symlink: {source}")
|
||||
return source
|
||||
|
||||
|
||||
def fetch(workspace: pathlib.Path) -> pathlib.Path:
|
||||
"""Fetch once, or verify an exact clean cached checkout for offline reuse."""
|
||||
lock = _load_lock()
|
||||
_patches(lock)
|
||||
source = _workspace_source(workspace, lock)
|
||||
if source.exists():
|
||||
raise DependencyError(f"destination already exists; refusing to reuse possibly edited source: {source}")
|
||||
_verify_source(source, lock, require_clean=True)
|
||||
print(f"reused verified offline cache: {source}")
|
||||
return source
|
||||
source.parent.mkdir(parents=True, exist_ok=True)
|
||||
_run("git", "clone", "--no-checkout", repository, str(source))
|
||||
_run("git", "clone", "--no-checkout", lock["upstream"], str(source))
|
||||
_git(source, "checkout", "--detach", lock["commit"])
|
||||
_verify_source(source, lock, require_clean=True)
|
||||
print(f"fetched and verified exact source: {source}")
|
||||
return source
|
||||
|
||||
|
||||
def apply(source: pathlib.Path) -> None:
|
||||
@@ -198,18 +297,11 @@ def smoke(binary: pathlib.Path) -> None:
|
||||
print(output)
|
||||
|
||||
|
||||
def reproduce(work_dir: pathlib.Path, repository: str) -> None:
|
||||
resolved = work_dir.resolve()
|
||||
build_root = (ROOT / "build").resolve()
|
||||
if build_root not in resolved.parents:
|
||||
raise DependencyError(f"--work-dir must be below {build_root}: {resolved}")
|
||||
if resolved.exists():
|
||||
raise DependencyError(
|
||||
f"work directory already exists; refusing to erase possible local edits: {resolved}"
|
||||
)
|
||||
source = resolved / "source"
|
||||
build_dir = resolved / "build"
|
||||
materialize(source, repository)
|
||||
def reproduce(workspace: pathlib.Path) -> None:
|
||||
source = fetch(workspace)
|
||||
build_dir = workspace.resolve() / "build"
|
||||
if build_dir.exists():
|
||||
raise DependencyError(f"build directory already exists; refusing to erase possible local edits: {build_dir}")
|
||||
apply(source)
|
||||
smoke(build(source, build_dir))
|
||||
|
||||
@@ -219,6 +311,9 @@ def inspect() -> None:
|
||||
patches = _patches(lock)
|
||||
print(json.dumps({
|
||||
"commit": lock["commit"],
|
||||
"commit_tree": lock["commit_tree"],
|
||||
"retrieval": lock["retrieval"],
|
||||
"upstream_license": lock["upstream_license"],
|
||||
"patch_count": len(patches),
|
||||
"patches": [patch.name for patch in patches],
|
||||
"model_downloads": False,
|
||||
@@ -231,9 +326,8 @@ def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
subcommands = parser.add_subparsers(dest="command", required=True)
|
||||
subcommands.add_parser("inspect")
|
||||
materialize_parser = subcommands.add_parser("materialize")
|
||||
materialize_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
|
||||
materialize_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
|
||||
fetch_parser = subcommands.add_parser("fetch")
|
||||
fetch_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
|
||||
apply_parser = subcommands.add_parser("apply")
|
||||
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
|
||||
build_parser = subcommands.add_parser("build")
|
||||
@@ -242,14 +336,13 @@ def main() -> int:
|
||||
smoke_parser = subcommands.add_parser("smoke")
|
||||
smoke_parser.add_argument("--binary", type=pathlib.Path, required=True)
|
||||
reproduce_parser = subcommands.add_parser("reproduce")
|
||||
reproduce_parser.add_argument("--work-dir", type=pathlib.Path, default=ROOT / "build/dgr-004-smoke")
|
||||
reproduce_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
|
||||
reproduce_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
if args.command == "inspect":
|
||||
inspect()
|
||||
elif args.command == "materialize":
|
||||
materialize(args.source_dir, args.source_repository)
|
||||
elif args.command == "fetch":
|
||||
fetch(args.workspace)
|
||||
elif args.command == "apply":
|
||||
apply(args.source_dir)
|
||||
elif args.command == "build":
|
||||
@@ -257,9 +350,9 @@ def main() -> int:
|
||||
elif args.command == "smoke":
|
||||
smoke(args.binary)
|
||||
else:
|
||||
reproduce(args.work_dir, args.source_repository)
|
||||
reproduce(args.workspace)
|
||||
except DependencyError as error:
|
||||
print(f"DGR-004 dependency error: {error}", file=sys.stderr)
|
||||
print(f"DGR-027 dependency error: {error}", file=sys.stderr)
|
||||
return 2
|
||||
return 0
|
||||
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Offline guards for DGR-004's pinned llama.cpp dependency boundary."""
|
||||
"""Offline guards for DGR-027's pinned llama.cpp dependency boundary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import subprocess
|
||||
@@ -33,6 +34,10 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
|
||||
|
||||
assert commit == lock["commit"]
|
||||
assert len(commit) == 40
|
||||
assert lock["retrieval"]["method"] == "git-clone-detached-commit"
|
||||
assert lock["retrieval"]["workspace"] == "build/llama.cpp"
|
||||
assert lock["expected_source"]["git_tree"] == lock["commit_tree"]
|
||||
assert lock["upstream_license"] == "MIT"
|
||||
assert patches == lock["patch_series"]
|
||||
assert patches == sorted(patches)
|
||||
assert patches
|
||||
@@ -42,6 +47,188 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
|
||||
assert "Subject: [PATCH" in patch.read_text()
|
||||
|
||||
|
||||
def test_fetch_refuses_a_workspace_outside_the_ignored_build_root(tmp_path: pathlib.Path) -> None:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, str(SCRIPT), "fetch", "--workspace", str(tmp_path / "llama.cpp")],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
assert completed.returncode == 2
|
||||
assert "--workspace must equal" in completed.stderr
|
||||
|
||||
|
||||
def test_workspace_refuses_a_symlinked_build_ancestor(tmp_path: pathlib.Path, monkeypatch) -> None:
|
||||
spec = importlib.util.spec_from_file_location("llama_cpp_dependency_symlink", SCRIPT)
|
||||
assert spec and spec.loader
|
||||
dependency = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(dependency)
|
||||
root = tmp_path / "repo"
|
||||
outside = tmp_path / "outside"
|
||||
root.mkdir()
|
||||
outside.mkdir()
|
||||
(root / "build").symlink_to(outside, target_is_directory=True)
|
||||
monkeypatch.setattr(dependency, "ROOT", root)
|
||||
|
||||
try:
|
||||
dependency._workspace_source(
|
||||
root / "build/llama.cpp",
|
||||
{"retrieval": {"workspace": "build/llama.cpp"}},
|
||||
)
|
||||
except dependency.DependencyError as error:
|
||||
assert "may not traverse a symlink" in str(error)
|
||||
else:
|
||||
raise AssertionError("symlinked build ancestor must be refused")
|
||||
|
||||
(root / "build").unlink()
|
||||
workspace = root / "build/llama.cpp"
|
||||
workspace.mkdir(parents=True)
|
||||
(workspace / "source").symlink_to(outside, target_is_directory=True)
|
||||
try:
|
||||
dependency._workspace_source(
|
||||
workspace,
|
||||
{"retrieval": {"workspace": "build/llama.cpp"}},
|
||||
)
|
||||
except dependency.DependencyError as error:
|
||||
assert "source checkout may not be a symlink" in str(error)
|
||||
else:
|
||||
raise AssertionError("symlinked source checkout must be refused")
|
||||
|
||||
|
||||
def test_fetch_refuses_a_branch_or_repository_override() -> None:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, str(SCRIPT), "fetch", "--source-repository", "main"],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
assert completed.returncode == 2
|
||||
assert "unrecognized arguments" in completed.stderr
|
||||
|
||||
|
||||
def test_fetch_reuses_only_a_verified_cached_tree_offline(
|
||||
tmp_path: pathlib.Path, monkeypatch
|
||||
) -> None:
|
||||
spec = importlib.util.spec_from_file_location("llama_cpp_dependency", SCRIPT)
|
||||
assert spec and spec.loader
|
||||
dependency = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(dependency)
|
||||
|
||||
root = tmp_path / "repo"
|
||||
source = root / "build/llama.cpp/source"
|
||||
upstream = tmp_path / "upstream"
|
||||
upstream.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(upstream)], check=True)
|
||||
subprocess.run(["git", "-C", str(upstream), "config", "user.email", "test@example.invalid"], check=True)
|
||||
subprocess.run(["git", "-C", str(upstream), "config", "user.name", "test"], check=True)
|
||||
(upstream / "CMakeLists.txt").write_text("cmake_minimum_required(VERSION 3.14)\n")
|
||||
(upstream / "LICENSE").write_text("MIT\n")
|
||||
(upstream / "tool.sh").write_text("#!/bin/sh\nexit 0\n")
|
||||
(upstream / "tool.sh").chmod(0o755)
|
||||
subprocess.run(["git", "-C", str(upstream), "add", "."], check=True)
|
||||
subprocess.run(["git", "-C", str(upstream), "commit", "-qm", "fixture"], check=True)
|
||||
commit = subprocess.run(
|
||||
["git", "-C", str(upstream), "rev-parse", "HEAD"], check=True, capture_output=True, text=True
|
||||
).stdout.strip()
|
||||
tree = subprocess.run(
|
||||
["git", "-C", str(upstream), "rev-parse", "HEAD^{tree}"], check=True, capture_output=True, text=True
|
||||
).stdout.strip()
|
||||
blob = subprocess.run(
|
||||
["git", "-C", str(upstream), "rev-parse", "HEAD:CMakeLists.txt"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
source.parent.mkdir(parents=True)
|
||||
subprocess.run(["git", "clone", "-q", str(upstream), str(source)], check=True)
|
||||
|
||||
llama_dir = root / "packages/node/native/llama"
|
||||
patch_dir = llama_dir / "patches"
|
||||
patch_dir.mkdir(parents=True)
|
||||
(llama_dir / "UPSTREAM_COMMIT").write_text(f"{commit}\n")
|
||||
(patch_dir / "series").write_text("0001-fixture.patch\n")
|
||||
patch = patch_dir / "0001-fixture.patch"
|
||||
patch.write_text("fixture patch\n")
|
||||
(patch_dir / "SHA256SUMS").write_text(f"{_sha256(patch)} {patch.name}\n")
|
||||
(llama_dir / "UPSTREAM_LOCK.json").write_text(json.dumps({
|
||||
"upstream": str(upstream),
|
||||
"commit": commit,
|
||||
"commit_tree": tree,
|
||||
"expected_source": {"git_tree": tree},
|
||||
"retrieval": {"method": "git-clone-detached-commit", "workspace": "build/llama.cpp"},
|
||||
"patched_tree": tree,
|
||||
"patch_series": [patch.name],
|
||||
"required_upstream_blobs": {"CMakeLists.txt": blob},
|
||||
"patched_paths": [],
|
||||
"build": {},
|
||||
"upstream_license": "MIT",
|
||||
}))
|
||||
monkeypatch.setattr(dependency, "ROOT", root)
|
||||
monkeypatch.setattr(dependency, "LLAMA_DIR", llama_dir)
|
||||
monkeypatch.setattr(dependency, "LOCK_PATH", llama_dir / "UPSTREAM_LOCK.json")
|
||||
monkeypatch.setattr(dependency, "PATCH_DIR", patch_dir)
|
||||
|
||||
try:
|
||||
dependency.fetch(root / "build/llama.cpp")
|
||||
except dependency.DependencyError as error:
|
||||
assert "detached HEAD" in str(error)
|
||||
else:
|
||||
raise AssertionError("attached branch cache must be refused")
|
||||
subprocess.run(["git", "-C", str(source), "checkout", "--detach", "-q", commit], check=True)
|
||||
assert dependency.fetch(root / "build/llama.cpp") == source
|
||||
|
||||
tracked_path = source / "CMakeLists.txt"
|
||||
for flag, clear_flag in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
subprocess.run(["git", "-C", str(source), "update-index", flag, "CMakeLists.txt"], check=True)
|
||||
tracked_path.write_text("injected tracked build input\n")
|
||||
try:
|
||||
dependency.fetch(root / "build/llama.cpp")
|
||||
except dependency.DependencyError as error:
|
||||
assert "tracked source content differs" in str(error)
|
||||
else:
|
||||
raise AssertionError(f"tracked cache injection hidden by {flag} must be refused")
|
||||
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "CMakeLists.txt"], check=True)
|
||||
subprocess.run(["git", "-C", str(source), "checkout", "--", "CMakeLists.txt"], check=True)
|
||||
|
||||
executable_path = source / "tool.sh"
|
||||
for flag, clear_flag in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
subprocess.run(["git", "-C", str(source), "update-index", flag, "tool.sh"], check=True)
|
||||
executable_path.chmod(0o644)
|
||||
try:
|
||||
dependency.fetch(root / "build/llama.cpp")
|
||||
except dependency.DependencyError as error:
|
||||
assert "executable mode differs" in str(error)
|
||||
else:
|
||||
raise AssertionError(f"tracked mode change hidden by {flag} must be refused")
|
||||
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "tool.sh"], check=True)
|
||||
subprocess.run(["git", "-C", str(source), "checkout", "--", "tool.sh"], check=True)
|
||||
|
||||
(source / "untracked.txt").write_text("edited\n")
|
||||
try:
|
||||
dependency.fetch(root / "build/llama.cpp")
|
||||
except dependency.DependencyError as error:
|
||||
assert "local edits" in str(error)
|
||||
else:
|
||||
raise AssertionError("dirty cached source must be refused")
|
||||
(source / "untracked.txt").unlink()
|
||||
(source / ".git/info/exclude").write_text("injected.cmake\n")
|
||||
(source / "injected.cmake").write_text("unmanifested input\n")
|
||||
try:
|
||||
dependency.fetch(root / "build/llama.cpp")
|
||||
except dependency.DependencyError as error:
|
||||
assert "unmanifested files" in str(error)
|
||||
else:
|
||||
raise AssertionError("ignored cached source input must be refused")
|
||||
|
||||
|
||||
def test_dependency_script_reports_the_locked_boundary_without_network() -> None:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, str(SCRIPT), "inspect"],
|
||||
|
||||
Reference in New Issue
Block a user