feat: checkpoint batching and release-gate stories
This commit is contained in:
@@ -0,0 +1,43 @@
|
||||
# DGR-016 outreach draft
|
||||
|
||||
Subject: Narrow llama.cpp hooks for range loading, boundary I/O, and local KV ownership
|
||||
|
||||
Hi Georgi and llama.cpp maintainers,
|
||||
|
||||
We have been building a distributed GGUF route on top of a Meshnet control
|
||||
plane, and the narrow upstreamable seam is now clear enough to summarize.
|
||||
|
||||
We are not asking llama.cpp to own Meshnet routing, billing, relay transport,
|
||||
or any volunteer-network policy. The upstream ask is limited to generic local
|
||||
hooks that make partitioned inference easier to implement and easier to review:
|
||||
|
||||
1. Range-aware loading and ownership introspection for contiguous layer ranges.
|
||||
2. Architecture-defined boundary input/output using an explicit named-tensor
|
||||
bundle.
|
||||
3. Layer-filtered KV ownership and stable local sequence mapping.
|
||||
|
||||
Why we think this is generally useful:
|
||||
|
||||
- Nakshatra already demonstrates the value of a narrow layer-worker seam and
|
||||
partial GGUF loading.
|
||||
- prima.cpp shows the same idea from a different angle with selective loading,
|
||||
local KV, and boundary residual transport.
|
||||
- Both projects suggest the same conclusion: the missing API is not Meshnet
|
||||
specific, it is a local runtime seam that any layer-partitioned supervisor can
|
||||
use.
|
||||
|
||||
The package we would upstream is intentionally split into one concern per patch
|
||||
so review stays small:
|
||||
|
||||
- range-aware loading and tensor ownership;
|
||||
- boundary I/O for intermediate residual state;
|
||||
- layer-filtered KV and sequence ownership.
|
||||
|
||||
If useful, we can send the concrete MRE/test mapping next. We already have
|
||||
deterministic examples covering the loader, boundary contract, and KV/session
|
||||
semantics in the Meshnet tree, and we can trim them into upstream-focused test
|
||||
cases.
|
||||
|
||||
Thanks,
|
||||
Meshnet maintainers
|
||||
|
||||
Reference in New Issue
Block a user