# Performant concurrent distributed GGUF runtime

Status: active benchmark-gated implementation program.

## Objective

Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.

## Critical path

```text
Meshnet control plane
  -> versioned gRPC/Protobuf Shard protocol
  -> project-owned standalone C++ worker
  -> small pinned llama.cpp patch stack
```

Transformers/safetensors remains the correctness baseline. vLLM remains an optional complete managed provider and a design donor; it is not forked into the public mesh.

## Planning artifacts

- **[Mandatory Ralph context](RALPH-CONTEXT.md)** — read first in every fresh iteration
- [Task evidence contract](evidence/README.md)
- [Implementation strategy](implementation-strategy.md)
- [Current architecture](architecture.md)
- [PRD](PRD.md)
- [Ralph backlog](prd.json)
- [ADR-0020](ADR-0020-distributed-gguf-runtime.md)
- [Milestones](milestones.md)
- [Issues](issues/)
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
- [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md)
- [vLLM assessment](../../docs/research/vllm-distributed-gguf-assessment.md)

## Ralph execution

Use supervised one-story iterations for this high-risk runtime:

```bash
ralph-tui run \
  --prd .scratch/distributed-gguf-runtime/prd.json \
  --agent claude --model opus \
  --iterations 1 --no-tui --no-setup --verify
```

Inspect the diff, run the story gates, and commit one verified story before the next iteration. Real-model stories require the explicit environment gate and mounted-drive model storage.