docs: add ADRs and user stories for real model inference stack (US-011–014)

ADR-0008: binary activation wire format — raw bfloat16 over HTTP, zstd compression,
128-token chunked prefill; replaces base64 JSON (~33% overhead removed).

ADR-0009: coverage-first shard assignment and tracker-as-first-layer-node —
any node serving layers[0..k] becomes the inference entry point for that model;
bin-packing fills all coverage gaps before adding redundancy; tracker issues
LOAD_SHARD/DROP_SHARD rebalance directives; nodes declare VRAM + quantization.

US-011: binary wire format migration
US-012: real PyTorch layer execution (transformers + bitsandbytes, test on GPT-2)
US-013: coverage-first tracker bin-packing with VRAM-aware shard assignment
US-014: tracker-as-node (tracker node serves first layers + handles client requests)

CONTEXT.md: Tracker Node, Coverage Map, Rebalance Directive terms added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs/adr/0008-binary-activation-wire-format.md

@@ -0,0 +1,48 @@
+# Binary activation wire format with zstd compression and chunked prefill
+
+Activation tensors between shard nodes are sent as raw binary (little-endian bfloat16 bytes) over HTTP with metadata in request headers and optional zstd body compression. The previous base64-encoded JSON format is retired.
+
+## Problem
+
+Base64 adds 33% size overhead plus CPU cost on both ends. For a 700B model at bfloat16 (hidden_dim=16384), a 2048-token prefill generates 64MB of activation data per shard boundary. At 10 boundaries that is 640MB of transfer just for prefill. Encoding that as base64 JSON makes it ~860MB plus repeated parse overhead.
+
+## Decision
+
+**Control plane** (registration, heartbeat, route selection, health) stays JSON over HTTP — these are small and infrequent.
+
+**Activation plane** (node-to-node tensor transfer) uses a binary HTTP body:
+
+```
+POST /forward HTTP/1.1
+Content-Type: application/octet-stream
+X-Meshnet-Shape: 1,128,16384
+X-Meshnet-Dtype: bfloat16
+X-Meshnet-Session: <uuid>
+X-Meshnet-Chunk-Index: 0
+X-Meshnet-Chunk-Total: 16
+X-Meshnet-Encoding: zstd
+Content-Length: <compressed_byte_count>
+
+<raw zstd-compressed little-endian bfloat16 tensor bytes>
+```
+
+**Chunked prefill**: long prompts are split into 128-token chunks before the first forward pass. Each chunk is sent and forwarded independently through the pipeline. This keeps peak transfer per boundary at ~4MB (128 × 16384 × 2 bytes) rather than 64MB+ for a full 2048-token prefill. Generation (one token at a time) is already naturally chunked.
+
+**Compression**: zstd level 1 (near-lz4 speed, better ratio). bfloat16 hidden states post-LayerNorm have high compressibility (many near-zero values). Expected 2–4× reduction on typical activations. `X-Meshnet-Encoding` is optional — omit for uncompressed binary; receiving nodes must handle both.
+
+**Activation dtype**: always bfloat16 at every shard boundary, regardless of the weight quantization a node uses internally. Weights may be NF4 or INT8; computation upcasts to bfloat16; the result crosses the wire in bfloat16. This avoids dtype negotiation between nodes with different quantization tiers.
+
+## Considered Options
+
+- **base64 JSON**: existing format, works everywhere, ~33% overhead, no streaming, high CPU cost — retired
+- **msgpack with embedded binary**: compact metadata + binary blob, requires msgpack dep on all nodes — rejected (HTTP headers are sufficient for metadata)
+- **Raw TCP with length-prefixed frames**: maximally efficient, no HTTP overhead — rejected (breaks existing HTTP server infrastructure and complicates firewall traversal)
+- **Binary HTTP body with headers**: chosen — no new protocol, standard HTTP chunked transfer, works with existing server framework, easy to debug with curl
+
+## Consequences
+
+- Node forward endpoint changes from `POST /forward` with JSON body to `POST /forward` with binary body and header metadata
+- All nodes must be updated together (wire format version mismatch = 400 error with `X-Meshnet-Wire-Version` header for debugging)
+- The stub wire format (`base64 JSON dict`) used in US-001 through US-010 is replaced; stub nodes can emit zeroed binary tensors trivially
+- Chunk boundaries in prefill must align to token positions (no mid-token splits)
+- KV cache invalidation across chunks is the calling node's responsibility; the receiving node treats each chunk as an independent forward pass segment

docs/adr/0009-coverage-first-shard-assignment-and-tracker-as-node.md

@@ -0,0 +1,85 @@
+# Coverage-first shard assignment and tracker-as-first-layer-node
+
+The tracker assigns shard ranges to nodes using a coverage-first, speed-weighted bin-packing algorithm. Tracker nodes must host at least the first layer shard of every model they coordinate, making them the natural inference entry point. Any node serving layers[0..k] can become a tracker node for that model.
+
+## Problem
+
+A 700B model sharded across volunteer nodes of wildly different VRAM (8GB to 128GB+) and different quantization levels (NF4 / INT8 / bfloat16) must be fully covered at all times for inference to work at all. 199 copies of 99% of a model are worthless if the last 1% of layers has zero coverage. Replication only matters after full coverage is achieved.
+
+## Decision
+
+### Coverage-first bin-packing
+
+The tracker maintains a **coverage map**: a sorted list of (start_layer, end_layer, node_count) for each model. A layer range with node_count=0 is a coverage gap — the model is unservable until the gap is filled. The tracker's shard assignment algorithm:
+
+1. Identify uncovered layer ranges (gaps) first
+2. Assign new or idle nodes to fill gaps, largest VRAM-fit first
+3. Once fully covered, assign additional nodes to the most-replicated-needed ranges (latency hotspots) before adding redundancy elsewhere
+4. Speed-weight: give faster nodes wider shard ranges — fill their VRAM completely, then assign the remainder to the next-fastest available node
+
+Example: 700B NF4 model (~350GB weights). Node A has 128GB, Node B and C each have 80GB and are idle.
+- Node A gets layers[0..k_a] filling ~120GB (reserve 8GB for KV cache)
+- Node B gets layers[k_a..k_b] filling ~72GB
+- Node C gets layers[k_b..N] (the remainder)
+- If Node B benchmarks 2× faster than Node C, the tracker shifts the B/C boundary so B carries more layers
+
+### Tracker-as-first-layer-node
+
+Any node that advertises a new model to the network becomes a **tracker node** for that model. Tracker nodes have one hard requirement: they must hold and serve `layers[0..k]` (the first-layer shard) for every model they coordinate.
+
+The reason is functional: a tracker node is also the inference entry point. When a client request arrives, the tracker node:
+1. Tokenizes the input (owns the tokenizer)
+2. Runs `model.embed_tokens` + `model.layers[0..k]`
+3. Selects the optimal onward route from the coverage map
+4. Forwards activations to the next node in the route
+5. Receives the final hidden state back and streams tokens to the client
+
+This collapses the separate "gateway" role: the tracker node that starts an inference request IS the gateway for that request. A standalone HTTP proxy/load-balancer may sit in front to pick which tracker node handles the request, but it carries no model weights.
+
+Multiple tracker nodes for the same model = multiple entry points = horizontal scale for both routing decisions and the first-layer compute.
+
+### Last-layer node (tail)
+
+The node assigned `layers[N-k..N]` also runs `model.norm` and `model.lm_head`. It returns decoded token IDs (not hidden states) to the tracker node, which assembles the response. The tail shard assignment is marked `is_tail: true` in the shard registry.
+
+### Adaptive quantization
+
+Nodes declare their capabilities at registration:
+```json
+{
+  "vram_bytes": 25769803776,
+  "quantizations": ["nf4", "int8", "bfloat16"],
+  "benchmark_tokens_per_sec": 12.4,
+  "benchmark_model": "meta-llama/Llama-3-8B"
+}
+```
+
+The tracker uses declared VRAM + quantization to compute max assignable layers:
+```
+max_layers = floor(available_vram / bytes_per_layer_at_declared_quant)
+```
+where `bytes_per_layer_at_declared_quant` is a per-model constant stored in the model preset metadata. KV cache headroom (configurable, default 20% of VRAM) is subtracted before the calculation.
+
+A node may serve different models at different quantizations. The tracker tracks (model, quantization, shard_range) triples per node.
+
+### Continuous rebalancing
+
+The tracker monitors coverage every 30 seconds and on every node join/leave event. When a coverage gap appears (node drops offline), the tracker issues **rebalance directives** to idle nodes or nodes with redundant coverage:
+- `LOAD_SHARD(model, start_layer, end_layer, quantization)` — download and begin serving
+- `DROP_SHARD(model, start_layer, end_layer)` — stop serving (safe only when coverage > 1 for that range)
+
+Nodes obey directives asynchronously; the tracker waits up to a configurable timeout before marking the gap critical.
+
+## Considered Options
+
+- **Static shard assignment** (assign once at registration): simple, breaks when nodes leave — rejected
+- **Replication-first** (maximize redundancy before optimizing speed): wastes resources on popular shards, leaves rare shards uncovered — rejected (user: "we have no use of 199 copies of 99% of the model")
+- **Coverage-first, speed-weighted bin-packing with continuous rebalancing**: chosen
+
+## Consequences
+
+- The standalone `meshnet-gateway` service from US-005 becomes a thin load-balancer that routes to tracker nodes; tracker nodes do the actual inference orchestration
+- Tracker nodes must download more model data (tokenizer + first-layer shard) — this is the price of being an entry point
+- Benchmark data is self-reported by nodes at registration; the validator can detect fraudulent benchmarks (a node claiming 100 tokens/sec but delivering 2 gets slashed for under-performance)
+- VRAM reservation for KV cache means nodes can host fewer layers than their raw VRAM suggests — this is intentional; running out of KV cache during inference causes OOM crashes
+- New CONTEXT.md terms: **Tracker Node** (node serving first-layer shard + inference routing for a model), **Coverage Map** (tracker's per-model layer-range → node-count mapping), **Rebalance Directive** (tracker instruction to a node to load or drop a shard)