feature-gguf-distributed

CONTEXT.md

@@ -18,23 +18,51 @@ _Avoid_: partition, slice, chunk, segment
 The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard.
 _Avoid_: torrent, cluster, pool
 
-**Inference Route**:
-An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
-_Avoid_: pipeline, chain, path
-
-**Gateway**:
-The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results back.
-_Avoid_: proxy, relay, orchestrator, primary
-
-### Tracker
+**Inference Route**:
+An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
+_Avoid_: pipeline, chain, path
+
+**Route Session**:
+An active inference request bound to one Inference Route and one stable session id while the request is being served.
+_Avoid_: conversation, job, token stream
+
+**Activation Seam**:
+The boundary between two adjacent shard executions where hidden states pass from one node to the next.
+_Avoid_: handoff, layer gap, boundary hop
+
+**Hot KV State**:
+The live attention/cache state a node holds for its own shard during a Route Session.
+_Avoid_: centralized KV cache, global cache, remote cache
+
+**Prefix Snapshot**:
+A persisted copy of reusable Route Session state for a prompt prefix, used for reuse, retry, or failover.
+_Avoid_: hot cache, CDN cache, active KV
+
+**Model Artifact**:
+A versioned model file or tokenizer file that nodes download, verify, and keep locally to serve a Model Preset.
+_Avoid_: model blob, weights dump, asset
+
+**Artifact Manifest**:
+The canonical record that identifies the Model Artifacts, their integrity checks, and the model parts they support.
+_Avoid_: torrent file, metadata JSON, download list
+
+**Gateway**:
+The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results and progress to the client when possible.
+_Avoid_: proxy, relay, orchestrator, primary
+
+**Generation Telemetry**:
+Realtime progress information for an active Route Session, including phase, generated token count, and tokens-per-second speed.
+_Avoid_: logs, debug output
+
+### Tracker
 
 **Tracker**:
 The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback.
 _Avoid_: coordinator, scheduler, director
 
-**Tracker Node**:
-A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results back. Any node advertising a new model to the network becomes its tracker node.
-_Avoid_: primary node, master node, gateway node
+**Tracker Node**:
+A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results and progress when possible. Any node advertising a new model to the network becomes its tracker node.
+_Avoid_: primary node, master node, gateway node
 
 **Coverage Map**:
 The tracker's per-model mapping of layer ranges to node counts: `[(start_layer, end_layer, node_count), ...]`. A layer range with `node_count=0` is a coverage gap — the model is unroutable until the gap is filled. Coverage-first bin-packing fills all gaps before adding redundancy.