neuron-tai/.scratch/distributed-gguf-runtime/issues/03-prefill-decode-wire-protocol.md

03 — Prefill/decode wire protocol

Status: ready-for-agent

Goal

Define and implement the activation protocol needed by both PyTorch and future GGUF backends.

Scope

• Add route/session lifecycle headers.
• Separate prefill from decode-step.
• Keep binary bfloat16 activation bodies.
• Preserve relay compatibility.
• Add route id and model artifact hash validation.

Draft Endpoints

• POST /sessions/{session_id}/prefill
• POST /sessions/{session_id}/decode-step
• DELETE /sessions/{session_id}
• GET /sessions/{session_id}/status

Acceptance

• Old /forward remains temporarily or fails with clear version message.
• Tests cover relay preservation of session headers.
• Decode-step payload is independent of prompt length.