30 lines
759 B
Markdown
30 lines
759 B
Markdown
# 03 — Prefill/decode wire protocol
|
|
|
|
Status: ready-for-agent
|
|
|
|
## Goal
|
|
|
|
Define and implement the activation protocol needed by both PyTorch and future GGUF backends.
|
|
|
|
## Scope
|
|
|
|
- Add route/session lifecycle headers.
|
|
- Separate `prefill` from `decode-step`.
|
|
- Keep binary bfloat16 activation bodies.
|
|
- Preserve relay compatibility.
|
|
- Add route id and model artifact hash validation.
|
|
|
|
## Draft Endpoints
|
|
|
|
- `POST /sessions/{session_id}/prefill`
|
|
- `POST /sessions/{session_id}/decode-step`
|
|
- `DELETE /sessions/{session_id}`
|
|
- `GET /sessions/{session_id}/status`
|
|
|
|
## Acceptance
|
|
|
|
- Old `/forward` remains temporarily or fails with clear version message.
|
|
- Tests cover relay preservation of session headers.
|
|
- Decode-step payload is independent of prompt length.
|
|
|