d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
30 lines
1.7 KiB
Markdown
30 lines
1.7 KiB
Markdown
Status: ready-for-agent
|
|
|
|
# 05 — OpenAI-compatible gateway
|
|
|
|
## What to build
|
|
|
|
Expose a production-shape HTTP API from the gateway so that any client using the OpenAI Python SDK (or any OpenAI-compatible tool) works by changing only the `base_url`. The gateway translates OpenAI-format requests into inference route execution and streams responses back in OpenAI-format.
|
|
|
|
Endpoints to implement:
|
|
- `POST /v1/chat/completions` — streaming (`text/event-stream`) and non-streaming
|
|
- `GET /v1/models` — returns the list of model presets currently routable on the network
|
|
- `GET /v1/health` — liveness check
|
|
|
|
The gateway selects an inference route from the tracker, executes the shard pipeline, and assembles the streamed response. If the tracker returns no route for the requested model, the gateway responds with a standard OpenAI-format error (`model_not_available`).
|
|
|
|
Authentication (API key → SOL/USDC balance) is a stub in this issue — return 200 for any non-empty `Authorization` header. Real payment gating comes in issue 06.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `openai.OpenAI(base_url="http://localhost:8080/v1", api_key="test").chat.completions.create(model="stub-model", messages=[...])` returns a valid response
|
|
- [ ] Streaming works: `stream=True` returns `text/event-stream` chunks in OpenAI SSE format
|
|
- [ ] `GET /v1/models` returns a JSON array of available model preset names
|
|
- [ ] A request for an unavailable model returns an OpenAI-format error response with HTTP 503
|
|
- [ ] LangChain `ChatOpenAI(base_url=..., api_key=...)` works against the gateway
|
|
- [ ] An integration test covers streaming and non-streaming paths end-to-end through a real tracker and two stub nodes
|
|
|
|
## Blocked by
|
|
|
|
- `03-tracker-registration-and-routing.md`
|