docs: consolidate all docs under docs/ — single source of truth

Move issues (01–29) and PRD from .scratch/distributed-inference-network/
into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path
and rewrite docs/agents/issue-tracker.md to reflect the new layout.

The distributed_inference_network.egg-info/docs/ mirror is a build
artifact already covered by *.egg-info/ in .gitignore — not committed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
2026-07-01 14:18:26 +03:00
parent 78834e5045
commit d1e75ddded
34 changed files with 6 additions and 6 deletions

View File

@@ -0,0 +1,34 @@
# US-014 — Tracker-as-first-layer-node (inference entry point)
Merge the inference orchestration role of the gateway into tracker nodes that serve the first-layer shard. A tracker node for a model receives client requests directly, runs the head shard, routes activations through the network, collects the tail output, and streams tokens back to the client. The standalone gateway becomes a thin load-balancer that routes to whichever tracker node is least loaded.
## Context
Per ADR-0009:
- Any node serving `layers[0..k]` for a model can act as its tracker node
- Tracker nodes own: tokenizer, `embed_tokens`, `layers[0..k]`
- Tracker nodes select the optimal onward route from the live coverage map (they already maintain it)
- Multiple tracker nodes for the same model = horizontal scale at both the routing and the first-layer compute level
- The standalone `meshnet-gateway` process from US-005 becomes a dumb round-robin load-balancer; it no longer orchestrates shard pipelines
This collapses a network hop and a process boundary: previously gateway → tracker → node; now client → tracker-node (which is both tracker and first-layer node).
## Acceptance Criteria
- A node started with `--tracker-mode` (or automatically when assigned `shard_start=0`) exposes both its existing `/forward` endpoint and a new `/v1/chat/completions` OpenAI-compatible endpoint
- The tracker-node's `/v1/chat/completions` handler: tokenizes input, embeds, runs its own layers, selects onward route from coverage map, forwards binary activations to next node, receives tail output, decodes tokens, streams SSE back to client
- Multiple tracker nodes for the same model can each independently handle requests (verified by sending requests to both and getting valid responses)
- The existing `meshnet-gateway` is updated to proxy requests to tracker nodes (round-robin or least-connections) rather than orchestrating the pipeline itself
- The gateway can discover tracker nodes from the tracker registry (`GET /v1/tracker-nodes/<model_preset>` returns the list of endpoints for tracker nodes for that model)
- An integration test: register two tracker nodes and two mid-shard nodes for GPT-2; send 10 requests to the gateway; assert both tracker nodes received roughly equal load and all responses are valid
- Existing US-005 OpenAI-compatible tests still pass (gateway still exposes `/v1/chat/completions`, just proxies to tracker nodes now)
- `python -m pytest` passes
- Commit only this story's changes
## Implementation Notes
- Tracker-mode detection: `shard_start == 0` OR `--tracker-mode` CLI flag
- The tracker-node process runs two HTTP servers on different ports: port 7000 for node-to-node (`/forward`), port 8080 for client-facing (`/v1/chat/completions`) — or a single server with both route prefixes
- Route selection inside the tracker node: same `GET /v1/route/<model_preset>` call to the central tracker registry, but the tracker node itself is already the first hop — the returned route starts from the *second* node
- Streaming: tail node returns token IDs one by one (or in batches) back to the tracker node via chunked HTTP; tracker node converts to SSE and streams to client
- The gateway's existing pipeline orchestration code (`_run_pipeline`) can be extracted into a shared `meshnet_common.pipeline` module reused by both tracker-node and the legacy gateway path