popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

new story

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-01 13:18:46 +02:00
parent b035338e58
commit 50ec507c7a
2 changed files with 75 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
.scratch/distributed-inference-network/prd.json
View File

@@ -684,10 +684,36 @@
"US-022"
],
"completionNotes": "_relay_hop() added to torch_server.py. _get_remaining_route returns list[dict]. relay_bridge.py updated with body_base64 support. Tracker injects relay_addr into downstream hop dicts. 157 tests pass."
},
{
"id": "US-030",
"title": "30 — Manual route selection + hop-penalty benchmarking",
"description": "Two additions to the tracker. (1) Callers can pin an explicit inference route by passing an optional \"route\": [\"<node_id>\", ...] field in the POST /v1/chat/completions body. The tracker uses those nodes in order instead of auto-selecting; clients that omit the field are unaffected. (2) A new privileged POST /v1/benchmark/hop-penalty endpoint runs the same prompt through up to three routes (1-node, 2-node, 3-node) using whatever nodes are registered, records per-hop latency, and appends results to benchmark_results.json in the tracker's working directory. The routing algorithm is not changed — this story is data collection only. Auth is a header-presence stub (non-empty Authorization header required for benchmark endpoints).",
"acceptanceCriteria": [
"POST /v1/chat/completions accepts optional \"route\": [node_id, ...] in the request body; if present, tracker routes to those nodes in order; if absent, existing auto-routing is unchanged",
"Missing or invalid node IDs in route return HTTP 400 with a descriptive error message",
"POST /v1/benchmark/hop-penalty requires a non-empty Authorization header; missing/empty returns HTTP 401",
"Benchmark body: {\"model\": \"...\", \"prompt\": \"...\", \"max_new_tokens\": 64 (optional)}",
"Benchmark fans out to up to 3 routes (1-node, 2-node, 3-node) using currently registered nodes; routes with insufficient coverage are skipped, not errored",
"Benchmark response includes per-route entries: {\"route\": [node_id, ...], \"total_ms\": float, \"per_hop_ms\": [float, ...], \"tokens_generated\": int}",
"Results appended to <tracker_cwd>/benchmark_results.json (created if absent) as a JSON array; each entry includes ISO timestamp, model, sha256 of prompt, and per-route breakdown",
"GET /v1/benchmark/results returns the stored results array; also requires non-empty Authorization header",
"Integration test: pin a 1-node route and a 2-node route for the same prompt; assert 2-node result has 2 per_hop_ms entries; assert both records appear in benchmark_results.json",
"Clients that never send route or call /v1/benchmark/* are completely unaffected (existing tests pass unchanged)",
"python -m pytest passes from repo root",
"Commit only this story's changes"
],
"priority": 30,
"status": "open",
"notes": "Source issue: .scratch/distributed-inference-network/issues/30-manual-route-and-hop-benchmark.md. Design decisions grilled 2026-07-01: route via body field, explicit-only benchmark trigger, auth stub, routing algorithm unchanged, persist to JSON file.",
"dependsOn": [
"US-014",
"US-019"
]
}
],
"metadata": {
"updatedAt": "2026-06-29T15:35:00.000Z",
"updatedAt": "2026-07-01T00:00:00.000Z",
"statusVocabulary": {
"open": "Not started",
"in-design": "Decisions pending before implementation can begin",