When the tracker restarts, nodes' registrations are lost. The heartbeat loop
was catching the 404 and printing a warning but never re-registering, leaving
the node permanently invisible until manually restarted.
Fix: on HTTP 404 heartbeat response, the loop re-posts the original
registration payload to /v1/nodes/register and updates the node_id and
heartbeat URL for subsequent beats. This also handles tracker expiry races.
The register_payload is now passed into _start_heartbeat so the thread has
everything needed for re-registration without reaching back into run_startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Route lookup was using the client-provided model name ("qwen2.5-0.5b") but
the tracker registers nodes under their full hf_repo ("Qwen/Qwen2.5-0.5B-Instruct").
This caused a 404 on /v1/route and the non-tail node fell back to the
"no downstream route available" error message.
Fix: _get_remaining_route now uses server.backend.model_id (the actual hf_repo)
for the tracker query. Skips self by port matching rather than blind route[0] drop.
Also prints a warning when route lookup fails so the cause is visible.
Distributed generation was also only producing 1 token (single greedy argmax
in decode_tail). Replaced with an autoregressive loop: head node encodes the
growing sequence and forwards to the downstream shard each step, collecting
one token per iteration up to max_tokens or EOS.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/route now resolves HF model nodes (by hf_repo or short name)
in addition to preset models, using the same greedy interval-cover logic.
This allows distributed inference routing across two nodes each holding
half the model.
Endpoint dedup: re-registering the same endpoint atomically replaces the
old entry so stale registrations don't accumulate across node restarts.
Purge logging: tracker now prints when a node expires due to missed
heartbeats so operators can see dead nodes being removed.
Timing fix: heartbeat timeout raised from 30s to 90s (3 missed beats);
node heartbeat interval lowered from 30s to 20s to maintain margin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/network/assign now accepts an optional `hf_repo` query param
to restrict assignment to a specific model, and returns `gap_found: bool`
so callers know whether they received a real gap vs a redundancy slot.
Node startup with --model-id (no explicit shard args) now queries the
tracker first for an uncovered gap for that model before defaulting to
full coverage (0..n-1). This means a second node with --model-id will
serve only the missing layers, not the whole model again.
Auto-join fallback (no --model-id) now prints why it fell through
instead of silently switching to stub-model.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker now prints a line when a node registers and on every heartbeat
received. Node prints its assigned node_id after successful registration
and starts a daemon heartbeat thread (30s interval) that logs each send.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker:
- _NodeEntry gains hf_repo + num_layers fields (parsed from register body)
- GET /v1/network/assign — finds the biggest uncovered shard gap across
registered HF-model nodes; returns {hf_repo, shard_start, shard_end, num_layers}
- Returns 503 when no HF-model nodes are registered yet
Node startup:
- When model_id is set: registers with tracker including hf_repo + num_layers
so other nodes can auto-join this model
- When model_id is empty/None: queries /v1/network/assign, gets assigned the
missing layers, loads TorchNodeServer with the assigned shard automatically
- Fixes empty-string model_id leaking from DEFAULTS (treats "" same as None)
Usage: `meshnet-node start --tracker http://localhost:8080 --quantization bfloat16`
Node discovers what to serve and joins the network without any model flags.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Single-node mode now uses HF model.generate() instead of one-shot
decode_tail(), giving correct multi-token output with KV cache.
model_backend.py:
- generate_text(messages, max_new_tokens, temperature, top_p) — full
autoregressive generation via model.generate() with chat template
- generate_text_streaming() — yields token strings via TextIteratorStreamer
- _encode_messages() — applies chat template (tokenize=False then tokenize),
falls back to joining user messages; avoids BatchEncoding issues
torch_server.py:
- _handle_chat_completions: fast path when backend is head+tail — calls
generate_text() or generate_text_streaming() directly instead of the
single-token encode_prompt+decode_tail pipeline
- _stream_openai_response: new SSE streaming handler for token iterators
- Parses max_tokens, temperature, top_p from request body
- Distributed path (partial shards) unchanged
Verified: streaming and non-streaming both work with Qwen2.5-0.5B-Instruct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
model_backend.py was using Python-style exclusive end (layers[start:end])
while all callers (CLI, tests, QUICKSTART) use inclusive 0-based indexing.
Result: 24-layer model with shard_end=23 ran only 23 layers and never
set is_tail=True, so decode_tail() was never called and responses were empty.
- is_tail: == total_layers → >= total_layers - 1
- _run_layers: layers[start:end] → layers[start:end+1]
- Validation: > total_layers → >= total_layers (was also wrong)
Inference confirmed: Qwen2.5-0.5B-Instruct now returns real LLM output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No need for --shard-start/--shard-end in the basic start command;
fix layer count for Qwen2.5-0.5B from 28 to 24 (verified via AutoConfig).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker: GET /v1/tracker-nodes/<model> returns head-shard nodes registered
with tracker_mode=true. tracker_mode stored per node in registry.
Node (stub + torch): head nodes (shard_start==0 or --tracker-mode flag)
expose /v1/chat/completions alongside /forward. Pipeline: tokenize →
embed → own layers → GET /v1/route (skip self) → binary forward through
remaining hops → decode → SSE back to client.
Gateway: proxies round-robin to tracker-nodes when found, falls back to
existing direct pipeline when none registered. All US-005 tests still pass.
CLI: --tracker-mode and --tracker-url flags on meshnet-node start.
Tests: 4 new integration tests — valid responses on 10 requests, both
tracker-nodes receive load, exact 5/5 round-robin split. 78 pass, 1 skipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Tracker: add GET /v1/tracker-nodes/<model> returning nodes registered
with tracker_mode=true whose shard_start matches the model's first layer
- Node: StubNodeServer and TorchNodeServer accept tracker_mode/tracker_url;
when tracker_mode=True (or auto-detected via shard_start==0 for Torch),
/v1/chat/completions is served alongside /forward
- TorchNodeServer: full pipeline implementation — encode_prompt → route
selection via tracker → binary forward through remaining hops → decode
- Gateway: _handle_chat_completions checks _get_tracker_nodes() first and
proxies round-robin to tracker-nodes; falls back to existing direct
pipeline when none found (preserves all US-005 backward compat)
- CLI: --tracker-mode and --tracker-url flags added to meshnet-node start
- Test: two stub tracker-nodes + two mid-shard nodes for gpt2; 10 requests;
round-robin 5/5 split verified; all OpenAI-format responses validated
- All 78 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Recovered from interrupted Codex session (ended 13:02, changes uncommitted).
All 74 tests pass.
- Tracker accepts vram_bytes, ram_bytes, quantizations[], benchmark_tokens_per_sec
in node registration payload
- GET /v1/coverage/<model_preset> returns [{start_layer, end_layer, node_count}]
- Coverage-first bin-packing: fills gaps before adding redundancy
- Speed-weighted assignment: faster nodes get wider shard ranges
- LOAD_SHARD/DROP_SHARD rebalance directives delivered via heartbeat responses
- Model is unroutable when any layer range has node_count=0
- model_presets.json config for bytes_per_layer at each quantization level
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each story shows an optional second line with agent · worktree · summary:
- _active_worktrees(): parses git worktree list --porcelain, maps feat/<id> branches
- _story_meta(): joins agent name, worktree path, completionNotes/last commit subject
- print_dashboard: renders metadata line indented below story title
- --compact flag on show/watch suppresses metadata for tight one-line-per-story view
Also wires worktree-by-default groundwork: _active_worktrees called once per
render, worktree paths shown relative to repo root.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Part 1 — status-field awareness:
- _is_done/_needs_attention/_is_active/_is_in_design helpers with passes fallback
- _story_sets returns 6-tuple: done/attention/active/in_design/ready/blocked
- Dashboard shows ⚠ Attention Required section with status_reason
- auto skips to-revise/needs-review; --include-revise overrides
- _review_report includes Attention Required section
Part 2 — agent-agnostic runner:
- --agent codex|claude|openrouter|custom on run-next/auto/review
- set-agent subcommand persists choice to .ralph-tui/agent-config.json
- _run_openrouter stub (needs OPENROUTER_API_KEY)
- _run_custom_agent: --agent-cmd script receives prompt file as $1
- list-parallel: shows ready stories safe to run concurrently
Part 3 (parallel opt-in): auto --parallel N flag available but not default
Missing (follow-up patch): worktree-by-default for single runs,
per-story metadata line in dashboard (_active_worktrees/_story_meta)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add _is_done/_needs_attention/_is_active/_is_in_design status helpers with
passes-field backward compatibility for old prd.json files
- Rewrite _story_sets to return 6 buckets: done, attention, active, in_design,
ready, blocked
- Dashboard shows ⚠ Attention required section with status_reason for
to-revise/needs-review stories; progress bar counts attention as complete
- _review_report includes dedicated Attention Required section
- auto skips attention stories by default; --include-revise flag overrides
- _status_symbol updated to handle all status values (⚠ ? ⚡ ✎ • →)
- Agent-agnostic runner: --agent codex|claude|openrouter|custom on run-next,
auto, review; default agent loaded from .ralph-tui/agent-config.json
- set-agent subcommand writes agent-config.json (agent + optional model)
- _run_openrouter stub with clear error when OPENROUTER_API_KEY not set
- Custom agent support: --agent custom --agent-cmd ./script.sh runs script
with prompt file as $1
- list-parallel subcommand prints ready stories with no shared dep chain
- auto --parallel N creates git worktrees, runs N agents concurrently,
merges on success, preserves on failure for manual resolution
- Mark US-015 done in prd.json
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each story shows an optional second line:
agent · status · worktree path (if not main) · last output summary
Sources: git worktree list, agent-config.json, session.json,
completionNotes, git log on feat/<id> branch, iteration logs.
--compact flag suppresses metadata lines.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
auto --parallel N: creates one worktree per ready task, runs N agents
concurrently, merges on success, preserves worktree on conflict.
list-parallel: shows stories safe to run concurrently.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Status-field awareness:
- Replace passes:bool with status field in ralph_progress.py
- Six buckets: done, attention, active, in-design, ready, blocked
- Dashboard shows ⚠ Attention Required section for to-revise/needs-review
- auto skips attention stories by default; --include-revise overrides
Agent agnosticism:
- --agent codex|claude|openrouter|custom on all run commands
- Persistent config in .ralph-tui/agent-config.json via set-agent subcommand
- OpenRouter adapter: calls API directly when ralph-tui lacks native plugin
- custom: --agent-cmd <script> receives prompt file as $1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Status vocabulary: open | in-design | in-progress | needs-review | to-revise | done | blocked
Vocabulary definition stored in metadata.statusVocabulary for tooling.
US-002 → to-revise (base64 wire format replaced by binary in US-011)
US-005 → to-revise (gateway orchestration role superseded by US-014 tracker-as-node)
US-014 → open
All others → done
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Token named TAI, fixed supply (BTC-inspired), no open market during bootstrap
- Price anchored to inference revenue: team holds ~100% → 36% over ~5 years
- 95% soft buyback floor; ~10% protocol spread funds wind-down reserve
- Wind-down guarantee: protocol can always buy back all issued TAI at issue price
- Open questions section captures unresolved parameters for token launch
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /v1/health, GET /v1/models, POST /v1/chat/completions (streaming + non-streaming)
- OpenAI SDK, LangChain ChatOpenAI, and SSE streaming integration tests
- Tracker-backed GET /v1/models endpoint
- OpenAI-format errors for unavailable model (503) and pipeline failures
- Malformed JSON body handled with 400 instead of crash
- Test deps (openai, langchain-openai) declared in root pyproject dev extras
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
