Single-node mode now uses HF model.generate() instead of one-shot
decode_tail(), giving correct multi-token output with KV cache.
model_backend.py:
- generate_text(messages, max_new_tokens, temperature, top_p) — full
autoregressive generation via model.generate() with chat template
- generate_text_streaming() — yields token strings via TextIteratorStreamer
- _encode_messages() — applies chat template (tokenize=False then tokenize),
falls back to joining user messages; avoids BatchEncoding issues
torch_server.py:
- _handle_chat_completions: fast path when backend is head+tail — calls
generate_text() or generate_text_streaming() directly instead of the
single-token encode_prompt+decode_tail pipeline
- _stream_openai_response: new SSE streaming handler for token iterators
- Parses max_tokens, temperature, top_p from request body
- Distributed path (partial shards) unchanged
Verified: streaming and non-streaming both work with Qwen2.5-0.5B-Instruct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
model_backend.py was using Python-style exclusive end (layers[start:end])
while all callers (CLI, tests, QUICKSTART) use inclusive 0-based indexing.
Result: 24-layer model with shard_end=23 ran only 23 layers and never
set is_tail=True, so decode_tail() was never called and responses were empty.
- is_tail: == total_layers → >= total_layers - 1
- _run_layers: layers[start:end] → layers[start:end+1]
- Validation: > total_layers → >= total_layers (was also wrong)
Inference confirmed: Qwen2.5-0.5B-Instruct now returns real LLM output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No need for --shard-start/--shard-end in the basic start command;
fix layer count for Qwen2.5-0.5B from 28 to 24 (verified via AutoConfig).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker: GET /v1/tracker-nodes/<model> returns head-shard nodes registered
with tracker_mode=true. tracker_mode stored per node in registry.
Node (stub + torch): head nodes (shard_start==0 or --tracker-mode flag)
expose /v1/chat/completions alongside /forward. Pipeline: tokenize →
embed → own layers → GET /v1/route (skip self) → binary forward through
remaining hops → decode → SSE back to client.
Gateway: proxies round-robin to tracker-nodes when found, falls back to
existing direct pipeline when none registered. All US-005 tests still pass.
CLI: --tracker-mode and --tracker-url flags on meshnet-node start.
Tests: 4 new integration tests — valid responses on 10 requests, both
tracker-nodes receive load, exact 5/5 round-robin split. 78 pass, 1 skipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Tracker: add GET /v1/tracker-nodes/<model> returning nodes registered
with tracker_mode=true whose shard_start matches the model's first layer
- Node: StubNodeServer and TorchNodeServer accept tracker_mode/tracker_url;
when tracker_mode=True (or auto-detected via shard_start==0 for Torch),
/v1/chat/completions is served alongside /forward
- TorchNodeServer: full pipeline implementation — encode_prompt → route
selection via tracker → binary forward through remaining hops → decode
- Gateway: _handle_chat_completions checks _get_tracker_nodes() first and
proxies round-robin to tracker-nodes; falls back to existing direct
pipeline when none found (preserves all US-005 backward compat)
- CLI: --tracker-mode and --tracker-url flags added to meshnet-node start
- Test: two stub tracker-nodes + two mid-shard nodes for gpt2; 10 requests;
round-robin 5/5 split verified; all OpenAI-format responses validated
- All 78 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Recovered from interrupted Codex session (ended 13:02, changes uncommitted).
All 74 tests pass.
- Tracker accepts vram_bytes, ram_bytes, quantizations[], benchmark_tokens_per_sec
in node registration payload
- GET /v1/coverage/<model_preset> returns [{start_layer, end_layer, node_count}]
- Coverage-first bin-packing: fills gaps before adding redundancy
- Speed-weighted assignment: faster nodes get wider shard ranges
- LOAD_SHARD/DROP_SHARD rebalance directives delivered via heartbeat responses
- Model is unroutable when any layer range has node_count=0
- model_presets.json config for bytes_per_layer at each quantization level
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each story shows an optional second line with agent · worktree · summary:
- _active_worktrees(): parses git worktree list --porcelain, maps feat/<id> branches
- _story_meta(): joins agent name, worktree path, completionNotes/last commit subject
- print_dashboard: renders metadata line indented below story title
- --compact flag on show/watch suppresses metadata for tight one-line-per-story view
Also wires worktree-by-default groundwork: _active_worktrees called once per
render, worktree paths shown relative to repo root.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Part 1 — status-field awareness:
- _is_done/_needs_attention/_is_active/_is_in_design helpers with passes fallback
- _story_sets returns 6-tuple: done/attention/active/in_design/ready/blocked
- Dashboard shows ⚠ Attention Required section with status_reason
- auto skips to-revise/needs-review; --include-revise overrides
- _review_report includes Attention Required section
Part 2 — agent-agnostic runner:
- --agent codex|claude|openrouter|custom on run-next/auto/review
- set-agent subcommand persists choice to .ralph-tui/agent-config.json
- _run_openrouter stub (needs OPENROUTER_API_KEY)
- _run_custom_agent: --agent-cmd script receives prompt file as $1
- list-parallel: shows ready stories safe to run concurrently
Part 3 (parallel opt-in): auto --parallel N flag available but not default
Missing (follow-up patch): worktree-by-default for single runs,
per-story metadata line in dashboard (_active_worktrees/_story_meta)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add _is_done/_needs_attention/_is_active/_is_in_design status helpers with
passes-field backward compatibility for old prd.json files
- Rewrite _story_sets to return 6 buckets: done, attention, active, in_design,
ready, blocked
- Dashboard shows ⚠ Attention required section with status_reason for
to-revise/needs-review stories; progress bar counts attention as complete
- _review_report includes dedicated Attention Required section
- auto skips attention stories by default; --include-revise flag overrides
- _status_symbol updated to handle all status values (⚠ ? ⚡ ✎ • →)
- Agent-agnostic runner: --agent codex|claude|openrouter|custom on run-next,
auto, review; default agent loaded from .ralph-tui/agent-config.json
- set-agent subcommand writes agent-config.json (agent + optional model)
- _run_openrouter stub with clear error when OPENROUTER_API_KEY not set
- Custom agent support: --agent custom --agent-cmd ./script.sh runs script
with prompt file as $1
- list-parallel subcommand prints ready stories with no shared dep chain
- auto --parallel N creates git worktrees, runs N agents concurrently,
merges on success, preserves on failure for manual resolution
- Mark US-015 done in prd.json
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each story shows an optional second line:
agent · status · worktree path (if not main) · last output summary
Sources: git worktree list, agent-config.json, session.json,
completionNotes, git log on feat/<id> branch, iteration logs.
--compact flag suppresses metadata lines.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
auto --parallel N: creates one worktree per ready task, runs N agents
concurrently, merges on success, preserves worktree on conflict.
list-parallel: shows stories safe to run concurrently.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Status-field awareness:
- Replace passes:bool with status field in ralph_progress.py
- Six buckets: done, attention, active, in-design, ready, blocked
- Dashboard shows ⚠ Attention Required section for to-revise/needs-review
- auto skips attention stories by default; --include-revise overrides
Agent agnosticism:
- --agent codex|claude|openrouter|custom on all run commands
- Persistent config in .ralph-tui/agent-config.json via set-agent subcommand
- OpenRouter adapter: calls API directly when ralph-tui lacks native plugin
- custom: --agent-cmd <script> receives prompt file as $1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Status vocabulary: open | in-design | in-progress | needs-review | to-revise | done | blocked
Vocabulary definition stored in metadata.statusVocabulary for tooling.
US-002 → to-revise (base64 wire format replaced by binary in US-011)
US-005 → to-revise (gateway orchestration role superseded by US-014 tracker-as-node)
US-014 → open
All others → done
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Token named TAI, fixed supply (BTC-inspired), no open market during bootstrap
- Price anchored to inference revenue: team holds ~100% → 36% over ~5 years
- 95% soft buyback floor; ~10% protocol spread funds wind-down reserve
- Wind-down guarantee: protocol can always buy back all issued TAI at issue price
- Open questions section captures unresolved parameters for token launch
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /v1/health, GET /v1/models, POST /v1/chat/completions (streaming + non-streaming)
- OpenAI SDK, LangChain ChatOpenAI, and SSE streaming integration tests
- Tracker-backed GET /v1/models endpoint
- OpenAI-format errors for unavailable model (503) and pipeline failures
- Malformed JSON body handled with 400 instead of crash
- Test deps (openai, langchain-openai) declared in root pyproject dev extras
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
