Stream chat responses in the dashboard with live progress and unified styles

Chat now sends stream=true and renders SSE tokens incrementally with live
tok/s status, a stop button (AbortController), and a blinking cursor; because
streamed requests emit tracker 'proxy progress' events, the Call wall now
shows in-flight requests with live TPS too. Chat colors route through :root
tokens instead of hardcoded hex values.

ADR-0020 documents the changes and the mixed-topology routing flaw: a partial
GPU head (0-21) + full CPU node (0-39) gets downstream start_layer=0 instead
of 22, corrupting activations into 1-token generations that were billed and
polluted throughput stats. Fix steps recorded, not yet implemented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md

@@ -0,0 +1,127 @@
+# ADR-0020: Dashboard chat streaming, live request progress, and the mixed-topology routing flaw
+
+## Status: Accepted (chat/streaming/styles implemented); routing flaw documented, fix pending
+
+## Context
+
+Live alpha testing (2026-07-07) with `Qwen3.6-35B-A3B` split across two LAN nodes surfaced
+three UX gaps and one routing correctness flaw:
+
+1. **No visibility while a request is processing.** The Call wall showed
+   "no in-flight requests" during a 52-second generation. Cause: the dashboard chat sent
+   `stream: false`, and the tracker only emits `proxy progress` console events (the Call
+   wall's live-status source, `_tracker_log_proxy_progress`, `server.py` ~2199) for
+   **streamed** requests. Non-streamed proxying produces only
+   `route selected → connected → complete`, and short requests complete inside the
+   dashboard's 4-second poll window.
+2. **Chat did not stream.** The nodes support SSE token-by-token generation
+   (`generate_text_streaming`, hardened earlier for split shards), and the tracker proxy
+   passes `text/event-stream` through (`server.py` ~3256), but the chat panel blocked on
+   full JSON and showed nothing until completion.
+3. **Chat panel styles drifted.** The "new chat layout" redesign left hardcoded one-off
+   colors (`#1f4788`, `#2563b8`, `#10151d`, `#1a1012`, `#5c2020`, `#ffb4b4`) mixed with
+   the CSS custom-property palette.
+
+## Decisions
+
+### 1. Chat streams by default (SSE)
+
+`dashboard.html` `sendChat()` now sends `stream: true` and consumes the SSE body with a
+`ReadableStream` reader:
+
+- Assistant tokens render incrementally into the last bubble (direct DOM update, full
+  re-render only at boundaries), with a blinking `▍` cursor while streaming.
+- Chat status shows live progress: `generating… N tokens · X tok/s`.
+- The send button becomes a stop button (`■`) during generation, backed by an
+  `AbortController`; a stopped generation keeps the partial text.
+- Non-SSE responses (JSON fallback, errors) are still handled; `data: {"error": ...}`
+  stream events surface as error bubbles.
+- `streaming` flags are stripped when loading persisted sessions so an interrupted
+  generation never leaves a stuck cursor.
+
+### 2. Live in-flight visibility rides on streaming
+
+No tracker change was needed: because chat now streams, the tracker emits `proxy progress`
+events (throttled to stdout, updated in place in the console ring via
+`update_console_key`), and the existing Call wall state machine
+(`buildCallWallStates`) renders processing rows with live tokens/TPS/queue.
+
+**Known limitation (accepted):** non-streamed API requests still show no progress between
+`proxy connected` and `proxy complete` — there is nothing to report until the node
+returns. Callers wanting live visibility should use `stream: true`.
+
+### 3. Chat style tokens
+
+All chat colors route through `:root` custom properties (`--hover-bg`, `--chat-user-bg`
+`#1f6feb`, `--chat-user-border`, `--chat-error-bg/border/fg`). No hardcoded hex values
+remain in chat rules, so future palette changes are single-line edits.
+
+## Documented flaw: mixed-topology routing (partial GPU head + full CPU node)
+
+### Observed (2026-07-07, tracker 192.168.0.179:8080)
+
+Two nodes registered for `qwen3.6-35b-a3b`:
+
+| node | hardware | shard | benchmark |
+|---|---|---|---|
+| `5gMLrmyB-ec3afe6f1a03` (192.168.0.20) | RTX 4060, CUDA | 0–21 (partial, fast) | 11,164 |
+| `7j77FsPY-55249b0583e5` (192.168.0.179) | CPU | 0–39 (full, slow) | 425 |
+
+When the tracker selected the GPU node as head, it injected:
+
+```
+downstream=[{"endpoint": "http://192.168.0.179:7000", "start_layer": 0}]
+```
+
+`start_layer: 0` — not 22. The downstream full node re-ran **all 40 layers from layer 0
+on hidden states that had already passed through the head's layers 0–21**, producing
+garbage logits. Evidence from the logs:
+
+- GPU-headed requests: `generation complete tokens=1` and billed `out=0`/`out=1`/`out=3`
+  — near-instant EOS from corrupt activations.
+- The same prompt routed directly to the CPU full node: 209 tokens over 52 s (healthy).
+- Observed TPS for GPU-headed requests was meaningless (2.5–19.0 "tok/s" on 0–3 token
+  outputs), and those samples now pollute the rolling per-`(node, model)` throughput
+  stats used for routing preference.
+- Clients were **billed** for these broken 1-token responses.
+
+### Root cause
+
+The route planner treats the full-coverage node as a standalone complete route
+(`route=7j77FsPY…[0-39]`) but still injects it as the head's downstream with the
+downstream node's own `shard_start` (0) instead of `head.shard_end + 1` (22). A partial
+head + full-model downstream is a topology the planner never had to handle before —
+prior split tests used disjoint shards (0–11 + 12–23) where `shard_start` happened to
+equal the correct continuation layer.
+
+### Required fix (not yet implemented)
+
+1. **Correct continuation layer:** when hop N ends at layer `e`, hop N+1 must execute
+   from `start_layer = e + 1` regardless of the downstream node's own `shard_start`
+   (the `X-Meshnet-Start-Layer` overlapping-shard mechanism from ADR-0012 exists for
+   exactly this; the planner must set it for full-model downstream nodes too).
+2. **Route preference sanity:** with a healthy single-node full route available, prefer
+   it over a multi-hop route unless the pipeline is estimated faster; a fast head that
+   forces a slow full-model tail wins nothing (every token still crosses the CPU node).
+3. **Stat hygiene:** exclude or flag throughput samples from responses with ≤ a few
+   output tokens, so broken routes don't skew routing preference.
+4. **Billing guard (consider):** suspiciously short completions from multi-hop routes
+   during this window were billed; a minimum-viability check (or refund path) may be
+   warranted once audits land.
+
+### Verification for the fix
+
+Reproduce with a partial GPU head (0–21) + full CPU node (0–39): a chat request routed
+through the GPU head must produce output equivalent to the direct CPU route, with
+`downstream start_layer=22` visible in `proxy route selected`, and multi-token streamed
+output on the Call wall.
+
+## Verification of this ADR's implemented changes
+
+- `pytest tests/test_dashboard.py` — 5 passed (stale "Chat / inference" panel assertion
+  updated to the tabbed layout).
+- Embedded dashboard JS parses (`new Function(script)` under Node 22).
+- Live check: open `/dashboard` → Chat, send a prompt to `qwen3.6-35b-a3b` — tokens
+  must appear incrementally with live tok/s in the status line, the Call wall must show
+  the request as `processing` with live TPS, and the send button must stop generation
+  mid-stream keeping partial text.

packages/tracker/meshnet_tracker/dashboard.html

@@ -7,7 +7,10 @@
 <style>
   :root { --bg:#0d1117; --panel:#161b22; --border:#30363d; --fg:#e6edf3;
           --dim:#8b949e; --accent:#58a6ff; --ok:#3fb950; --bad:#f85149; --warn:#d29922;
-          --chat-input-bg:#21262d; --chat-user-bg:#1f4788; --chat-user-border:#388bfd; }
+          --hover-bg:#10151d;
+          --chat-input-bg:#21262d; --chat-user-bg:#1f6feb; --chat-user-border:#388bfd;
+          --chat-error-bg:rgba(248,81,73,.08); --chat-error-border:rgba(248,81,73,.35);
+          --chat-error-fg:#ffa198; }
   * { box-sizing:border-box; }
   html, body { height:100%; }
   body { margin:0; background:var(--bg); color:var(--fg);
@@ -85,7 +88,7 @@
     border:1px solid var(--border); border-radius:8px; padding:10px 12px;
     background:transparent; color:var(--fg);
   }
-  .chat-new-btn:hover { background:#10151d; border-color:var(--accent); }
+  .chat-new-btn:hover { background:var(--hover-bg); border-color:var(--accent); }
   .chat-session-list {
     flex:1; overflow:auto; padding:0 8px 12px; display:flex; flex-direction:column; gap:2px;
   }
@@ -98,8 +101,8 @@
     padding:10px 32px 10px 12px; border:1px solid transparent; border-radius:8px;
     background:transparent; color:var(--fg); cursor:pointer;
   }
-  .chat-session-item:hover { background:#10151d; }
-  .chat-session-item.active { background:#10151d; border-color:#30363d; }
+  .chat-session-item:hover { background:var(--hover-bg); }
+  .chat-session-item.active { background:var(--hover-bg); border-color:var(--border); }
   .chat-session-title {
     font-size:13px; white-space:nowrap; overflow:hidden; text-overflow:ellipsis;
   }
@@ -114,7 +117,7 @@
   }
   .chat-session-item:hover .chat-session-delete,
   .chat-session-item.active .chat-session-delete { opacity:1; }
-  .chat-session-delete:hover { color:var(--bad); background:#1a1012; }
+  .chat-session-delete:hover { color:var(--bad); background:var(--chat-error-bg); }
   .chat-main { display:flex; flex-direction:column; min-height:0; min-width:0; color-scheme:dark; }
   .chat-toolbar {
     display:flex; gap:12px; align-items:center; flex-shrink:0;
@@ -156,8 +159,14 @@
     border-bottom-left-radius:4px; max-width:100%; color:var(--fg);
   }
   .chat-bubble.error {
-    background:#1a1012; border:1px solid #5c2020; color:#ffb4b4; border-bottom-left-radius:4px;
+    background:var(--chat-error-bg); border:1px solid var(--chat-error-border);
+    color:var(--chat-error-fg); border-bottom-left-radius:4px;
   }
+  .chat-bubble.assistant.streaming::after {
+    content:"▍"; color:var(--accent); margin-left:2px;
+    animation:chat-blink 1s steps(2) infinite;
+  }
+  @keyframes chat-blink { 50% { opacity:0; } }
   .chat-compose-wrap {
     flex-shrink:0; padding:12px 16px 16px; border-top:1px solid var(--border);
     background:var(--panel);
@@ -183,7 +192,7 @@
     background:var(--chat-user-bg); color:#f0f6fc;
   }
   .chat-compose button:hover:not(:disabled) {
-    border-color:var(--accent); background:#2563b8;
+    border-color:var(--accent); background:var(--chat-user-border);
   }
   .chat-compose button:disabled { opacity:.45; cursor:not-allowed; }
   .console {
@@ -242,7 +251,7 @@
         <div class="chat-compose-wrap">
           <div class="chat-compose">
             <textarea id="chat-prompt" placeholder="Message…" rows="1" aria-label="Message"></textarea>
-            <button type="button" onclick="sendChat()" id="chat-send" title="Send (Enter)">↑</button>
+            <button type="button" onclick="onChatSendClick()" id="chat-send" title="Send (Enter)">↑</button>
           </div>
         </div>
       </div>
@@ -732,7 +741,11 @@ function loadChatSessionsStore() {
   try {
     const raw = localStorage.getItem(CHAT_SESSIONS_KEY);
     const parsed = raw ? JSON.parse(raw) : [];
-    return Array.isArray(parsed) ? parsed : [];
+    if (!Array.isArray(parsed)) return [];
+    for (const session of parsed) {
+      for (const msg of session.messages || []) delete msg.streaming;
+    }
+    return parsed;
   } catch {
     return [];
   }
@@ -950,7 +963,8 @@ function renderChatHistory() {
   history.className = "chat-messages";
   const rows = chatHistory.map(msg => {
     const role = msg.role === "user" ? "user" : msg.role === "assistant" ? "assistant" : "error";
-    return `<div class="chat-row ${role}"><div class="chat-bubble ${role}">${esc(msg.content)}</div></div>`;
+    const streaming = msg.streaming ? " streaming" : "";
+    return `<div class="chat-row ${role}"><div class="chat-bubble ${role}${streaming}">${esc(msg.content)}</div></div>`;
   }).join("");
   history.innerHTML = `<div class="chat-messages-inner">${rows}</div>`;
   history.scrollTop = history.scrollHeight;
@@ -1222,6 +1236,47 @@ async function renderAccountPanel() {
   if (account.role === "admin") await renderAdminPanel();
 }
 
+let chatAbortController = null;
+
+function onChatSendClick() {
+  if (chatBusy && chatAbortController) { chatAbortController.abort(); return; }
+  sendChat();
+}
+
+function setChatSendMode(streaming) {
+  const btn = $("chat-send");
+  if (!btn) return;
+  btn.textContent = streaming ? "■" : "↑";
+  btn.title = streaming ? "Stop generating" : "Send (Enter)";
+}
+
+function updateStreamingChatBubble(content) {
+  const history = $("chat-history");
+  if (!history) return;
+  const bubbles = history.querySelectorAll(".chat-bubble.assistant.streaming");
+  const bubble = bubbles[bubbles.length - 1];
+  if (!bubble) { renderChatHistory(); return; }
+  bubble.textContent = content;
+  const nearBottom = history.scrollHeight - history.scrollTop - history.clientHeight < 80;
+  if (nearBottom) history.scrollTop = history.scrollHeight;
+}
+
+function finalizeAssistantMessage(msg) {
+  delete msg.streaming;
+  if (!chatHistory.includes(msg)) chatHistory.push(msg);
+  if (!msg.content) msg.content = "(empty response)";
+  renderChatHistory();
+  persistActiveChatSession();
+}
+
+function removeAssistantMessage(msg) {
+  const index = chatHistory.indexOf(msg);
+  if (index >= 0) {
+    chatHistory.splice(index, 1);
+    renderChatHistory();
+  }
+}
+
 async function sendChat() {
   const promptEl = $("chat-prompt");
   const prompt = promptEl.value.trim();
@@ -1239,44 +1294,107 @@ async function sendChat() {
         .map(msg => ({ role: msg.role, content: msg.content })),
       { role: "user", content: prompt },
     ],
-    stream: false,
+    stream: true,
     max_tokens: 15120,
   };
   chatBusy = true;
-  $("chat-send").disabled = true;
+  setChatSendMode(true);
   promptEl.value = "";
   promptEl.style.height = "auto";
   chatHistory.push({ role: "user", content: prompt, model: selectedChatModel });
   renderChatHistory();
   persistActiveChatSession();
   renderChatStatus("sending request…");
-  const r = await apiCall("/v1/chat/completions", "POST", body, bearerToken);
-  chatBusy = false;
-  $("chat-send").disabled = false;
-  if (!r.ok) {
-    const error = r.data && r.data.error
-      ? (typeof r.data.error === "string" ? r.data.error : r.data.error.message || "request failed")
-      : "request failed";
+  const assistantMsg = { role: "assistant", content: "", model: selectedChatModel, streaming: true };
+  let tokens = 0;
+  let usage = null;
+  const started = Date.now();
+  chatAbortController = new AbortController();
+  try {
+    const headers = { "Content-Type": "application/json" };
+    if (bearerToken) headers["Authorization"] = "Bearer " + bearerToken;
+    const resp = await fetch("/v1/chat/completions", {
+      method: "POST",
+      headers,
+      body: JSON.stringify(body),
+      signal: chatAbortController.signal,
+    });
+    if (!resp.ok) {
+      let error = "request failed";
+      try {
+        const data = await resp.json();
+        error = typeof data.error === "string" ? data.error : (data.error && data.error.message) || error;
+      } catch { /* keep default */ }
+      throw new Error(error);
+    }
+    const contentType = resp.headers.get("Content-Type") || "";
+    if (!resp.body || !contentType.includes("text/event-stream")) {
+      const data = await resp.json();
+      assistantMsg.content = (data.choices && data.choices[0] && data.choices[0].message && data.choices[0].message.content) || "";
+      usage = data.usage || null;
+      tokens = (usage && usage.completion_tokens) || 0;
+    } else {
+      chatHistory.push(assistantMsg);
+      renderChatHistory();
+      const reader = resp.body.getReader();
+      const decoder = new TextDecoder();
+      let buffered = "";
+      for (;;) {
+        const { done, value } = await reader.read();
+        if (done) break;
+        buffered += decoder.decode(value, { stream: true });
+        let newline;
+        while ((newline = buffered.indexOf("\n")) >= 0) {
+          const line = buffered.slice(0, newline).replace(/\r$/, "");
+          buffered = buffered.slice(newline + 1);
+          if (!line.startsWith("data:")) continue;
+          const payload = line.slice(5).trim();
+          if (!payload || payload === "[DONE]") continue;
+          let chunk;
+          try { chunk = JSON.parse(payload); } catch { continue; }
+          if (chunk.error) {
+            throw new Error(typeof chunk.error === "string" ? chunk.error : chunk.error.message || "stream error");
+          }
+          if (chunk.usage) usage = chunk.usage;
+          const delta = chunk.choices && chunk.choices[0] && chunk.choices[0].delta;
+          const piece = delta && delta.content;
+          if (piece) {
+            assistantMsg.content += piece;
+            tokens += 1;
+            updateStreamingChatBubble(assistantMsg.content);
+            const secs = Math.max((Date.now() - started) / 1000, 0.001);
+            renderChatStatus(`generating… ${tokens} tokens · ${(tokens / secs).toFixed(1)} tok/s`);
+          }
+        }
+      }
+    }
+    finalizeAssistantMessage(assistantMsg);
+    renderChatStatus(usage
+      ? `done: ${usage.total_tokens ?? "?"} tokens`
+      : `done: ${tokens} tokens`);
+  } catch (err) {
+    if (err && err.name === "AbortError") {
+      if (assistantMsg.content) {
+        finalizeAssistantMessage(assistantMsg);
+        renderChatStatus(`stopped after ${tokens} tokens`);
+      } else {
+        removeAssistantMessage(assistantMsg);
+        renderChatStatus("stopped");
+      }
+    } else {
+      removeAssistantMessage(assistantMsg);
+      const error = (err && err.message) || "request failed";
       chatHistory.push({ role: "error", content: error, model: selectedChatModel });
       renderChatHistory();
       persistActiveChatSession();
       renderChatStatus(error);
-    promptEl.focus();
-    return;
     }
-  const reply = (r.data && r.data.choices && r.data.choices[0] && r.data.choices[0].message && r.data.choices[0].message.content) || "";
-  const usage = r.data && r.data.usage;
-  chatHistory.push({
-    role: "assistant",
-    content: reply || "(empty response)",
-    model: selectedChatModel,
-  });
-  renderChatHistory();
-  persistActiveChatSession();
-  renderChatStatus(usage
-    ? `done: ${usage.total_tokens ?? "?"} tokens`
-    : "done");
+  } finally {
+    chatBusy = false;
+    chatAbortController = null;
+    setChatSendMode(false);
     promptEl.focus();
+  }
 }
 
 function bindChatPromptShortcuts() {

tests/test_dashboard.py

@@ -14,7 +14,7 @@ PANELS = [
     "Node pending payouts", "Settlement history",
     "Strikes / bans / forfeitures", "Model usage", "Call wall",
     "Usage summary", "Node throughput", "Request history",
-    "Chat / inference",
+    'id="tab-chat"',
     "Console output",
 ]