93 lines
5.4 KiB
Markdown
93 lines
5.4 KiB
Markdown
Status: ready-for-agent
|
|
|
|
Scoped 2026-07-07 from an interrupted Claude session. This is a resume/cleanup task for routing and live-progress work that is partly committed and partly left dirty in the working tree.
|
|
|
|
# 24 - Finish learned-routing telemetry and live-progress cleanup
|
|
|
|
## Current state
|
|
|
|
The main dynamic routing feature is already committed at `518c259` (`routing improvements - dynamic (wip)`):
|
|
|
|
- `packages/tracker/meshnet_tracker/routing_stats.py` - decayed-EWMA route stats store, epsilon-greedy route selection, diagnostics.
|
|
- `packages/tracker/meshnet_tracker/server.py` - route enumeration per head, bandit selection in the chat proxy, epoch bumps on node join/leave, `/v1/routing`, route sample recording with 8-token hygiene.
|
|
- `packages/tracker/meshnet_tracker/cli.py` - `--route-explore-share`, `--route-weight-alpha`, `--route-stats-half-life` and env vars.
|
|
- `packages/tracker/meshnet_tracker/dashboard.html` - "Routing (learned)" panel.
|
|
- `docs/adr/0021-dynamic-statistical-routing.md` - design record.
|
|
- `tests/test_dynamic_routing.py` - includes the exact GPU(0-21)+CPU(0-39) topology, hybrid downstream `start_layer=22`, 0.6/0.4 traffic split for a 1.5 TPS ratio, and scout-rate behavior.
|
|
|
|
The current working tree still has uncommitted follow-up work:
|
|
|
|
- `packages/node/meshnet_node/torch_server.py` - tracks in-flight chat requests, exposes `TorchNodeServer.current_requests`, prints generation progress with TPS.
|
|
- `packages/node/meshnet_node/startup.py` - sends `current_requests` in heartbeat payloads and increases heartbeat cadence while busy.
|
|
- `packages/tracker/meshnet_tracker/server.py` - accepts heartbeat `current_requests`, includes them in `/v1/network/map`, and logs `proxy connecting` before upstream connection.
|
|
- `packages/tracker/meshnet_tracker/dashboard.html` - enriches the call wall from heartbeat `current_requests` so active requests remain visible even before terminal proxy events.
|
|
- `tests/test_real_model_backend.py` and `tests/test_tracker_routing.py` - targeted coverage for current-request snapshots, heartbeat sanitization/storage, and TPS progress logging.
|
|
- `QUICKSTART.md` - documents optional linear-attention fast-path packages for Qwen3.5/3.6 GPU nodes.
|
|
|
|
There is also an untracked local file, `.claude/settings.local.json`, which should not be included unless the owner explicitly wants local Claude settings committed.
|
|
|
|
## Known blocker found during resume
|
|
|
|
Targeted pytest currently fails during import before reaching the new tests:
|
|
|
|
```text
|
|
TypeError: unsupported operand type(s) for |: 'builtin_function_or_method' and 'NoneType'
|
|
```
|
|
|
|
Immediate cause: `packages/tracker/meshnet_tracker/server.py:1490` annotates `ws_lock: threading.Lock | None = None`. `threading.Lock` is a factory function at runtime, not a type, so `| None` evaluates eagerly and crashes. This exists on `HEAD` too, not just in the dirty telemetry changes.
|
|
|
|
Fix options:
|
|
|
|
- Add `from __future__ import annotations` at the top of `server.py`, then run enough tests to catch any annotation side effects.
|
|
- Or change that annotation to a safe runtime type such as `Any | None` / remove the union annotation. Keep the change minimal.
|
|
|
|
## What to do next
|
|
|
|
1. Fix the import-time `threading.Lock | None` crash.
|
|
2. Re-run the targeted tests:
|
|
|
|
```bash
|
|
.\.venv\Scripts\python.exe -m pytest tests/test_tracker_routing.py::test_tracker_heartbeat_stores_current_requests tests/test_tracker_routing.py::test_normalize_current_requests_sanitizes_payload tests/test_real_model_backend.py::test_current_requests_snapshot_while_generating tests/test_real_model_backend.py::test_distributed_generating_log_includes_tps -q
|
|
```
|
|
|
|
3. Run the relevant routing regression tests:
|
|
|
|
```bash
|
|
.\.venv\Scripts\python.exe -m pytest tests/test_dynamic_routing.py tests/test_tracker_routing.py -q
|
|
```
|
|
|
|
4. If practical, run the non-integration suite:
|
|
|
|
```bash
|
|
.\.venv\Scripts\python.exe -m pytest tests/ -q -m "not integration"
|
|
```
|
|
|
|
5. Confirm or document the pre-existing failure from the interrupted session: `test_proxy_chat_splits_payout_by_tracker_assigned_route_span` reportedly failed on `HEAD` too and was unrelated.
|
|
6. Commit the intentional work in two commits if it remains naturally split:
|
|
- learned routing is already committed in `518c259`; leave it alone unless fixing regressions there.
|
|
- commit the live-progress/current-request telemetry cleanup separately after tests pass.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] Importing `meshnet_tracker.server` no longer crashes on the lock annotation.
|
|
- [ ] Current-request heartbeat payloads are sanitized and surfaced in `/v1/network/map`.
|
|
- [ ] Node-side in-flight chat snapshots report request id, model, token count, elapsed seconds, tokens/sec, and routing completion.
|
|
- [ ] Dashboard call wall can show active requests from heartbeat data, not only tracker console terminal events.
|
|
- [ ] Targeted telemetry tests pass.
|
|
- [ ] Dynamic routing tests still pass, including GPU(0-21)+CPU(0-39) hybrid-route enumeration and traffic split behavior.
|
|
- [ ] Full or non-integration suite result is recorded; unrelated pre-existing failures are named explicitly.
|
|
- [ ] `.claude/settings.local.json` remains uncommitted unless intentionally approved.
|
|
|
|
## ADR links
|
|
|
|
- [ADR-0020](../../docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md)
|
|
- [ADR-0021](../../docs/adr/0021-dynamic-statistical-routing.md)
|
|
|
|
## Blocked by
|
|
|
|
None. The import-time annotation crash is the first fix.
|
|
|
|
## Blocks
|
|
|
|
Clean handoff/commit of the interrupted live routing progress work.
|