# Dashboard test runner (operator workflow)

The tracker dashboard **Testing** tab can discover repository pytest targets and
run one collected test or approved suite at a time with live logs. The feature is
**disabled by default** and **admin-only**.

## Enable intentionally

Start the tracker with the test runner API enabled using either:

- VS Code launch configuration **`Tracker: local + dashboard test runner (8080)`**
  (uses the project tracker runtime at `.venv-rocm/bin/python` and
  `meshnet_tracker.cli`), or
- CLI flag **`--enable-test-runner`**, or
- Environment variable **`MESHNET_ENABLE_TEST_RUNNER=1`**

The default **`Tracker: local (8080)`** launch configuration does **not** enable
the test runner.

Verify the flag is available:

```bash
uv run python -m meshnet_tracker.cli --help | grep enable-test-runner
```

Log in to the dashboard as an admin account, open **Testing**, and use **Refresh
collection** before running targets.

## Child pytest interpreter

The runner spawns pytest as a subprocess without a shell. It uses
**`MESHNET_PYTHON`** when set (typically via `.env.<hostname>` loaded by
`meshnet_tracker.cli`); otherwise it falls back to the tracker process
interpreter. Point this at the venv that has dev extras and package dependencies
installed (see [test-env.md](test-env.md)).

## Default safe suites

These named suites are always available when the test runner is enabled and the
files exist in the checkout:

| Suite ID | Paths | Notes |
| --- | --- | --- |
| `suite:smoke` | `tests/test_smoke.py` | Fast sanity checks |
| `suite:dashboard` | `tests/test_dashboard.py` | Dashboard HTML/API regressions |
| `suite:routing` | `tests/test_tracker_routing.py`, `tests/test_dynamic_routing.py` | Tracker routing logic |

Collection also lists individual pytest node IDs (excluding real-inference
modules by default). You can run `suite:all` or `tag:<name>` after collection.

These suites use mocks/stubs or in-process fakes. They do **not** require live
GPU nodes, paid API credits, or a running mesh beyond the tracker itself.

## Real-inference suite (explicitly gated)

Modules matching `tests/test_real_*.py` are **never collected** and **never**
included in default suites unless you set:

```bash
export MESHNET_ENABLE_REAL_INFERENCE_TESTS=1
```

With that gate, an additional suite appears:

| Suite ID | Paths |
| --- | --- |
| `suite:real-inference` | `tests/test_real_distributed_inference.py`, `tests/test_real_model_backend.py` |

### Implications

- **`tests/test_real_distributed_inference.py`** — integration test against a
  **live tracker and registered model shards**. Requires env vars such as
  `MESHNET_REAL_INFERENCE_URL`, `MESHNET_REAL_INFERENCE_API_KEY`,
  `MESHNET_REAL_INFERENCE_MODEL`, and `MESHNET_REAL_INFERENCE_ROUTE`. Uses real
  chat completions and **consumes caller billing / API credit** on the target
  tracker.
- **`tests/test_real_model_backend.py`** — loads real PyTorch model code paths;
  needs **`torch`**, **`transformers`**, and related optional deps, and can
  require **substantial GPU/CPU RAM** depending on which cases run.

Do not enable `MESHNET_ENABLE_REAL_INFERENCE_TESTS=1` on shared or production
trackers unless you intend to spend credits and tie up hardware.

## Safety summary

| Control | Purpose |
| --- | --- |
| Disabled by default | No test subprocess unless operator opts in |
| Admin-only API/UI | Non-admins cannot start runs |
| Fixed suite list | API cannot pass arbitrary shell commands |
| No `shell=True` | pytest argv is fixed server-side |
| One run at a time | Concurrent starts are rejected |
| Real-inference env gate | Live inference tests stay out of default collection |