doctor docs
This commit is contained in:
Dobromir Popov
@@ -0,0 +1,34 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 01 — Generic capability and recipe report
|
||||
|
||||
## What to build
|
||||
|
||||
Create a model-agnostic node capability domain object and local versioned recipe-manifest reader. It must represent a selected Model Artifact identity/revision/config fingerprint, Shard range, named recipe ID/version, device/backend identity, validation timestamp/duration, success/failure state, and sanitized diagnostics.
|
||||
|
||||
Do not add Qwen-, FLA-, Triton-, CUDA-, ROCm-, or vendor-specific branches. A recipe is generic data; specific runtime behavior remains in the existing backend.
|
||||
|
||||
**Code refs:**
|
||||
|
||||
- `packages/node/meshnet_node/model_catalog.py` — existing generic HF config/model metadata helpers
|
||||
- `packages/node/meshnet_node/hardware.py` — device identity/executability inventory
|
||||
- `packages/node/meshnet_node/model_backend.py` — model/shard loading path
|
||||
- `packages/node/pyproject.toml` — package data declarations
|
||||
|
||||
## Test-first
|
||||
|
||||
1. Write a unit test building reports for two arbitrary fixture model IDs and asserting no model-specific normalization/branch is required.
|
||||
2. Write a manifest-version validation test: valid local manifest loads; malformed/unknown schema produces actionable non-secret diagnostics.
|
||||
3. Implement the smallest schema, serialization, and local manifest reader needed by later stories.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Capability report has a stable JSON-serializable schema with model identity/fingerprint, shard range, recipe ID/version, backend/device identity, status, timing, and sanitized diagnostic fields
|
||||
- [ ] Generic arbitrary model IDs are preserved; no Qwen or optional-kernel name is a product default or code-path discriminator
|
||||
- [ ] Local recipe manifest has an explicit schema/catalogue version
|
||||
- [ ] Malformed manifest/report input fails locally with actionable diagnostics and never leaks environment secrets
|
||||
- [ ] Unit tests cover serialization, schema validation, and model-agnostic behavior
|
||||
|
||||
## Blocked by
|
||||
|
||||
None.
|
||||
@@ -0,0 +1,35 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 02 — Doctor selected model/shard with a bounded real forward
|
||||
|
||||
## What to build
|
||||
|
||||
Add `meshnet-node doctor`. By default it validates only the selected Model Artifact and Shard from flags/config. Reuse the production model-loading/backend execution path and execute a bounded real forward through the selected Shard; a generic Torch allocation or synthetic benchmark is insufficient.
|
||||
|
||||
It emits concise human output plus capability-report JSON. Add explicit `--all-recipes` plumbing for support/CI without making ordinary startup validate unrelated/downloaded models. The default tests must inject a fake/lightweight backend; a real-model test is integration-marked and environment-gated with model identity supplied externally.
|
||||
|
||||
**Code refs:**
|
||||
|
||||
- `packages/node/meshnet_node/cli.py` — subcommand parser and config/flag resolution
|
||||
- `packages/node/meshnet_node/model_backend.py` — `TorchModelShard`, `encode_prompt`, `forward_bytes`
|
||||
- `packages/node/meshnet_node/torch_server.py` — production backend construction
|
||||
- `tests/test_node_startup.py`, `tests/test_real_model_backend.py` — startup/backend test patterns
|
||||
|
||||
## Test-first
|
||||
|
||||
1. Red: `doctor` reports generic hardware availability as ready without invoking model validation.
|
||||
2. Red: an injected backend forward failure still produces a success capability.
|
||||
3. Green: selected model/shard invokes one bounded generic forward and yields success only on completion.
|
||||
4. Add an `integration`-marked, env-gated test whose model ID/source is configurable; it has no model-specific default.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] `meshnet-node doctor` resolves the same selected model/shard/config path as startup
|
||||
- [ ] Default doctor performs a bounded real forward through the selected shard before reporting success
|
||||
- [ ] `--all-recipes` is explicit and does not change default onboarding cost
|
||||
- [ ] Failure returns non-zero, writes a failed capability report, and prints a user-actionable category without raw traceback by default
|
||||
- [ ] Unit tests require no GPU or model download; a separately marked integration smoke test is model-configurable
|
||||
|
||||
## Blocked by
|
||||
|
||||
`01-generic-capability-report.md`.
|
||||
@@ -0,0 +1,34 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 03 — Fail-closed startup admission lifecycle
|
||||
|
||||
## What to build
|
||||
|
||||
Gate `run_startup` on a fresh, matching successful capability report before routable Tracker registration. A Node selected for a Model Preset/shard must fail closed if its recipe cannot perform the doctor forward: no ready/registered endpoint and no paid request acceptance.
|
||||
|
||||
Keep local diagnostic behavior useful: a failed report may be persisted/exposed locally, but the Node must not advertise the failed model/shard as ready. Define a bounded freshness/match rule so a report cannot be reused for a different model revision, shard, recipe, or backend identity.
|
||||
|
||||
**Code refs:**
|
||||
|
||||
- `packages/node/meshnet_node/startup.py` — download/load/start/register sequence
|
||||
- `packages/node/meshnet_node/cli.py` — `start` and default startup error paths
|
||||
- `packages/node/meshnet_node/torch_server.py` — server lifecycle
|
||||
- `tests/test_node_startup.py` — fake startup and registration capture patterns
|
||||
|
||||
## Test-first
|
||||
|
||||
1. Red: backend validation failure still causes `/v1/nodes/register` to be called.
|
||||
2. Red: a success report for one arbitrary model/shard is reused for another.
|
||||
3. Green: matching successful validation reaches registration; failed/stale/mismatched validation exits before registration.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Explicit selected model/shard fails closed before routable registration when validation fails
|
||||
- [ ] Startup sends only a matching successful capability report with its registration payload
|
||||
- [ ] Failed, stale, model-mismatched, shard-mismatched, recipe-mismatched, and backend-mismatched reports are rejected locally
|
||||
- [ ] Existing stub/test startup remains usable through an explicit test-safe capability path, not a production bypass
|
||||
- [ ] Tests prove the tracker receives no registration on a failed validation
|
||||
|
||||
## Blocked by
|
||||
|
||||
`01-generic-capability-report.md`, `02-doctor-real-forward.md`.
|
||||
@@ -0,0 +1,34 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 04 — Tracker validated-capability registration and routing gate
|
||||
|
||||
## What to build
|
||||
|
||||
Extend Node registration, tracker state, network-map visibility, and route selection so a candidate is eligible only when it presents a successful capability report matching the route Model Artifact and Shard. Treat recipe/backend/capability data as evidence, not a trusted performance assertion. Preserve legacy behavior only through an explicit, documented compatibility policy; no new paid route may rely on an absent proof once admission is enforced.
|
||||
|
||||
**Code refs:**
|
||||
|
||||
- `packages/tracker/meshnet_tracker/server.py` — `/v1/nodes/register`, tracker node state, route selection, network map
|
||||
- `tests/test_tracker_routing.py` — registration and route tests
|
||||
- `packages/node/meshnet_node/startup.py` — registration payload producer
|
||||
- `docs/adr/0011-auto-shard-and-network-assignment.md` — tracker-owned assignment context
|
||||
- `docs/adr/0013-rolling-stats-smart-routing.md` — performance routing context
|
||||
|
||||
## Test-first
|
||||
|
||||
1. Red: a node with no/failed/mismatched capability report can register as route-eligible for a model/shard.
|
||||
2. Red: route selection includes a candidate whose report is for a different arbitrary model or shard.
|
||||
3. Green: valid matching candidates route normally; network map exposes only sanitized admission status.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Registration validates the generic capability-report schema and records sanitized capability state
|
||||
- [ ] Route selection excludes invalid, absent, failed, stale, model-mismatched, shard-mismatched, recipe-mismatched, or catalogue-version-incompatible candidates
|
||||
- [ ] Valid matching candidates retain normal coverage-first and throughput routing behavior
|
||||
- [ ] Network map/operator view exposes an actionable admission state without raw exceptions or secrets
|
||||
- [ ] Protocol compatibility policy for older Nodes is tested and documented
|
||||
- [ ] Deterministic tracker tests cover arbitrary model IDs, not a Qwen fixture
|
||||
|
||||
## Blocked by
|
||||
|
||||
`01-generic-capability-report.md`, `03-fail-closed-startup-admission.md`.
|
||||
@@ -0,0 +1,34 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 05 — Model-agnostic operator documentation and hardware-lane contract
|
||||
|
||||
## What to build
|
||||
|
||||
Document the capability-admission lifecycle, `doctor` usage, failure states, model-agnostic recipe semantics, and the certified hardware-lane release check. Correct setup guidance so it does not imply that an optional accelerator path is universally supported merely because a package can be installed.
|
||||
|
||||
Use generic commands/placeholders in primary docs. Any concrete model used for development belongs in a clearly labelled optional example or environment-gated test configuration, never a support guarantee.
|
||||
|
||||
**Code refs:**
|
||||
|
||||
- `QUICKSTART.md` — node installation/ROCm/optional-backend guidance
|
||||
- `packages/node/meshnet_node/cli.py` — doctor user-facing output
|
||||
- `docs/adr/0023-model-agnostic-node-capability-admission.md`
|
||||
- `tests/test_node_startup.py`, `tests/test_real_model_backend.py` — integration marker conventions
|
||||
|
||||
## Test-first / verification
|
||||
|
||||
1. Add tests for concise doctor output/category mapping where practical.
|
||||
2. Verify documentation commands use the generic selected-model interface and explain the distinction between validated versus merely detected hardware.
|
||||
3. Add a release-CI runbook contract for an opt-in `integration` doctor run per certified hardware lane, with model identity supplied by CI configuration.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Docs explain that readiness requires a successful real-forward capability report
|
||||
- [ ] Docs distinguish detected hardware, validated recipe, and routable Node states
|
||||
- [ ] Docs make no model/vendor/optional-kernel universal support promise
|
||||
- [ ] Certified-lane CI contract is documented, including model-configurable integration environment and expected evidence
|
||||
- [ ] Signed Node updates are listed as a follow-up; P0 is explicit that it does not dynamically install executable recipes or system dependencies
|
||||
|
||||
## Blocked by
|
||||
|
||||
`02-doctor-real-forward.md`, `04-tracker-validated-capability-routing.md`.
|
||||
Reference in New Issue
Block a user