neuron-tai/docs/adr/0023-model-agnostic-node-capability-admission.md

ADR-0023: Model-agnostic Node capability admission

Status: Accepted (P0 planned)

Context

A Node currently inventories hardware, benchmarks a generic Torch operation, loads its model, registers with the Tracker, and can be routed before its exact model/backend path has completed a bounded real forward. Optional JIT or model-kernel failures can therefore surface only after a live /forward request reaches the Node.

This is incompatible with a consumer-grade node experience. A Node must never advertise a Shard it cannot actually execute. The solution must not be coupled to a development model; model-specific hardcoding would recreate the support burden for every new Model Artifact.

Decision

• Introduce a generic versioned capability report keyed by Model Artifact identity, Shard range, named recipe, backend/device identity, and local validation result.
• A recipe is data and can be one of several possible execution paths for the same Model Preset. Every recipe validates itself using a bounded real forward.
• meshnet-node doctor validates the selected model/shard by default. An explicit all-recipes mode supports CI and diagnosis.
• Startup fails closed for an explicitly selected Model Preset when no matching recipe validates. The Node must not become routable or accept paid work.
• Nodes register only locally validated capabilities. The Tracker routes only matching validated capabilities and uses measured performance as part of normal route selection.
• P0 carries the version of a local recipe manifest. New executable recipes arrive only through signed Node releases in a future feature. P0 does not download executable recipes, dynamically install dependencies, install OS packages/drivers, or implement an updater.
• A future Tracker-provided Model Artifact Manifest may be signed data only; it cannot instruct a Node to execute arbitrary code.

Tracker admission and the compatibility policy for older Nodes

A Node ships its capability report with POST /v1/nodes/register (capability_report), alongside an independent declaration of the recipe it serves with (recipe_id, recipe_version). The Tracker does not re-run the forward. It decides whether the presented proof covers what the Node advertises, records the verdict as a sanitized state, and routes accordingly.

Registration always succeeds — a Node with a bad proof is registered and visible, it is simply not routable. "Registered but dark" is a state an operator must be able to see and diagnose, so the verdict is returned in the registration response, logged, and exposed per node on GET /v1/network/map under capability (state, detail, proven model/shard/recipe/backend/device, timestamps). The detail is credential-redacted and clipped; a raw exception or token never reaches an operator view.

Verdicts: admitted, absent, invalid, failed, stale, model-mismatch, shard-mismatch, recipe-mismatch, catalogue-incompatible. Only admitted is proof. The proof does not travel with a reassignment: if the Tracker later moves a Node to a range it never validated, the Node is re-verdicted shard-mismatch and stops routing until it re-registers with a proof for the range it now advertises.

Freshness is checked when the proof is presented, not continuously — a long-lived Node's proof does not expire out from under it while it is heartbeating; liveness is already carried by heartbeat expiry.

Compatibility policy (--capability-policy, $MESHNET_TRACKER_CAPABILITY_POLICY):

• compat (default, transitional) — a Node that presents no report at all still routes, preserving pre-capability Node behaviour during the fleet rollout. Every other verdict is refused. Presenting a broken, failed, stale or mismatched proof is a stronger negative signal than presenting none, so it is never grandfathered.
• enforce — only admitted routes. Absent proof is not routable, and no paid route can rest on an unproven Node.

compat is a deprecating default: it exists to let a mixed fleet upgrade without an outage, and enforce becomes the default once the deployed Nodes emit reports. The policy is a single explicit switch, checked in one gate (_admitted_nodes) that every route path — proxy head selection, /v1/route, /v1/routes, and bandit route enumeration — passes through. The gate only ever removes candidates; coverage-first selection and throughput-weighted preference among the survivors are untouched, and nothing in a report can raise a Node's routing weight (performance stays measured, per ADR-0013/ADR-0021).

The Tracker also refuses a report whose recipe catalogue predates MIN_CATALOGUE_VERSION: recipe ids from an older catalogue may since have been redefined, so the proof cannot be matched to a name reliably.

Hardware claims are evidence, not a support matrix

Operator docs must distinguish three states and never collapse them: detected hardware (a GPU, a torch build, or an optional package is present — proves nothing), validated recipe (this machine ran a real forward for this model/shard/recipe/device, and there is a capability report to show for it), and routable Node (the Tracker admitted that proof for what the Node advertises). Each is strictly stronger than the last.

Consequently no doc promises that a model, vendor, or optional kernel works universally. A concrete model appears only as a clearly-labelled example or as environment-supplied test configuration. Hardware support is claimed per certified lane, where a lane is certified by an opt-in integration doctor run whose model identity comes from CI configuration and whose retained evidence is the capability report — see docs/dev/certified-hardware-lanes.md. A lane certifies hardware, not models: a new Model Artifact is unproven there until doctor has run it.

Consequences

• First startup has a bounded validation cost before registration, but failures occur before traffic rather than under a paid request.
• The registration and routing protocols gain compatibility/capability fields and require a transition policy for older Nodes.
• Hardware support claims become evidence-based and can be tested independently of specific development models.
• The signed Node update channel is deliberately deferred until this capability contract is stable.