# ADR-0011: Auto-shard from memory budget and tracker-managed network assignment

## Status: Accepted

## Context

Early node startup required explicit `--shard-start` and `--shard-end` flags. This is
fine for expert operators but a barrier to new participants who don't know how many layers
their GPU can hold. Two improvements were needed:

1. **Auto-detect shard range**: fetch `num_hidden_layers` from the model's `config.json`
   and compute how many layers fit in available VRAM.
2. **Network-aware assignment**: instead of each node picking its own shard, the tracker
   knows the current coverage map and can tell the node which gap to fill.

## Decisions

### 1. Layer count from HuggingFace config

`AutoConfig.from_pretrained(model_id)` downloads only `config.json` (~1 KB, no weights).
`cfg.num_hidden_layers` gives the total layer count. The node uses this to set
`shard_end = num_layers - 1` when no explicit range is given.

A curated `MODEL_CATALOG` in `model_catalog.py` provides layer counts for common models
without any network call — HuggingFace is only hit for uncatalogued repos.

### 2. VRAM-aware shard sizing

`hardware.detect_hardware()` returns `vram_mb`. The node sends this to
`/v1/network/assign?device=cuda&vram_mb=<n>&hf_repo=<repo>`. The tracker responds with
a `{shard_start, shard_end}` gap that fits within the reported VRAM budget using the
`bytes_per_layer` table from the model preset.

When the tracker has no registered nodes for the model yet, `gap_found: false` is
returned and the node defaults to the full model.

### 3. --memory override

`--memory MB` allows overriding the detected VRAM. Useful for CPU nodes (which report 0
VRAM) that want to serve a specific slice using system RAM.

### 4. Tracker network assignment endpoint

`GET /v1/network/assign` replaces the old `GET /v1/nodes/assign`. It accepts
`device`, `vram_mb`, and optionally `hf_repo`. It returns:

```json
{
  "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
  "shard_start": 12,
  "shard_end": 23,
  "num_layers": 24,
  "gap_found": true,
  "price_per_token": 0.0
}
```

`price_per_token` is reserved at 0.0 for future billing integration.

## Alternatives rejected

**Fixed shard table per model**: would require updating the code for every new model.
HuggingFace config fetch is more general.

**Node computes its own gap**: requires the node to know the full coverage map. The
tracker already has this; having the tracker compute the assignment is cleaner.

## Consequences

- Nodes can join the network with a single command: `meshnet-node start --tracker <url>`
- The tracker is now the authoritative source for shard assignment
- VRAM budgets are advisory — nodes can still pin a range with explicit flags
- `price_per_token: 0.0` is a stable protocol field; future billing sets it to a real value