Dobromir Popov 4e292eaaae fix: shard_end convention — inclusive (0-based) not exclusive ...

model_backend.py was using Python-style exclusive end (layers[start:end])
while all callers (CLI, tests, QUICKSTART) use inclusive 0-based indexing.
Result: 24-layer model with shard_end=23 ran only 23 layers and never
set is_tail=True, so decode_tail() was never called and responses were empty.

- is_tail: == total_layers → >= total_layers - 1
- _run_layers: layers[start:end] → layers[start:end+1]
- Validation: > total_layers → >= total_layers (was also wrong)

Inference confirmed: Qwen2.5-0.5B-Instruct now returns real LLM output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-06-29 18:37:01 +03:00

15 KiB

Raw Blame History

View Raw