fix: tracker must not advertise incomplete snapshot as a full model source
files_for_layer_range() silently dropped required weight files that were missing from the tracker's local snapshot directory, so a snapshot with zero (or partial) shard files downloaded still reported back a "complete" file list (metadata only, or a partial subset). Nodes then trusted that subset as fully cached and skipped the real download, only discovering the missing weight shards at model-load time. Now, if any weight file required for the requested layer range is absent on disk, the tracker returns an empty file list for that range so it isn't offered as a source at all, letting the node fall through to peers/HuggingFace for the real data. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -99,9 +99,18 @@ def select_safetensors_files_for_layers(
|
||||
if not isinstance(tensor_name, str) or not isinstance(rel_file, str):
|
||||
continue
|
||||
if _tensor_belongs_to_range(tensor_name, start_layer, end_layer, inferred_total_layers):
|
||||
selected.add(_normalise_relative_file(rel_file))
|
||||
rel = _normalise_relative_file(rel_file)
|
||||
if not (root / rel).is_file():
|
||||
# A required weight file is missing from this local snapshot: the
|
||||
# snapshot is incomplete for this layer range, so it must not be
|
||||
# advertised as a source at all. Silently dropping just this file
|
||||
# (as before) made a partial snapshot look "fully cached" to
|
||||
# downstream clients, which then skipped the real download and
|
||||
# only discovered the missing weights at model-load time.
|
||||
return []
|
||||
selected.add(rel)
|
||||
|
||||
return sorted(rel for rel in selected if (root / rel).is_file())
|
||||
return sorted(selected)
|
||||
|
||||
|
||||
def _tensor_belongs_to_range(
|
||||
|
||||
Reference in New Issue
Block a user