Skip to content

vllm_mlx.utils.download

Resumable model download with retry/timeout support.

View the complete module source at #L1-L144.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.utils.download

Resumable model download with retry/timeout support.

Pre-downloads models via huggingface_hub.snapshot_download() with configurable timeout and retry logic before passing to mlx-lm/mlx-vlm.

vllm_mlx.utils.download.logger module-attribute 

logger = logging.getLogger(__name__)

vllm_mlx.utils.download.LLM_ALLOW_PATTERNS module-attribute 

LLM_ALLOW_PATTERNS = ['*.json', 'model*.safetensors', '*.py', 'tokenizer.model', '*.tiktoken', 'tiktoken.model', '*.txt', '*.jsonl', '*.jinja']

vllm_mlx.utils.download.MLLM_ALLOW_PATTERNS module-attribute 

MLLM_ALLOW_PATTERNS = ['*.json', '*.safetensors', '*.py', '*.model', '*.tiktoken', '*.txt', '*.jinja']

vllm_mlx.utils.download.DownloadConfig dataclass 

DownloadConfig(download_timeout: int = 300, max_retries: int = 3, retry_backoff_base: float = 2.0, offline: bool = False)

Configuration for model download behavior.

vllm_mlx.utils.download.DownloadConfig.download_timeout class-attribute instance-attribute 

download_timeout: int = 300

vllm_mlx.utils.download.DownloadConfig.max_retries class-attribute instance-attribute 

max_retries: int = 3

vllm_mlx.utils.download.DownloadConfig.retry_backoff_base class-attribute instance-attribute 

retry_backoff_base: float = 2.0

vllm_mlx.utils.download.DownloadConfig.offline class-attribute instance-attribute 

offline: bool = False

vllm_mlx.utils.download.ensure_model_downloaded 

ensure_model_downloaded(model_name: str, config: DownloadConfig | None = None, is_mllm: bool = False) -> Path

Ensure a model is available locally, downloading with retry if needed.

Parameters:

  • model_name (str) –

    HuggingFace model name or local path.

  • config (DownloadConfig | None, default: None ) –

    Download configuration. Uses defaults if None.

  • is_mllm (bool, default: False ) –

    If True, use MLLM download patterns (broader file set).

Returns:

  • Path –

    Path to the local model directory.

Raises:

  • RuntimeError –

    If download fails after all retries.

  • KeyboardInterrupt –

    Propagated immediately without retry.

Source code in vllm_mlx/utils/download.py 
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
def ensure_model_downloaded(
    model_name: str,
    config: DownloadConfig | None = None,
    is_mllm: bool = False,
) -> Path:
    """
    Ensure a model is available locally, downloading with retry if needed.

    Args:
        model_name: HuggingFace model name or local path.
        config: Download configuration. Uses defaults if None.
        is_mllm: If True, use MLLM download patterns (broader file set).

    Returns:
        Path to the local model directory.

    Raises:
        RuntimeError: If download fails after all retries.
        KeyboardInterrupt: Propagated immediately without retry.
    """
    if config is None:
        config = DownloadConfig()

    model_path = Path(model_name)
    if model_path.exists():
        logger.info(f"Model found at local path: {model_path}")
        return model_path

    if config.offline:
        logger.info(f"Offline mode: looking for cached {model_name}")
        try:
            result = Path(snapshot_download(model_name, local_files_only=True))
            logger.info(f"Found cached model at {result}")
            return result
        except Exception as e:
            raise RuntimeError(
                f"Model '{model_name}' not found in local cache. "
                f"Download it first without --offline flag."
            ) from e

    allow_patterns = MLLM_ALLOW_PATTERNS if is_mllm else LLM_ALLOW_PATTERNS

    # Set HF download timeout via environment variable
    old_timeout = os.environ.get("HF_HUB_DOWNLOAD_TIMEOUT")
    os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = str(config.download_timeout)

    last_error = None
    try:
        for attempt in range(1, config.max_retries + 1):
            try:
                logger.info(
                    f"Downloading model {model_name} "
                    f"(attempt {attempt}/{config.max_retries}, "
                    f"timeout={config.download_timeout}s)"
                )
                result = Path(
                    snapshot_download(
                        model_name,
                        allow_patterns=allow_patterns,
                    )
                )
                logger.info(f"Model downloaded successfully to {result}")
                return result
            except KeyboardInterrupt:
                logger.warning("Download interrupted by user.")
                raise
            except Exception as e:
                last_error = e
                if attempt < config.max_retries:
                    wait = config.retry_backoff_base**attempt
                    logger.warning(
                        f"Download attempt {attempt} failed: {e}. "
                        f"Retrying in {wait:.0f}s..."
                    )
                    time.sleep(wait)
                else:
                    logger.error(
                        f"Download failed after {config.max_retries} attempts."
                    )

        raise RuntimeError(
            f"Failed to download '{model_name}' after {config.max_retries} "
            f"attempts. Last error: {last_error}\n"
            f"Run the same command again to resume the download."
        )
    finally:
        # Restore original env var
        if old_timeout is None:
            os.environ.pop("HF_HUB_DOWNLOAD_TIMEOUT", None)
        else:
            os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = old_timeout

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.utils.download.DownloadConfig · class 
vllm_mlx.utils.download.DownloadConfig(download_timeout: int = 300, max_retries: int = 3, retry_backoff_base: float = 2.0, offline: bool = False)

Configuration for model download behavior.

Parameters

Name Type Required Default Description
download_timeout int no 300 Optional constructor field; defaults to 300.
max_retries int no 3 Optional constructor field; defaults to 3.
retry_backoff_base float no 2.0 Optional constructor field; defaults to 2.0.
offline bool no False Optional constructor field; defaults to False.

Returns

  • Constructs: vllm_mlx.utils.download.DownloadConfig

Exceptions and behavior

Class DownloadConfig declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L45-L51.

vllm_mlx.utils.download.ensure_model_downloaded · function 
vllm_mlx.utils.download.ensure_model_downloaded(model_name: str, config: DownloadConfig | None = None, is_mllm: bool = False) -> Path

Ensure a model is available locally, downloading with retry if needed.

Parameters

Name Type Required Default Description
model_name str yes none HuggingFace model name or local path.
config DownloadConfig \| None no None Download configuration. Uses defaults if None.
is_mllm bool no False If True, use MLLM download patterns (broader file set).

Returns

  • Type: Path
  • Direct return expressions: model_path; result

Exceptions and behavior

Function ensure_model_downloaded calls DownloadConfig, Path, model_path.exists, logger.info; can raise RuntimeError; has 2 explicit return paths. Directly raised exceptions: RuntimeError.

View source #L54-L144.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
DownloadConfig class DownloadConfig(download_timeout: int = 300, max_retries: int = 3, retry_backoff_base: float = 2.0, offline: bool = False) Configuration for model download behavior. #L45-L51
ensure_model_downloaded function ensure_model_downloaded(model_name: str, config: DownloadConfig \| None = None, is_mllm: bool = False) -> Path Ensure a model is available locally, downloading with retry if needed. #L54-L144