vllm_mlx.model_workflow

Model acquisition, inspection, and conversion workflow helpers.

View the complete module source at #L1-L661.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.model_workflow

Model acquisition, inspection, and conversion workflow helpers.

The functions in this module intentionally avoid loading model weights. They collect repository/file metadata, download artifacts, and record manifests so a model can be qualified before it is served.

vllm_mlx.model_workflow.MODEL_MANIFEST_NAME module-attribute

MODEL_MANIFEST_NAME = 'vllm_mlx_model_manifest.json'

vllm_mlx.model_workflow.CONVERSION_MANIFEST_NAME module-attribute

CONVERSION_MANIFEST_NAME = 'vllm_mlx_conversion_manifest.json'

vllm_mlx.model_workflow.REGISTRATION_MANIFEST_NAME module-attribute

REGISTRATION_MANIFEST_NAME = 'vllm_mlx_registration_manifest.json'

vllm_mlx.model_workflow.QUALIFICATION_REQUEST_NAME module-attribute

QUALIFICATION_REQUEST_NAME = 'vllm_mlx_qualification_request.json'

vllm_mlx.model_workflow._MODEL_ID_RE module-attribute

_MODEL_ID_RE = re.compile('^[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*$')

vllm_mlx.model_workflow._NON_MLX_QUANT_METHODS module-attribute

_NON_MLX_QUANT_METHODS = frozenset({'gptq', 'awq', 'squeezellm', 'marlin', 'fp8'})

vllm_mlx.model_workflow.AcquisitionOptions dataclass

AcquisitionOptions(revision: str | None = None, target_dir: str | None = None, staging_dir: str | None = None, is_mllm: bool = False, fast_transfer: bool = True, local_files_only: bool = False)

Options for Hugging Face model acquisition.

vllm_mlx.model_workflow.AcquisitionOptions.revision class-attribute instance-attribute

revision: str | None = None

vllm_mlx.model_workflow.AcquisitionOptions.target_dir class-attribute instance-attribute

target_dir: str | None = None

vllm_mlx.model_workflow.AcquisitionOptions.staging_dir class-attribute instance-attribute

staging_dir: str | None = None

vllm_mlx.model_workflow.AcquisitionOptions.is_mllm class-attribute instance-attribute

is_mllm: bool = False

vllm_mlx.model_workflow.AcquisitionOptions.fast_transfer class-attribute instance-attribute

fast_transfer: bool = True

vllm_mlx.model_workflow.AcquisitionOptions.local_files_only class-attribute instance-attribute

local_files_only: bool = False

vllm_mlx.model_workflow.ConversionOptions dataclass

ConversionOptions(source_path: str, output_path: str, quantize: bool = False, q_bits: int | None = None, q_group_size: int | None = None, q_mode: str | None = None, quant_predicate: str | None = None, dtype: str | None = None, trust_remote_code: bool = False, dry_run: bool = False)

Options for the mlx-lm conversion backend.

vllm_mlx.model_workflow.ConversionOptions.source_path instance-attribute

source_path: str

vllm_mlx.model_workflow.ConversionOptions.output_path instance-attribute

output_path: str

vllm_mlx.model_workflow.ConversionOptions.quantize class-attribute instance-attribute

quantize: bool = False

vllm_mlx.model_workflow.ConversionOptions.q_bits class-attribute instance-attribute

q_bits: int | None = None

vllm_mlx.model_workflow.ConversionOptions.q_group_size class-attribute instance-attribute

q_group_size: int | None = None

vllm_mlx.model_workflow.ConversionOptions.q_mode class-attribute instance-attribute

q_mode: str | None = None

vllm_mlx.model_workflow.ConversionOptions.quant_predicate class-attribute instance-attribute

quant_predicate: str | None = None

vllm_mlx.model_workflow.ConversionOptions.dtype class-attribute instance-attribute

dtype: str | None = None

vllm_mlx.model_workflow.ConversionOptions.trust_remote_code class-attribute instance-attribute

trust_remote_code: bool = False

vllm_mlx.model_workflow.ConversionOptions.dry_run class-attribute instance-attribute

dry_run: bool = False

vllm_mlx.model_workflow.RegistrationOptions dataclass

RegistrationOptions(artifact_path: str, model_id: str | None = None, served_model_name: str | None = None, preset_alias: str | None = None, output_path: str | None = None, mllm: bool | None = None, tool_call_parser: str | None = None, reasoning_parser: str | None = None, default_temperature: float | None = None, default_top_p: float | None = None, default_top_k: int | None = None, default_min_p: float | None = None, default_presence_penalty: float | None = None, default_repetition_penalty: float | None = None, chat_template_kwargs: dict[str, Any] | None = None, feature_flags: list[str] | None = None)

Options for generating a portable model registration manifest.

vllm_mlx.model_workflow.RegistrationOptions.artifact_path instance-attribute

artifact_path: str

vllm_mlx.model_workflow.RegistrationOptions.model_id class-attribute instance-attribute

model_id: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.served_model_name class-attribute instance-attribute

served_model_name: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.preset_alias class-attribute instance-attribute

preset_alias: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.output_path class-attribute instance-attribute

output_path: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.mllm class-attribute instance-attribute

mllm: bool | None = None

vllm_mlx.model_workflow.RegistrationOptions.tool_call_parser class-attribute instance-attribute

tool_call_parser: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.reasoning_parser class-attribute instance-attribute

reasoning_parser: str | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_temperature class-attribute instance-attribute

default_temperature: float | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_top_p class-attribute instance-attribute

default_top_p: float | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_top_k class-attribute instance-attribute

default_top_k: int | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_min_p class-attribute instance-attribute

default_min_p: float | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_presence_penalty class-attribute instance-attribute

default_presence_penalty: float | None = None

vllm_mlx.model_workflow.RegistrationOptions.default_repetition_penalty class-attribute instance-attribute

default_repetition_penalty: float | None = None

vllm_mlx.model_workflow.RegistrationOptions.chat_template_kwargs class-attribute instance-attribute

chat_template_kwargs: dict[str, Any] | None = None

vllm_mlx.model_workflow.RegistrationOptions.feature_flags class-attribute instance-attribute

feature_flags: list[str] | None = None

vllm_mlx.model_workflow.QualificationOptions dataclass

QualificationOptions(model_id: str, server_url: str = 'http://127.0.0.1:8080', workload_path: str | None = None, output_path: str | None = None, result_path: str | None = None, repetitions: int | None = None, dry_run: bool = False, extra_args: list[str] | None = None)

Options for creating or running a bench-serve qualification handoff.

vllm_mlx.model_workflow.QualificationOptions.model_id instance-attribute

model_id: str

vllm_mlx.model_workflow.QualificationOptions.server_url class-attribute instance-attribute

server_url: str = 'http://127.0.0.1:8080'

vllm_mlx.model_workflow.QualificationOptions.workload_path class-attribute instance-attribute

workload_path: str | None = None

vllm_mlx.model_workflow.QualificationOptions.output_path class-attribute instance-attribute

output_path: str | None = None

vllm_mlx.model_workflow.QualificationOptions.result_path class-attribute instance-attribute

result_path: str | None = None

vllm_mlx.model_workflow.QualificationOptions.repetitions class-attribute instance-attribute

repetitions: int | None = None

vllm_mlx.model_workflow.QualificationOptions.dry_run class-attribute instance-attribute

dry_run: bool = False

vllm_mlx.model_workflow.QualificationOptions.extra_args class-attribute instance-attribute

extra_args: list[str] | None = None

vllm_mlx.model_workflow._now_iso

_now_iso() -> str

Source code in vllm_mlx/model_workflow.py

def _now_iso() -> str:
    return datetime.now(timezone.utc).isoformat()

vllm_mlx.model_workflow._bytes_to_gb

_bytes_to_gb(size: int | float | None) -> float | None

Source code in vllm_mlx/model_workflow.py

def _bytes_to_gb(size: int | float | None) -> float | None:
    if size is None:
        return None
    return round(float(size) / (1024**3), 3)

vllm_mlx.model_workflow._read_json

_read_json(path: Path) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _read_json(path: Path) -> dict[str, Any]:
    try:
        return json.loads(path.read_text())
    except (FileNotFoundError, json.JSONDecodeError):
        return {}

vllm_mlx.model_workflow._write_json

_write_json(path: Path, payload: dict[str, Any]) -> None

Source code in vllm_mlx/model_workflow.py

def _write_json(path: Path, payload: dict[str, Any]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")

vllm_mlx.model_workflow._local_file_inventory

_local_file_inventory(path: Path) -> tuple[list[dict[str, Any]], int]

Source code in vllm_mlx/model_workflow.py

def _local_file_inventory(path: Path) -> tuple[list[dict[str, Any]], int]:
    files = []
    total = 0
    for item in sorted(path.rglob("*")):
        if not item.is_file():
            continue
        try:
            size = item.stat().st_size
        except OSError:
            size = 0
        total += size
        files.append({"path": str(item.relative_to(path)), "size": size})
    return files, total

vllm_mlx.model_workflow._hf_file_inventory

_hf_file_inventory(model_id: str, *, revision: str | None, local_files_only: bool) -> tuple[list[dict[str, Any]], int | None, str | None]

Source code in vllm_mlx/model_workflow.py

def _hf_file_inventory(
    model_id: str, *, revision: str | None, local_files_only: bool
) -> tuple[list[dict[str, Any]], int | None, str | None]:
    if local_files_only:
        return [], None, revision

    info = HfApi().model_info(model_id, revision=revision, files_metadata=True)
    files = []
    total = 0
    total_known = True
    for sibling in getattr(info, "siblings", []) or []:
        filename = getattr(sibling, "rfilename", None)
        if not filename:
            continue
        size = getattr(sibling, "size", None)
        if size is None:
            total_known = False
        else:
            total += int(size)
        files.append({"path": filename, "size": size})
    return files, total if total_known else None, getattr(info, "sha", revision)

vllm_mlx.model_workflow._hf_config

_hf_config(model_id: str, *, revision: str | None, local_files_only: bool) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _hf_config(
    model_id: str, *, revision: str | None, local_files_only: bool
) -> dict[str, Any]:
    config_path = hf_hub_download(
        repo_id=model_id,
        filename="config.json",
        revision=revision,
        local_files_only=local_files_only,
    )
    return _read_json(Path(config_path))

vllm_mlx.model_workflow._config_value

_config_value(config: dict[str, Any], key: str) -> Any

Source code in vllm_mlx/model_workflow.py

def _config_value(config: dict[str, Any], key: str) -> Any:
    if key in config:
        return config[key]
    text_config = config.get("text_config")
    if isinstance(text_config, dict):
        return text_config.get(key)
    return None

vllm_mlx.model_workflow._model_family

_model_family(config: dict[str, Any]) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _model_family(config: dict[str, Any]) -> dict[str, Any]:
    architectures = _config_value(config, "architectures") or []
    if isinstance(architectures, str):
        architectures = [architectures]
    max_context = (
        _config_value(config, "max_position_embeddings")
        or _config_value(config, "max_sequence_length")
        or _config_value(config, "seq_length")
        or _config_value(config, "model_max_length")
    )
    return {
        "model_type": _config_value(config, "model_type"),
        "architectures": architectures,
        "torch_dtype": _config_value(config, "torch_dtype"),
        "max_context": max_context,
        "quantization": config.get("quantization") or config.get("quantization_config"),
        "has_text_config": isinstance(config.get("text_config"), dict),
        "has_vision_config": isinstance(config.get("vision_config"), dict),
        "mtp_num_hidden_layers": _config_value(config, "mtp_num_hidden_layers"),
    }

vllm_mlx.model_workflow._estimate_fit

_estimate_fit(*, total_bytes: int | None, model_files_bytes: int | None, config: dict[str, Any]) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _estimate_fit(
    *,
    total_bytes: int | None,
    model_files_bytes: int | None,
    config: dict[str, Any],
) -> dict[str, Any]:
    max_context = _model_family(config).get("max_context")
    warnings = []
    if isinstance(max_context, int) and max_context >= 262_144:
        warnings.append(
            "very large advertised context; choose an explicit serving context before loading"
        )

    # Conversion normally needs source weights, output weights, and temporary
    # shards. Keep this conservative without pretending to know architecture
    # residency exactly.
    disk_floor = None
    if total_bytes is not None:
        disk_floor = int(total_bytes * 2.2)

    memory_floor = model_files_bytes or total_bytes
    return {
        "download_size_gb": _bytes_to_gb(total_bytes),
        "model_file_size_gb": _bytes_to_gb(model_files_bytes),
        "estimated_conversion_disk_gb": _bytes_to_gb(disk_floor),
        "rough_load_memory_gb": _bytes_to_gb(memory_floor),
        "warnings": warnings,
    }

vllm_mlx.model_workflow._model_file_bytes

_model_file_bytes(files: list[dict[str, Any]]) -> int | None

Source code in vllm_mlx/model_workflow.py

def _model_file_bytes(files: list[dict[str, Any]]) -> int | None:
    total = 0
    known = False
    for entry in files:
        path = str(entry.get("path", ""))
        if not path.endswith((".safetensors", ".bin", ".gguf")):
            continue
        size = entry.get("size")
        if size is None:
            return None
        known = True
        total += int(size)
    return total if known else None

vllm_mlx.model_workflow._is_mlx_quantization

_is_mlx_quantization(quant: Any) -> bool

Return True only when quant looks like an mlx-lm quantization config.

PyTorch quantization configs (GPTQ, AWQ, ...) carry a quant_method key that MLX configs never set. Treating those as MLX-ready is a false positive reported in review.

Source code in vllm_mlx/model_workflow.py

def _is_mlx_quantization(quant: Any) -> bool:
    """Return True only when *quant* looks like an mlx-lm quantization config.

    PyTorch quantization configs (GPTQ, AWQ, ...) carry a ``quant_method``
    key that MLX configs never set.  Treating those as MLX-ready is a false
    positive reported in review.
    """
    if not isinstance(quant, dict):
        return False
    method = str(quant.get("quant_method", "")).lower()
    if method in _NON_MLX_QUANT_METHODS:
        return False
    # MLX configs written by mlx-lm always contain "bits".
    return "bits" in quant

vllm_mlx.model_workflow._looks_like_mlx_name

_looks_like_mlx_name(model: str, *, source: str) -> bool

Source code in vllm_mlx/model_workflow.py

def _looks_like_mlx_name(model: str, *, source: str) -> bool:
    name = model.lower() if source == "huggingface" else Path(model).name.lower()
    return (
        name.startswith("mlx-community/")
        or "-mlx" in name
        or "_mlx" in name
        or name.endswith("mlx")
    )

vllm_mlx.model_workflow._is_model_id

_is_model_id(value: str) -> bool

Source code in vllm_mlx/model_workflow.py

def _is_model_id(value: str) -> bool:
    return bool(_MODEL_ID_RE.fullmatch(value))

vllm_mlx.model_workflow._fast_transfer_env

_fast_transfer_env(requested: bool) -> tuple[dict[str, str], dict[str, Any]]

Source code in vllm_mlx/model_workflow.py

def _fast_transfer_env(requested: bool) -> tuple[dict[str, str], dict[str, Any]]:
    if not requested:
        return {}, {"requested": False, "enabled": False, "reason": "disabled"}
    if find_spec("hf_transfer") is None:
        return (
            {},
            {
                "requested": True,
                "enabled": False,
                "reason": "hf_transfer package is not installed",
            },
        )
    return (
        {"HF_HUB_ENABLE_HF_TRANSFER": "1"},
        {"requested": True, "enabled": True, "reason": "enabled"},
    )

vllm_mlx.model_workflow.inspect_model

inspect_model(model: str, *, revision: str | None = None, local_files_only: bool = False) -> dict[str, Any]

Inspect a local model path or Hugging Face model id without loading weights.

Source code in vllm_mlx/model_workflow.py

def inspect_model(
    model: str,
    *,
    revision: str | None = None,
    local_files_only: bool = False,
) -> dict[str, Any]:
    """Inspect a local model path or Hugging Face model id without loading weights."""
    model_path = Path(model).expanduser()
    warnings = []

    if model_path.exists():
        files, total_bytes = _local_file_inventory(model_path)
        config = _read_json(model_path / "config.json")
        resolved_revision = None
        source = "local"
        location = str(model_path)
    else:
        if not _is_model_id(model):
            raise ValueError(
                f"{model!r} is not an existing path or a Hugging Face model id"
            )
        source = "huggingface"
        location = model
        files, total_bytes, resolved_revision = _hf_file_inventory(
            model, revision=revision, local_files_only=local_files_only
        )
        try:
            config = _hf_config(
                model, revision=revision, local_files_only=local_files_only
            )
        except Exception as exc:
            config = {}
            warnings.append(f"could not read config.json: {exc}")

    model_files_bytes = _model_file_bytes(files)
    family = _model_family(config)
    estimate = _estimate_fit(
        total_bytes=total_bytes,
        model_files_bytes=model_files_bytes,
        config=config,
    )
    warnings.extend(estimate.pop("warnings"))
    has_name_signal = _looks_like_mlx_name(model, source=source) and (
        source == "huggingface" or bool(config)
    )
    has_mlx_signals = (
        _is_mlx_quantization(family.get("quantization")) or has_name_signal
    )

    return {
        "model": model,
        "source": source,
        "location": location,
        "revision": resolved_revision or revision,
        "inspected_at": _now_iso(),
        "file_count": len(files),
        "total_size_bytes": total_bytes,
        "total_size_gb": _bytes_to_gb(total_bytes),
        "model_files_size_gb": _bytes_to_gb(model_files_bytes),
        "model_family": family,
        "mlx": {
            "looks_like_mlx_artifact": has_mlx_signals,
            "needs_conversion": not has_mlx_signals,
        },
        "fit_estimate": estimate,
        "warnings": warnings,
    }

vllm_mlx.model_workflow.acquire_model

acquire_model(model_id: str, *, options: AcquisitionOptions | None = None) -> dict[str, Any]

Download a model repository and write a finalized artifact manifest.

Source code in vllm_mlx/model_workflow.py

def acquire_model(
    model_id: str,
    *,
    options: AcquisitionOptions | None = None,
) -> dict[str, Any]:
    """Download a model repository and write a finalized artifact manifest."""
    if options is None:
        options = AcquisitionOptions()
    if not _is_model_id(model_id):
        raise ValueError(f"{model_id!r} is not a Hugging Face model id")

    allow_patterns = MLLM_ALLOW_PATTERNS if options.is_mllm else LLM_ALLOW_PATTERNS
    env_updates, fast_transfer = _fast_transfer_env(options.fast_transfer)

    old_env = {key: os.environ.get(key) for key in env_updates}
    os.environ.update(env_updates)
    try:
        if options.target_dir:
            target = Path(options.target_dir).expanduser()
            if target.exists():
                raise FileExistsError(f"target path already exists: {target}")
            staging_root = (
                Path(options.staging_dir).expanduser()
                if options.staging_dir
                else target.parent
            )
            staging_root.mkdir(parents=True, exist_ok=True)
            staging = Path(
                tempfile.mkdtemp(prefix=f".{target.name}.staging-", dir=staging_root)
            )
            try:
                downloaded = Path(
                    snapshot_download(
                        model_id,
                        revision=options.revision,
                        allow_patterns=allow_patterns,
                        local_dir=str(staging),
                        local_files_only=options.local_files_only,
                    )
                )
                target.parent.mkdir(parents=True, exist_ok=True)
                shutil.move(str(downloaded), str(target))
            except Exception:
                shutil.rmtree(staging, ignore_errors=True)
                raise
            final_path = target
        else:
            final_path = Path(
                snapshot_download(
                    model_id,
                    revision=options.revision,
                    allow_patterns=allow_patterns,
                    local_files_only=options.local_files_only,
                )
            )
    finally:
        for key, value in old_env.items():
            if value is None:
                os.environ.pop(key, None)
            else:
                os.environ[key] = value

    inspection = inspect_model(str(final_path), revision=options.revision)
    manifest = {
        "kind": "vllm-mlx-model-artifact",
        "model_id": model_id,
        "revision": options.revision,
        "path": str(final_path),
        "created_at": _now_iso(),
        "allow_patterns": allow_patterns,
        "fast_transfer": fast_transfer,
        "local_files_only": options.local_files_only,
        "inspection": inspection,
    }
    manifest_path = final_path / MODEL_MANIFEST_NAME
    _write_json(manifest_path, manifest)
    manifest["manifest_path"] = str(manifest_path)
    return manifest

vllm_mlx.model_workflow._conversion_command

_conversion_command(options: ConversionOptions) -> list[str]

Source code in vllm_mlx/model_workflow.py

def _conversion_command(options: ConversionOptions) -> list[str]:
    command = [
        sys.executable,
        "-m",
        "mlx_lm",
        "convert",
        "--hf-path",
        options.source_path,
        "--mlx-path",
        options.output_path,
    ]
    if options.quantize:
        command.append("--quantize")
    if options.q_bits is not None:
        command.extend(["--q-bits", str(options.q_bits)])
    if options.q_group_size is not None:
        command.extend(["--q-group-size", str(options.q_group_size)])
    if options.q_mode:
        command.extend(["--q-mode", options.q_mode])
    if options.quant_predicate:
        command.extend(["--quant-predicate", options.quant_predicate])
    if options.dtype:
        command.extend(["--dtype", options.dtype])
    if options.trust_remote_code:
        command.append("--trust-remote-code")
    return command

vllm_mlx.model_workflow.convert_model

convert_model(options: ConversionOptions) -> dict[str, Any]

Run mlx-lm conversion and record the exact recipe.

Source code in vllm_mlx/model_workflow.py

def convert_model(options: ConversionOptions) -> dict[str, Any]:
    """Run mlx-lm conversion and record the exact recipe."""
    command = _conversion_command(options)
    started = _now_iso()
    source_inspection = inspect_model(options.source_path)
    result = {
        "kind": "vllm-mlx-conversion",
        "backend": "mlx-lm",
        "command": command,
        "source_path": options.source_path,
        "output_path": options.output_path,
        "started_at": started,
        "dry_run": options.dry_run,
        "recipe": {
            "quantize": options.quantize,
            "q_bits": options.q_bits,
            "q_group_size": options.q_group_size,
            "q_mode": options.q_mode,
            "quant_predicate": options.quant_predicate,
            "dtype": options.dtype,
            "trust_remote_code": options.trust_remote_code,
        },
        "environment": {
            "python": sys.version.split()[0],
            "platform": platform.platform(),
        },
        "source_inspection": source_inspection,
    }

    if options.dry_run:
        result["status"] = "dry_run"
        return result

    completed = subprocess.run(command, text=True, capture_output=True, check=False)
    result["returncode"] = completed.returncode
    result["stdout"] = completed.stdout
    result["stderr"] = completed.stderr
    result["completed_at"] = _now_iso()
    result["status"] = "succeeded" if completed.returncode == 0 else "failed"
    if completed.returncode != 0:
        return result

    output_path = Path(options.output_path).expanduser()
    output_inspection = inspect_model(str(output_path))
    result["output_inspection"] = output_inspection
    manifest_path = output_path / CONVERSION_MANIFEST_NAME
    _write_json(manifest_path, result)
    result["manifest_path"] = str(manifest_path)
    return result

vllm_mlx.model_workflow._existing_manifests

_existing_manifests(path: Path) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _existing_manifests(path: Path) -> dict[str, Any]:
    manifests: dict[str, Any] = {}
    for name, key in (
        (MODEL_MANIFEST_NAME, "acquisition"),
        (CONVERSION_MANIFEST_NAME, "conversion"),
    ):
        manifest_path = path / name
        if manifest_path.exists():
            manifests[key] = {
                "path": str(manifest_path),
                "payload": _read_json(manifest_path),
            }
    return manifests

vllm_mlx.model_workflow._drop_none

_drop_none(payload: dict[str, Any]) -> dict[str, Any]

Source code in vllm_mlx/model_workflow.py

def _drop_none(payload: dict[str, Any]) -> dict[str, Any]:
    return {key: value for key, value in payload.items() if value is not None}

vllm_mlx.model_workflow.register_model

register_model(options: RegistrationOptions) -> dict[str, Any]

Write a portable registration manifest for a finalized local artifact.

This deliberately does not mutate a production registry. The manifest is a handoff artifact that Ops or a deployment tool can apply after qualification.

Source code in vllm_mlx/model_workflow.py

def register_model(options: RegistrationOptions) -> dict[str, Any]:
    """Write a portable registration manifest for a finalized local artifact.

    This deliberately does not mutate a production registry. The manifest is a
    handoff artifact that Ops or a deployment tool can apply after qualification.
    """
    artifact = Path(options.artifact_path).expanduser()
    if not artifact.exists():
        raise FileNotFoundError(f"artifact path does not exist: {artifact}")
    if not artifact.is_dir():
        raise NotADirectoryError(f"artifact path must be a directory: {artifact}")

    inspection = inspect_model(str(artifact))
    model_id = options.model_id or artifact.name
    serving_defaults = _drop_none(
        {
            "temperature": options.default_temperature,
            "top_p": options.default_top_p,
            "top_k": options.default_top_k,
            "min_p": options.default_min_p,
            "presence_penalty": options.default_presence_penalty,
            "repetition_penalty": options.default_repetition_penalty,
            "chat_template_kwargs": options.chat_template_kwargs,
        }
    )
    parser_policy = _drop_none(
        {
            "tool_call_parser": options.tool_call_parser,
            "reasoning_parser": options.reasoning_parser,
        }
    )
    payload = {
        "kind": "vllm-mlx-model-registration",
        "schema_version": 1,
        "created_at": _now_iso(),
        "model_id": model_id,
        "served_model_name": options.served_model_name or model_id,
        "preset_alias": options.preset_alias,
        "artifact_path": str(artifact),
        "mllm": options.mllm,
        "feature_flags": options.feature_flags or [],
        "serving_defaults": serving_defaults,
        "parser_policy": parser_policy,
        "inspection": inspection,
        "source_manifests": _existing_manifests(artifact),
        "qualification_required": True,
        "production_ready": False,
    }

    output = (
        Path(options.output_path).expanduser()
        if options.output_path
        else artifact / REGISTRATION_MANIFEST_NAME
    )
    _write_json(output, payload)
    payload["manifest_path"] = str(output)
    return payload

vllm_mlx.model_workflow._qualification_command

_qualification_command(options: QualificationOptions) -> list[str]

Source code in vllm_mlx/model_workflow.py

def _qualification_command(options: QualificationOptions) -> list[str]:
    command = [
        sys.executable,
        "-m",
        "vllm_mlx.cli",
        "bench-serve",
        "--url",
        options.server_url,
        "--model",
        options.model_id,
        "--format",
        "json",
    ]
    if options.workload_path:
        command.extend(["--workload", options.workload_path])
    if options.repetitions is not None:
        command.extend(["--repetitions", str(options.repetitions)])
    if options.result_path:
        command.extend(["--output", options.result_path])
    if options.extra_args:
        command.extend(options.extra_args)
    return command

vllm_mlx.model_workflow.qualify_model

qualify_model(options: QualificationOptions) -> dict[str, Any]

Create or run a bench-serve qualification handoff.

Source code in vllm_mlx/model_workflow.py

def qualify_model(options: QualificationOptions) -> dict[str, Any]:
    """Create or run a bench-serve qualification handoff."""
    command = _qualification_command(options)
    payload = {
        "kind": "vllm-mlx-model-qualification",
        "schema_version": 1,
        "created_at": _now_iso(),
        "model_id": options.model_id,
        "server_url": options.server_url,
        "workload_path": options.workload_path,
        "result_path": options.result_path,
        "repetitions": options.repetitions,
        "dry_run": options.dry_run,
        "command": command,
        "production_ready": False,
    }

    if not options.dry_run:
        completed = subprocess.run(command, text=True, capture_output=True, check=False)
        payload["returncode"] = completed.returncode
        payload["stdout"] = completed.stdout
        payload["stderr"] = completed.stderr
        payload["completed_at"] = _now_iso()
        payload["status"] = "succeeded" if completed.returncode == 0 else "failed"
    else:
        payload["status"] = "dry_run"

    if options.output_path:
        output = Path(options.output_path).expanduser()
        _write_json(output, payload)
        payload["manifest_path"] = str(output)
    return payload

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.model_workflow.AcquisitionOptions · class

vllm_mlx.model_workflow.AcquisitionOptions(revision: str | None = None, target_dir: str | None = None, staging_dir: str | None = None, is_mllm: bool = False, fast_transfer: bool = True, local_files_only: bool = False)

Options for Hugging Face model acquisition.

Parameters

Name	Type	Required	Default	Description
revision	str \| None	no	None	Optional constructor field; defaults to None.
target_dir	str \| None	no	None	Optional constructor field; defaults to None.
staging_dir	str \| None	no	None	Optional constructor field; defaults to None.
is_mllm	bool	no	False	Optional constructor field; defaults to False.
fast_transfer	bool	no	True	Optional constructor field; defaults to True.
local_files_only	bool	no	False	Optional constructor field; defaults to False.

Returns

• Constructs: vllm_mlx.model_workflow.AcquisitionOptions

Exceptions and behavior

Class AcquisitionOptions declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L38-L46.

vllm_mlx.model_workflow.ConversionOptions · class

vllm_mlx.model_workflow.ConversionOptions(source_path: str, output_path: str, quantize: bool = False, q_bits: int | None = None, q_group_size: int | None = None, q_mode: str | None = None, quant_predicate: str | None = None, dtype: str | None = None, trust_remote_code: bool = False, dry_run: bool = False)

Options for the mlx-lm conversion backend.

Parameters

Name	Type	Required	Default	Description
source_path	str	yes	none	Required constructor field.
output_path	str	yes	none	Required constructor field.
quantize	bool	no	False	Optional constructor field; defaults to False.
q_bits	int \| None	no	None	Optional constructor field; defaults to None.
q_group_size	int \| None	no	None	Optional constructor field; defaults to None.
q_mode	str \| None	no	None	Optional constructor field; defaults to None.
quant_predicate	str \| None	no	None	Optional constructor field; defaults to None.
dtype	str \| None	no	None	Optional constructor field; defaults to None.
trust_remote_code	bool	no	False	Optional constructor field; defaults to False.
dry_run	bool	no	False	Optional constructor field; defaults to False.

Returns

• Constructs: vllm_mlx.model_workflow.ConversionOptions

Exceptions and behavior

Class ConversionOptions declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L50-L62.

vllm_mlx.model_workflow.RegistrationOptions · class

vllm_mlx.model_workflow.RegistrationOptions(artifact_path: str, model_id: str | None = None, served_model_name: str | None = None, preset_alias: str | None = None, output_path: str | None = None, mllm: bool | None = None, tool_call_parser: str | None = None, reasoning_parser: str | None = None, default_temperature: float | None = None, default_top_p: float | None = None, default_top_k: int | None = None, default_min_p: float | None = None, default_presence_penalty: float | None = None, default_repetition_penalty: float | None = None, chat_template_kwargs: dict[str, Any] | None = None, feature_flags: list[str] | None = None)

Options for generating a portable model registration manifest.

Parameters

Name	Type	Required	Default	Description
artifact_path	str	yes	none	Required constructor field.
model_id	str \| None	no	None	Optional constructor field; defaults to None.
served_model_name	str \| None	no	None	Optional constructor field; defaults to None.
preset_alias	str \| None	no	None	Optional constructor field; defaults to None.
output_path	str \| None	no	None	Optional constructor field; defaults to None.
mllm	bool \| None	no	None	Optional constructor field; defaults to None.
tool_call_parser	str \| None	no	None	Optional constructor field; defaults to None.
reasoning_parser	str \| None	no	None	Optional constructor field; defaults to None.
default_temperature	float \| None	no	None	Optional constructor field; defaults to None.
default_top_p	float \| None	no	None	Optional constructor field; defaults to None.
default_top_k	int \| None	no	None	Optional constructor field; defaults to None.
default_min_p	float \| None	no	None	Optional constructor field; defaults to None.
default_presence_penalty	float \| None	no	None	Optional constructor field; defaults to None.
default_repetition_penalty	float \| None	no	None	Optional constructor field; defaults to None.
chat_template_kwargs	dict[str, Any] \| None	no	None	Optional constructor field; defaults to None.
feature_flags	list[str] \| None	no	None	Optional constructor field; defaults to None.

Returns

• Constructs: vllm_mlx.model_workflow.RegistrationOptions

Exceptions and behavior

Class RegistrationOptions declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L66-L84.

vllm_mlx.model_workflow.QualificationOptions · class

vllm_mlx.model_workflow.QualificationOptions(model_id: str, server_url: str = 'http://127.0.0.1:8080', workload_path: str | None = None, output_path: str | None = None, result_path: str | None = None, repetitions: int | None = None, dry_run: bool = False, extra_args: list[str] | None = None)

Options for creating or running a bench-serve qualification handoff.

Parameters

Name	Type	Required	Default	Description
model_id	str	yes	none	Required constructor field.
server_url	str	no	'http://127.0.0.1:8080'	Optional constructor field; defaults to 'http://127.0.0.1:8080'.
workload_path	str \| None	no	None	Optional constructor field; defaults to None.
output_path	str \| None	no	None	Optional constructor field; defaults to None.
result_path	str \| None	no	None	Optional constructor field; defaults to None.
repetitions	int \| None	no	None	Optional constructor field; defaults to None.
dry_run	bool	no	False	Optional constructor field; defaults to False.
extra_args	list[str] \| None	no	None	Optional constructor field; defaults to None.

Returns

• Constructs: vllm_mlx.model_workflow.QualificationOptions

Exceptions and behavior

Class QualificationOptions declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L88-L98.

vllm_mlx.model_workflow._now_iso · function

vllm_mlx.model_workflow._now_iso() -> str

Function _now_iso calls datetime.now(timezone.utc).isoformat, datetime.now; returns datetime.now(timezone.utc).isoformat().

Parameters

This callable has no explicit inputs.

Returns

• Type: str
• Direct return expressions: datetime.now(timezone.utc).isoformat()

Exceptions and behavior

Function _now_iso calls datetime.now(timezone.utc).isoformat, datetime.now; returns datetime.now(timezone.utc).isoformat(). No direct raise statement appears in this definition.

View source #L101-L102.

vllm_mlx.model_workflow._bytes_to_gb · function

vllm_mlx.model_workflow._bytes_to_gb(size: int | float | None) -> float | None

Function _bytes_to_gb calls round, float; has 2 explicit return paths.

Parameters

Name	Type	Required	Default	Description
size	int \| float \| None	yes	none	Required positional or keyword input.

Returns

• Type: float | None
• Direct return expressions: None; round(float(size) / 1024 ** 3, 3)

Exceptions and behavior

Function _bytes_to_gb calls round, float; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L105-L108.

vllm_mlx.model_workflow._read_json · function

vllm_mlx.model_workflow._read_json(path: Path) -> dict[str, Any]

Function _read_json calls json.loads, path.read_text; has 2 explicit return paths.

Parameters

Name	Type	Required	Default	Description
path	Path	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: json.loads(path.read_text()); {}

Exceptions and behavior

Function _read_json calls json.loads, path.read_text; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L111-L115.

vllm_mlx.model_workflow._write_json · function

vllm_mlx.model_workflow._write_json(path: Path, payload: dict[str, Any]) -> None

Function _write_json calls path.parent.mkdir, path.write_text, json.dumps.

Parameters

Name	Type	Required	Default	Description
path	Path	yes	none	Required positional or keyword input.
payload	dict[str, Any]	yes	none	Required positional or keyword input.

Returns

• Type: None

Exceptions and behavior

Function _write_json calls path.parent.mkdir, path.write_text, json.dumps. No direct raise statement appears in this definition.

View source #L118-L120.

vllm_mlx.model_workflow._local_file_inventory · function

vllm_mlx.model_workflow._local_file_inventory(path: Path) -> tuple[list[dict[str, Any]], int]

Function _local_file_inventory calls sorted, path.rglob, item.is_file, item.stat; returns (files, total).

Parameters

Name	Type	Required	Default	Description
path	Path	yes	none	Required positional or keyword input.

Returns

• Type: tuple[list[dict[str, Any]], int]
• Direct return expressions: (files, total)

Exceptions and behavior

Function _local_file_inventory calls sorted, path.rglob, item.is_file, item.stat; returns (files, total). No direct raise statement appears in this definition.

View source #L123-L135.

vllm_mlx.model_workflow._hf_file_inventory · function

vllm_mlx.model_workflow._hf_file_inventory(model_id: str, *, revision: str | None, local_files_only: bool) -> tuple[list[dict[str, Any]], int | None, str | None]

Function _hf_file_inventory calls HfApi().model_info, HfApi, getattr, int; has 2 explicit return paths.

Parameters

Name	Type	Required	Default	Description
model_id	str	yes	none	Required positional or keyword input.
revision	str \| None	yes	none	Required keyword-only input.
local_files_only	bool	yes	none	Required keyword-only input.

Returns

• Type: tuple[list[dict[str, Any]], int | None, str | None]
• Direct return expressions: ([], None, revision); (files, total if total_known else None, getattr(info, 'sha', revision))

Exceptions and behavior

Function _hf_file_inventory calls HfApi().model_info, HfApi, getattr, int; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L138-L158.

vllm_mlx.model_workflow._hf_config · function

vllm_mlx.model_workflow._hf_config(model_id: str, *, revision: str | None, local_files_only: bool) -> dict[str, Any]

Function _hf_config calls hf_hub_download, _read_json, Path; returns _read_json(Path(config_path)).

Parameters

Name	Type	Required	Default	Description
model_id	str	yes	none	Required positional or keyword input.
revision	str \| None	yes	none	Required keyword-only input.
local_files_only	bool	yes	none	Required keyword-only input.

Returns

• Type: dict[str, Any]
• Direct return expressions: _read_json(Path(config_path))

Exceptions and behavior

Function _hf_config calls hf_hub_download, _read_json, Path; returns _read_json(Path(config_path)). No direct raise statement appears in this definition.

View source #L161-L170.

vllm_mlx.model_workflow._config_value · function

vllm_mlx.model_workflow._config_value(config: dict[str, Any], key: str) -> Any

Function _config_value calls config.get, isinstance, text_config.get; has 3 explicit return paths.

Parameters

Name	Type	Required	Default	Description
config	dict[str, Any]	yes	none	Required positional or keyword input.
key	str	yes	none	Required positional or keyword input.

Returns

• Type: Any
• Direct return expressions: config[key]; text_config.get(key); None

Exceptions and behavior

Function _config_value calls config.get, isinstance, text_config.get; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L173-L179.

vllm_mlx.model_workflow._model_family · function

vllm_mlx.model_workflow._model_family(config: dict[str, Any]) -> dict[str, Any]

Function _model_family calls _config_value, isinstance, config.get; returns {'model_type': _config_value(config, 'model_type'), 'architectures': architectures, 'torch_dtype': _config_value(config….

Parameters

Name	Type	Required	Default	Description
config	dict[str, Any]	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: {'model_type': _config_value(config, 'model_type'), 'architectures': architectures, 'torch_dtype': _config_value(config…

Exceptions and behavior

Function _model_family calls _config_value, isinstance, config.get; returns {'model_type': _config_value(config, 'model_type'), 'architectures': architectures, 'torch_dtype': _config_value(config…. No direct raise statement appears in this definition.

View source #L182-L201.

vllm_mlx.model_workflow._estimate_fit · function

vllm_mlx.model_workflow._estimate_fit(*, total_bytes: int | None, model_files_bytes: int | None, config: dict[str, Any]) -> dict[str, Any]

Function _estimate_fit calls _model_family(config).get, _model_family, isinstance, warnings.append; returns {'download_size_gb': _bytes_to_gb(total_bytes), 'model_file_size_gb': _bytes_to_gb(model_files_bytes), 'estimated_conve….

Parameters

Name	Type	Required	Default	Description
total_bytes	int \| None	yes	none	Required keyword-only input.
model_files_bytes	int \| None	yes	none	Required keyword-only input.
config	dict[str, Any]	yes	none	Required keyword-only input.

Returns

• Type: dict[str, Any]
• Direct return expressions: {'download_size_gb': _bytes_to_gb(total_bytes), 'model_file_size_gb': _bytes_to_gb(model_files_bytes), 'estimated_conve…

Exceptions and behavior

Function _estimate_fit calls _model_family(config).get, _model_family, isinstance, warnings.append; returns {'download_size_gb': _bytes_to_gb(total_bytes), 'model_file_size_gb': _bytes_to_gb(model_files_bytes), 'estimated_conve…. No direct raise statement appears in this definition.

View source #L204-L231.

vllm_mlx.model_workflow._model_file_bytes · function

vllm_mlx.model_workflow._model_file_bytes(files: list[dict[str, Any]]) -> int | None

Function _model_file_bytes calls str, entry.get, path.endswith, int; has 2 explicit return paths.

Parameters

Name	Type	Required	Default	Description
files	list[dict[str, Any]]	yes	none	Required positional or keyword input.

Returns

• Type: int | None
• Direct return expressions: None; total if known else None

Exceptions and behavior

Function _model_file_bytes calls str, entry.get, path.endswith, int; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L234-L246.

vllm_mlx.model_workflow._is_mlx_quantization · function

vllm_mlx.model_workflow._is_mlx_quantization(quant: Any) -> bool

Return True only when quant looks like an mlx-lm quantization config.

Parameters

Name	Type	Required	Default	Description
quant	Any	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: False; 'bits' in quant

Exceptions and behavior

Function _is_mlx_quantization calls isinstance, str(quant.get('quant_method', '')).lower, str, quant.get; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L252-L265.

vllm_mlx.model_workflow._looks_like_mlx_name · function

vllm_mlx.model_workflow._looks_like_mlx_name(model: str, *, source: str) -> bool

Function _looks_like_mlx_name calls model.lower, Path(model).name.lower, Path, name.startswith; returns name.startswith('mlx-community/') or '-mlx' in name or '_mlx' in name or name.endswith('mlx').

Parameters

Name	Type	Required	Default	Description
model	str	yes	none	Required positional or keyword input.
source	str	yes	none	Required keyword-only input.

Returns

• Type: bool
• Direct return expressions: name.startswith('mlx-community/') or '-mlx' in name or '_mlx' in name or name.endswith('mlx')

Exceptions and behavior

Function _looks_like_mlx_name calls model.lower, Path(model).name.lower, Path, name.startswith; returns name.startswith('mlx-community/') or '-mlx' in name or '_mlx' in name or name.endswith('mlx'). No direct raise statement appears in this definition.

View source #L268-L275.

vllm_mlx.model_workflow._is_model_id · function

vllm_mlx.model_workflow._is_model_id(value: str) -> bool

Function _is_model_id calls bool, _MODEL_ID_RE.fullmatch; returns bool(_MODEL_ID_RE.fullmatch(value)).

Parameters

Name	Type	Required	Default	Description
value	str	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: bool(_MODEL_ID_RE.fullmatch(value))

Exceptions and behavior

Function _is_model_id calls bool, _MODEL_ID_RE.fullmatch; returns bool(_MODEL_ID_RE.fullmatch(value)). No direct raise statement appears in this definition.

View source #L278-L279.

vllm_mlx.model_workflow._fast_transfer_env · function

vllm_mlx.model_workflow._fast_transfer_env(requested: bool) -> tuple[dict[str, str], dict[str, Any]]

Function _fast_transfer_env calls find_spec; has 3 explicit return paths.

Parameters

Name	Type	Required	Default	Description
requested	bool	yes	none	Required positional or keyword input.

Returns

• Type: tuple[dict[str, str], dict[str, Any]]
• Direct return expressions: ({}, {'requested': False, 'enabled': False, 'reason': 'disabled'}); ({}, {'requested': True, 'enabled': False, 'reason': 'hf_transfer package is not installed'}); ({'HF_HUB_ENABLE_HF_TRANSFER': '1'}, {'requested': True, 'enabled': True, 'reason': 'enabled'})

Exceptions and behavior

Function _fast_transfer_env calls find_spec; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L282-L297.

vllm_mlx.model_workflow.inspect_model · function

vllm_mlx.model_workflow.inspect_model(model: str, *, revision: str | None = None, local_files_only: bool = False) -> dict[str, Any]

Inspect a local model path or Hugging Face model id without loading weights.

Parameters

Name	Type	Required	Default	Description
model	str	yes	none	Required positional or keyword input.
revision	str \| None	no	None	Optional keyword-only input; defaults to None.
local_files_only	bool	no	False	Optional keyword-only input; defaults to False.

Returns

• Type: dict[str, Any]
• Direct return expressions: {'model': model, 'source': source, 'location': location, 'revision': resolved_revision or revision, 'inspected_at': _no…

Exceptions and behavior

Function inspect_model calls Path(model).expanduser, Path, model_path.exists, _local_file_inventory; can raise ValueError; returns {'model': model, 'source': source, 'location': location, 'revision': resolved_revision or revision, 'inspected_at': _no…. Directly raised exceptions: ValueError.

View source #L300-L366.

vllm_mlx.model_workflow.acquire_model · function

vllm_mlx.model_workflow.acquire_model(model_id: str, *, options: AcquisitionOptions | None = None) -> dict[str, Any]

Download a model repository and write a finalized artifact manifest.

Parameters

Name	Type	Required	Default	Description
model_id	str	yes	none	Required positional or keyword input.
options	AcquisitionOptions \| None	no	None	Optional keyword-only input; defaults to None.

Returns

• Type: dict[str, Any]
• Direct return expressions: manifest

Exceptions and behavior

Function acquire_model calls AcquisitionOptions, _is_model_id, ValueError, _fast_transfer_env; can raise ValueError, FileExistsError; returns manifest. Directly raised exceptions: ValueError, FileExistsError.

View source #L369-L446.

vllm_mlx.model_workflow._conversion_command · function

vllm_mlx.model_workflow._conversion_command(options: ConversionOptions) -> list[str]

Function _conversion_command calls command.append, command.extend, str; returns command.

Parameters

Name	Type	Required	Default	Description
options	ConversionOptions	yes	none	Required positional or keyword input.

Returns

• Type: list[str]
• Direct return expressions: command

Exceptions and behavior

Function _conversion_command calls command.append, command.extend, str; returns command. No direct raise statement appears in this definition.

View source #L449-L474.

vllm_mlx.model_workflow.convert_model · function

vllm_mlx.model_workflow.convert_model(options: ConversionOptions) -> dict[str, Any]

Run mlx-lm conversion and record the exact recipe.

Parameters

Name	Type	Required	Default	Description
options	ConversionOptions	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: result

Exceptions and behavior

Function convert_model calls _conversion_command, _now_iso, inspect_model, sys.version.split; returns result. No direct raise statement appears in this definition.

View source #L477-L525.

vllm_mlx.model_workflow._existing_manifests · function

vllm_mlx.model_workflow._existing_manifests(path: Path) -> dict[str, Any]

Function _existing_manifests calls manifest_path.exists, str, _read_json; returns manifests.

Parameters

Name	Type	Required	Default	Description
path	Path	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: manifests

Exceptions and behavior

Function _existing_manifests calls manifest_path.exists, str, _read_json; returns manifests. No direct raise statement appears in this definition.

View source #L528-L540.

vllm_mlx.model_workflow._drop_none · function

vllm_mlx.model_workflow._drop_none(payload: dict[str, Any]) -> dict[str, Any]

Function _drop_none calls payload.items; returns {key: value for key, value in payload.items() if value is not None}.

Parameters

Name	Type	Required	Default	Description
payload	dict[str, Any]	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: {key: value for key, value in payload.items() if value is not None}

Exceptions and behavior

Function _drop_none calls payload.items; returns {key: value for key, value in payload.items() if value is not None}. No direct raise statement appears in this definition.

View source #L543-L544.

vllm_mlx.model_workflow.register_model · function

vllm_mlx.model_workflow.register_model(options: RegistrationOptions) -> dict[str, Any]

Write a portable registration manifest for a finalized local artifact.

Parameters

Name	Type	Required	Default	Description
options	RegistrationOptions	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: payload

Exceptions and behavior

Function register_model calls Path(options.artifact_path).expanduser, Path, artifact.exists, FileNotFoundError; can raise FileNotFoundError, NotADirectoryError; returns payload. Directly raised exceptions: FileNotFoundError, NotADirectoryError.

View source #L547-L603.

vllm_mlx.model_workflow._qualification_command · function

vllm_mlx.model_workflow._qualification_command(options: QualificationOptions) -> list[str]

Function _qualification_command calls command.extend, str; returns command.

Parameters

Name	Type	Required	Default	Description
options	QualificationOptions	yes	none	Required positional or keyword input.

Returns

• Type: list[str]
• Direct return expressions: command

Exceptions and behavior

Function _qualification_command calls command.extend, str; returns command. No direct raise statement appears in this definition.

View source #L606-L627.

vllm_mlx.model_workflow.qualify_model · function

vllm_mlx.model_workflow.qualify_model(options: QualificationOptions) -> dict[str, Any]

Create or run a bench-serve qualification handoff.

Parameters

Name	Type	Required	Default	Description
options	QualificationOptions	yes	none	Required positional or keyword input.

Returns

• Type: dict[str, Any]
• Direct return expressions: payload

Exceptions and behavior

Function qualify_model calls _qualification_command, _now_iso, subprocess.run, Path(options.output_path).expanduser; returns payload. No direct raise statement appears in this definition.

View source #L630-L661.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
AcquisitionOptions	class	AcquisitionOptions(revision: str \| None = None, target_dir: str \| None = None, staging_dir: str \| None = None, is_mllm: bool = False, fast_transfer: bool = True, local_files_only: bool = False)	Options for Hugging Face model acquisition.	#L38-L46
ConversionOptions	class	ConversionOptions(source_path: str, output_path: str, quantize: bool = False, q_bits: int \| None = None, q_group_size: int \| None = None, q_mode: str \| None = None, quant_predicate: str \| None = None, dtype: str \| None = None, trust_remote_code: bool = False, dry_run: bool = False)	Options for the mlx-lm conversion backend.	#L50-L62
RegistrationOptions	class	RegistrationOptions(artifact_path: str, model_id: str \| None = None, served_model_name: str \| None = None, preset_alias: str \| None = None, output_path: str \| None = None, mllm: bool \| None = None, tool_call_parser: str \| None = None, reasoning_parser: str \| None = None, default_temperature: float \| None = None, default_top_p: float \| None = None, default_top_k: int \| None = None, default_min_p: float \| None = None, default_presence_penalty: float \| None = None, default_repetition_penalty: float \| None = None, chat_template_kwargs: dict[str, Any] \| None = None, feature_flags: list[str] \| None = None)	Options for generating a portable model registration manifest.	#L66-L84
QualificationOptions	class	QualificationOptions(model_id: str, server_url: str = 'http://127.0.0.1:8080', workload_path: str \| None = None, output_path: str \| None = None, result_path: str \| None = None, repetitions: int \| None = None, dry_run: bool = False, extra_args: list[str] \| None = None)	Options for creating or running a bench-serve qualification handoff.	#L88-L98
_now_iso	function	_now_iso() -> str	Function _now_iso calls datetime.now(timezone.utc).isoformat, datetime.now; returns datetime.now(timezone.utc).isoformat().	#L101-L102
_bytes_to_gb	function	_bytes_to_gb(size: int \| float \| None) -> float \| None	Function _bytes_to_gb calls round, float; has 2 explicit return paths.	#L105-L108
_read_json	function	_read_json(path: Path) -> dict[str, Any]	Function _read_json calls json.loads, path.read_text; has 2 explicit return paths.	#L111-L115
_write_json	function	_write_json(path: Path, payload: dict[str, Any]) -> None	Function _write_json calls path.parent.mkdir, path.write_text, json.dumps.	#L118-L120
_local_file_inventory	function	_local_file_inventory(path: Path) -> tuple[list[dict[str, Any]], int]	Function _local_file_inventory calls sorted, path.rglob, item.is_file, item.stat; returns (files, total).	#L123-L135
_hf_file_inventory	function	_hf_file_inventory(model_id: str, *, revision: str \| None, local_files_only: bool) -> tuple[list[dict[str, Any]], int \| None, str \| None]	Function _hf_file_inventory calls HfApi().model_info, HfApi, getattr, int; has 2 explicit return paths.	#L138-L158
_hf_config	function	_hf_config(model_id: str, *, revision: str \| None, local_files_only: bool) -> dict[str, Any]	Function _hf_config calls hf_hub_download, _read_json, Path; returns _read_json(Path(config_path)).	#L161-L170
_config_value	function	_config_value(config: dict[str, Any], key: str) -> Any	Function _config_value calls config.get, isinstance, text_config.get; has 3 explicit return paths.	#L173-L179
_model_family	function	_model_family(config: dict[str, Any]) -> dict[str, Any]	Function _model_family calls _config_value, isinstance, config.get; returns {'model_type': _config_value(config, 'model_type'), 'architectures': architectures, 'torch_dtype': _config_value(config….	#L182-L201
_estimate_fit	function	_estimate_fit(*, total_bytes: int \| None, model_files_bytes: int \| None, config: dict[str, Any]) -> dict[str, Any]	Function _estimate_fit calls _model_family(config).get, _model_family, isinstance, warnings.append; returns {'download_size_gb': _bytes_to_gb(total_bytes), 'model_file_size_gb': _bytes_to_gb(model_files_bytes), 'estimated_conve….	#L204-L231
_model_file_bytes	function	_model_file_bytes(files: list[dict[str, Any]]) -> int \| None	Function _model_file_bytes calls str, entry.get, path.endswith, int; has 2 explicit return paths.	#L234-L246
_is_mlx_quantization	function	_is_mlx_quantization(quant: Any) -> bool	Return True only when quant looks like an mlx-lm quantization config.	#L252-L265
_looks_like_mlx_name	function	_looks_like_mlx_name(model: str, *, source: str) -> bool	Function _looks_like_mlx_name calls model.lower, Path(model).name.lower, Path, name.startswith; returns name.startswith('mlx-community/') or '-mlx' in name or '_mlx' in name or name.endswith('mlx').	#L268-L275
_is_model_id	function	_is_model_id(value: str) -> bool	Function _is_model_id calls bool, _MODEL_ID_RE.fullmatch; returns bool(_MODEL_ID_RE.fullmatch(value)).	#L278-L279
_fast_transfer_env	function	_fast_transfer_env(requested: bool) -> tuple[dict[str, str], dict[str, Any]]	Function _fast_transfer_env calls find_spec; has 3 explicit return paths.	#L282-L297
inspect_model	function	inspect_model(model: str, *, revision: str \| None = None, local_files_only: bool = False) -> dict[str, Any]	Inspect a local model path or Hugging Face model id without loading weights.	#L300-L366
acquire_model	function	acquire_model(model_id: str, *, options: AcquisitionOptions \| None = None) -> dict[str, Any]	Download a model repository and write a finalized artifact manifest.	#L369-L446
_conversion_command	function	_conversion_command(options: ConversionOptions) -> list[str]	Function _conversion_command calls command.append, command.extend, str; returns command.	#L449-L474
convert_model	function	convert_model(options: ConversionOptions) -> dict[str, Any]	Run mlx-lm conversion and record the exact recipe.	#L477-L525
_existing_manifests	function	_existing_manifests(path: Path) -> dict[str, Any]	Function _existing_manifests calls manifest_path.exists, str, _read_json; returns manifests.	#L528-L540
_drop_none	function	_drop_none(payload: dict[str, Any]) -> dict[str, Any]	Function _drop_none calls payload.items; returns {key: value for key, value in payload.items() if value is not None}.	#L543-L544
register_model	function	register_model(options: RegistrationOptions) -> dict[str, Any]	Write a portable registration manifest for a finalized local artifact.	#L547-L603
_qualification_command	function	_qualification_command(options: QualificationOptions) -> list[str]	Function _qualification_command calls command.extend, str; returns command.	#L606-L627
qualify_model	function	qualify_model(options: QualificationOptions) -> dict[str, Any]	Create or run a bench-serve qualification handoff.	#L630-L661