Skip to content

vllm_mlx.plugin

vLLM Platform Plugin for MLX.

View the complete module source at #L1-L155.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.plugin

vLLM Platform Plugin for MLX.

This module provides the entry point for vLLM's platform plugin system, enabling automatic detection and activation of the MLX platform on Apple Silicon Macs.

vllm_mlx.plugin.logger module-attribute 

logger = logging.getLogger(__name__)

vllm_mlx.plugin.mlx_platform_plugin 

mlx_platform_plugin() -> str | None

Platform plugin entry point for vLLM.

This function is called by vLLM's platform detection system to determine if the MLX platform should be activated.

Returns:

  • str ( str | None ) –

    Fully qualified class name of MLXPlatform if conditions are met

  • None ( str | None ) –

    If MLX platform should not be activated

Source code in vllm_mlx/plugin.py 
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def mlx_platform_plugin() -> str | None:
    """
    Platform plugin entry point for vLLM.

    This function is called by vLLM's platform detection system to
    determine if the MLX platform should be activated.

    Returns:
        str: Fully qualified class name of MLXPlatform if conditions are met
        None: If MLX platform should not be activated
    """
    logger.debug("Checking if MLX platform is available")

    # Check if running on macOS
    if sys.platform != "darwin":
        logger.debug("MLX platform not available: not running on macOS")
        return None

    # Check if running on Apple Silicon (ARM)
    if platform.machine() != "arm64":
        logger.debug("MLX platform not available: not running on Apple Silicon")
        return None

    # Check if MLX is installed and working
    try:
        import mlx.core as mx

        # Verify MLX can actually run
        test_array = mx.array([1.0, 2.0, 3.0])
        _ = mx.sum(test_array)

        # Check default device
        default_device = mx.default_device()
        logger.debug(f"MLX default device: {default_device}")

    except ImportError:
        logger.debug("MLX platform not available: mlx not installed")
        return None
    except Exception as e:
        logger.debug(f"MLX platform not available: {e}")
        return None

    # Check if mlx-lm is available
    try:
        import mlx_lm

        logger.debug(f"mlx-lm version: {getattr(mlx_lm, '__version__', 'unknown')}")
    except ImportError:
        logger.warning("mlx-lm not installed. Install with: pip install mlx-lm")
        # Still allow MLX platform, but functionality will be limited
        pass

    logger.info("MLX platform is available on Apple Silicon")
    return "vllm_mlx.vllm_platform.MLXPlatform"

vllm_mlx.plugin.is_mlx_available 

is_mlx_available() -> bool

Check if MLX platform can be used.

Returns:

  • bool ( bool ) –

    True if MLX is available and working

Source code in vllm_mlx/plugin.py 
73
74
75
76
77
78
79
80
def is_mlx_available() -> bool:
    """
    Check if MLX platform can be used.

    Returns:
        bool: True if MLX is available and working
    """
    return mlx_platform_plugin() is not None

vllm_mlx.plugin.get_mlx_device_info 

get_mlx_device_info() -> dict

Get information about the MLX device.

Returns:

  • dict ( dict ) –

    Device information including chip name, memory, etc.

Source code in vllm_mlx/plugin.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
def get_mlx_device_info() -> dict:
    """
    Get information about the MLX device.

    Returns:
        dict: Device information including chip name, memory, etc.
    """
    import subprocess

    info = {
        "platform": "mlx",
        "available": False,
        "chip_name": "Unknown",
        "memory_gb": 0,
        "mlx_version": "Unknown",
        "mlx_lm_version": "Unknown",
        "mlx_vlm_version": "Unknown",
    }

    if not is_mlx_available():
        return info

    info["available"] = True

    # Get chip name
    try:
        result = subprocess.run(
            ["sysctl", "-n", "machdep.cpu.brand_string"],
            capture_output=True,
            text=True,
            check=True,
        )
        info["chip_name"] = result.stdout.strip()
    except Exception:
        pass

    # Get memory
    try:
        result = subprocess.run(
            ["sysctl", "-n", "hw.memsize"],
            capture_output=True,
            text=True,
            check=True,
        )
        info["memory_gb"] = int(result.stdout.strip()) / (1024**3)
    except Exception:
        pass

    # Get MLX version
    try:
        import mlx

        info["mlx_version"] = getattr(mlx, "__version__", "Unknown")
    except Exception:
        pass

    # Get mlx-lm version
    try:
        import mlx_lm

        info["mlx_lm_version"] = getattr(mlx_lm, "__version__", "Unknown")
    except Exception:
        pass

    # Get mlx-vlm version
    try:
        import mlx_vlm

        info["mlx_vlm_version"] = getattr(mlx_vlm, "__version__", "Unknown")
    except Exception:
        pass

    return info

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.plugin.mlx_platform_plugin · function 
vllm_mlx.plugin.mlx_platform_plugin() -> str | None

Platform plugin entry point for vLLM.

Parameters

This callable has no explicit inputs.

Returns

  • Type: str | None
  • Direct return expressions: None; 'vllm_mlx.vllm_platform.MLXPlatform'

Exceptions and behavior

Function mlx_platform_plugin calls logger.debug, platform.machine, mx.array, mx.sum; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L17-L70.

vllm_mlx.plugin.is_mlx_available · function 
vllm_mlx.plugin.is_mlx_available() -> bool

Check if MLX platform can be used.

Parameters

This callable has no explicit inputs.

Returns

  • Type: bool
  • Direct return expressions: mlx_platform_plugin() is not None

Exceptions and behavior

Function is_mlx_available calls mlx_platform_plugin; returns mlx_platform_plugin() is not None. No direct raise statement appears in this definition.

View source #L73-L80.

vllm_mlx.plugin.get_mlx_device_info · function 
vllm_mlx.plugin.get_mlx_device_info() -> dict

Get information about the MLX device.

Parameters

This callable has no explicit inputs.

Returns

  • Type: dict
  • Direct return expressions: info

Exceptions and behavior

Function get_mlx_device_info calls is_mlx_available, subprocess.run, result.stdout.strip, int; returns info. No direct raise statement appears in this definition.

View source #L83-L155.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
mlx_platform_plugin function mlx_platform_plugin() -> str \| None Platform plugin entry point for vLLM. #L17-L70
is_mlx_available function is_mlx_available() -> bool Check if MLX platform can be used. #L73-L80
get_mlx_device_info function get_mlx_device_info() -> dict Get information about the MLX device. #L83-L155