Skip to content

vllm_mlx.worker

MLX Worker for vLLM.

View the complete module source at #L1-L278.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.worker

MLX Worker for vLLM.

This module implements a vLLM worker that uses Apple's MLX framework for model execution on Apple Silicon.

vllm_mlx.worker.logger module-attribute 

logger = logging.getLogger(__name__)

vllm_mlx.worker.MLXWorker 

MLXWorker(vllm_config: VllmConfig, local_rank: int, rank: int, distributed_init_method: str, is_driver_worker: bool = False)

Worker implementation for MLX-based inference on Apple Silicon.

This worker uses mlx-lm for model loading and inference, providing native Apple Silicon GPU acceleration through Metal.

Unlike CUDA workers that use PyTorch with CUDA, this worker: - Uses MLX arrays instead of PyTorch tensors for model weights - Leverages unified memory (no CPU<->GPU transfers needed) - Uses Metal-optimized kernels for attention and other operations

Initialize MLX worker.

Parameters:

  • vllm_config (VllmConfig) –

    Complete vLLM configuration

  • local_rank (int) –

    Local device index (usually 0 for single GPU)

  • rank (int) –

    Global rank in distributed setup

  • distributed_init_method (str) –

    Distributed initialization method

  • is_driver_worker (bool, default: False ) –

    Whether this worker handles driver responsibilities

Source code in vllm_mlx/worker.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
def __init__(
    self,
    vllm_config: "VllmConfig",
    local_rank: int,
    rank: int,
    distributed_init_method: str,
    is_driver_worker: bool = False,
) -> None:
    """
    Initialize MLX worker.

    Args:
        vllm_config: Complete vLLM configuration
        local_rank: Local device index (usually 0 for single GPU)
        rank: Global rank in distributed setup
        distributed_init_method: Distributed initialization method
        is_driver_worker: Whether this worker handles driver responsibilities
    """
    self.vllm_config = vllm_config
    self.model_config = vllm_config.model_config
    self.cache_config = vllm_config.cache_config
    self.parallel_config = vllm_config.parallel_config
    self.scheduler_config = vllm_config.scheduler_config
    self.device_config = vllm_config.device_config
    self.load_config = vllm_config.load_config

    self.local_rank = local_rank
    self.rank = rank
    self.distributed_init_method = distributed_init_method
    self.is_driver_worker = is_driver_worker

    # MLX model and tokenizer
    self.model = None
    self.tokenizer = None
    self.model_runner = None

    # Device info
    self.device = torch.device("cpu")  # MLX uses its own device management

    logger.info(f"Initializing MLX Worker (rank={rank}, local_rank={local_rank})")

vllm_mlx.worker.MLXWorker.vllm_config instance-attribute 

vllm_config = vllm_config

vllm_mlx.worker.MLXWorker.model_config instance-attribute 

model_config = vllm_config.model_config

vllm_mlx.worker.MLXWorker.cache_config instance-attribute 

cache_config = vllm_config.cache_config

vllm_mlx.worker.MLXWorker.parallel_config instance-attribute 

parallel_config = vllm_config.parallel_config

vllm_mlx.worker.MLXWorker.scheduler_config instance-attribute 

scheduler_config = vllm_config.scheduler_config

vllm_mlx.worker.MLXWorker.device_config instance-attribute 

device_config = vllm_config.device_config

vllm_mlx.worker.MLXWorker.load_config instance-attribute 

load_config = vllm_config.load_config

vllm_mlx.worker.MLXWorker.local_rank instance-attribute 

local_rank = local_rank

vllm_mlx.worker.MLXWorker.rank instance-attribute 

rank = rank

vllm_mlx.worker.MLXWorker.distributed_init_method instance-attribute 

distributed_init_method = distributed_init_method

vllm_mlx.worker.MLXWorker.is_driver_worker instance-attribute 

is_driver_worker = is_driver_worker

vllm_mlx.worker.MLXWorker.model instance-attribute 

model = None

vllm_mlx.worker.MLXWorker.tokenizer instance-attribute 

tokenizer = None

vllm_mlx.worker.MLXWorker.model_runner instance-attribute 

model_runner = None

vllm_mlx.worker.MLXWorker.device instance-attribute 

device = torch.device('cpu')

vllm_mlx.worker.MLXWorker.vocab_size property 

vocab_size: int

Get vocabulary size.

vllm_mlx.worker.MLXWorker.init_device 

init_device() -> None

Initialize MLX device and verify it's working.

Source code in vllm_mlx/worker.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def init_device(self) -> None:
    """Initialize MLX device and verify it's working."""
    try:
        import mlx.core as mx

        # Verify MLX is using GPU
        default_device = mx.default_device()
        logger.info(f"MLX default device: {default_device}")

        # Get device info
        from vllm_mlx.plugin import get_mlx_device_info

        info = get_mlx_device_info()
        logger.info(
            f"MLX Device: {info['chip_name']} with {info['memory_gb']:.1f}GB"
        )

        # Initialize model runner
        from vllm_mlx.model_runner import MLXModelRunner

        self.model_runner = MLXModelRunner(self.vllm_config)

    except ImportError as e:
        raise ImportError(
            f"MLX is required for MLXWorker: {e}. "
            "Install with: pip install mlx mlx-lm"
        )

vllm_mlx.worker.MLXWorker.load_model 

load_model() -> None

Load model using mlx-lm.

Source code in vllm_mlx/worker.py 
105
106
107
108
109
110
111
def load_model(self) -> None:
    """Load model using mlx-lm."""
    if self.model_runner is None:
        raise RuntimeError("init_device() must be called before load_model()")

    self.model_runner.load_model()
    logger.info(f"Model loaded: {self.model_config.model}")

vllm_mlx.worker.MLXWorker.determine_available_memory 

determine_available_memory() -> int

Determine available memory for KV cache.

On Apple Silicon with unified memory, we use a portion of system RAM.

Source code in vllm_mlx/worker.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
def determine_available_memory(self) -> int:
    """
    Determine available memory for KV cache.

    On Apple Silicon with unified memory, we use a portion of system RAM.
    """
    import subprocess

    try:
        # Get total system memory
        result = subprocess.run(
            ["sysctl", "-n", "hw.memsize"],
            capture_output=True,
            text=True,
            check=True,
        )
        total_memory = int(result.stdout.strip())

        # Use configured GPU memory utilization
        utilization = self.cache_config.gpu_memory_utilization
        available = int(total_memory * utilization * 0.5)  # Be conservative

        logger.info(
            f"Available memory for KV cache: {available / (1024**3):.2f}GB "
            f"(utilization: {utilization})"
        )
        return available

    except Exception as e:
        logger.warning(f"Could not determine memory: {e}, using default 4GB")
        return 4 * 1024 * 1024 * 1024  # 4GB default

vllm_mlx.worker.MLXWorker.initialize_cache 

initialize_cache(num_gpu_blocks: int, num_cpu_blocks: int) -> None

Initialize KV cache with the given size.

Source code in vllm_mlx/worker.py 
145
146
147
148
149
150
151
152
153
def initialize_cache(self, num_gpu_blocks: int, num_cpu_blocks: int) -> None:
    """Initialize KV cache with the given size."""
    self.cache_config.num_gpu_blocks = num_gpu_blocks
    self.cache_config.num_cpu_blocks = num_cpu_blocks

    if self.model_runner:
        self.model_runner.initialize_cache(num_gpu_blocks)

    logger.info(f"Initialized cache: {num_gpu_blocks} GPU blocks")

vllm_mlx.worker.MLXWorker.get_kv_cache_spec 

get_kv_cache_spec() -> dict

Get KV cache specification.

Source code in vllm_mlx/worker.py 
155
156
157
158
159
def get_kv_cache_spec(self) -> dict:
    """Get KV cache specification."""
    if self.model_runner:
        return self.model_runner.get_kv_cache_spec()
    return {}

vllm_mlx.worker.MLXWorker.compile_or_warm_up_model 

compile_or_warm_up_model() -> None

Warm up model for inference.

Source code in vllm_mlx/worker.py 
161
162
163
164
165
def compile_or_warm_up_model(self) -> None:
    """Warm up model for inference."""
    if self.model_runner:
        self.model_runner.warm_up()
    logger.info("Model warm-up complete")

vllm_mlx.worker.MLXWorker.execute_model 

execute_model(scheduler_output: SchedulerOutput) -> ModelRunnerOutput | None

Execute model inference for the given scheduler output.

Parameters:

  • scheduler_output (SchedulerOutput) –

    Contains requests to process

Returns:

  • ModelRunnerOutput | None –

    ModelRunnerOutput with generation results

Source code in vllm_mlx/worker.py 
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
def execute_model(
    self,
    scheduler_output: "SchedulerOutput",
) -> "ModelRunnerOutput | None":
    """
    Execute model inference for the given scheduler output.

    Args:
        scheduler_output: Contains requests to process

    Returns:
        ModelRunnerOutput with generation results
    """
    if self.model_runner is None:
        raise RuntimeError("Model not loaded")

    return self.model_runner.execute_model(scheduler_output)

vllm_mlx.worker.MLXWorker.get_model 

get_model()

Get the underlying model.

Source code in vllm_mlx/worker.py 
185
186
187
188
189
def get_model(self):
    """Get the underlying model."""
    if self.model_runner:
        return self.model_runner.model
    return None

vllm_mlx.worker.MLXWorker.check_health 

check_health() -> None

Check worker health.

Source code in vllm_mlx/worker.py 
191
192
193
194
195
196
197
198
199
200
def check_health(self) -> None:
    """Check worker health."""
    try:
        import mlx.core as mx

        # Simple check - create and evaluate a small array
        test = mx.array([1.0, 2.0, 3.0])
        _ = mx.sum(test).item()
    except Exception as e:
        raise RuntimeError(f"MLX health check failed: {e}")

vllm_mlx.worker.MLXWorker.shutdown 

shutdown() -> None

Clean up resources.

Source code in vllm_mlx/worker.py 
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
def shutdown(self) -> None:
    """Clean up resources."""
    logger.info("Shutting down MLX Worker")

    # Clear model
    self.model = None
    self.tokenizer = None
    self.model_runner = None

    # Clear MLX cache
    try:
        import mlx.core as mx

        mx.clear_cache()
    except Exception:
        pass

    gc.collect()

vllm_mlx.worker.MLXWorker.add_lora 

add_lora(lora_request) -> bool

Report that dynamically adding a LoRA adapter is unsupported.

Source code in vllm_mlx/worker.py 
222
223
224
225
226
def add_lora(self, lora_request) -> bool:
    """Report that dynamically adding a LoRA adapter is unsupported."""

    logger.warning("LoRA not yet supported on MLX backend")
    return False

vllm_mlx.worker.MLXWorker.remove_lora 

remove_lora(lora_id: int) -> bool

Report that dynamically removing a LoRA adapter is unsupported.

Source code in vllm_mlx/worker.py 
228
229
230
231
def remove_lora(self, lora_id: int) -> bool:
    """Report that dynamically removing a LoRA adapter is unsupported."""

    return False

vllm_mlx.worker.MLXWorker.pin_lora 

pin_lora(lora_id: int) -> bool

Report that pinning a LoRA adapter is unsupported.

Source code in vllm_mlx/worker.py 
233
234
235
236
def pin_lora(self, lora_id: int) -> bool:
    """Report that pinning a LoRA adapter is unsupported."""

    return False

vllm_mlx.worker.MLXWorker.list_loras 

list_loras() -> set[int]

Return the empty set because runtime LoRA adapters are unsupported.

Source code in vllm_mlx/worker.py 
238
239
240
241
def list_loras(self) -> set[int]:
    """Return the empty set because runtime LoRA adapters are unsupported."""

    return set()

vllm_mlx.worker.MLXWorker.sleep 

sleep(level: int = 1) -> None

Leave the worker active because MLX unified memory has no sleep mode.

Source code in vllm_mlx/worker.py 
244
245
246
247
def sleep(self, level: int = 1) -> None:
    """Leave the worker active because MLX unified memory has no sleep mode."""

    logger.debug("Sleep mode not applicable for MLX (unified memory)")

vllm_mlx.worker.MLXWorker.wake_up 

wake_up(tags: list[str] | None = None) -> None

Perform no work because the MLX worker never enters sleep mode.

Source code in vllm_mlx/worker.py 
249
250
251
252
def wake_up(self, tags: list[str] | None = None) -> None:
    """Perform no work because the MLX worker never enters sleep mode."""

    logger.debug("Wake up not applicable for MLX (unified memory)")

vllm_mlx.worker.MLXWorker.get_cache_block_size_bytes 

get_cache_block_size_bytes() -> int

Get size of a cache block in bytes.

Source code in vllm_mlx/worker.py 
259
260
261
262
263
264
265
266
267
268
269
270
271
def get_cache_block_size_bytes(self) -> int:
    """Get size of a cache block in bytes."""
    if self.model_runner:
        return self.model_runner.get_cache_block_size_bytes()

    # Default calculation
    head_size = self.model_config.get_head_size()
    num_heads = self.model_config.get_num_kv_heads(self.parallel_config)
    num_layers = self.model_config.get_num_layers(self.parallel_config)

    # 2 for K and V, assuming float16
    block_size = self.cache_config.block_size
    return 2 * block_size * num_layers * num_heads * head_size * 2

vllm_mlx.worker.MLXWorker.profile 

profile(is_start: bool = True) -> None

Profiling (not yet implemented for MLX).

Source code in vllm_mlx/worker.py 
273
274
275
def profile(self, is_start: bool = True) -> None:
    """Profiling (not yet implemented for MLX)."""
    logger.debug("Profiling not yet implemented for MLX")

vllm_mlx.worker.MLXWorker.__repr__ 

__repr__() -> str
Source code in vllm_mlx/worker.py 
277
278
def __repr__(self) -> str:
    return f"<MLXWorker rank={self.rank} local_rank={self.local_rank}>"

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.worker.MLXWorker · class 
vllm_mlx.worker.MLXWorker(vllm_config: 'VllmConfig', local_rank: int, rank: int, distributed_init_method: str, is_driver_worker: bool = False)

Worker implementation for MLX-based inference on Apple Silicon.

Parameters

Name Type Required Default Description
vllm_config 'VllmConfig' yes none Complete vLLM configuration
local_rank int yes none Local device index (usually 0 for single GPU)
rank int yes none Global rank in distributed setup
distributed_init_method str yes none Distributed initialization method
is_driver_worker bool no False Whether this worker handles driver responsibilities

Returns

  • Constructs: vllm_mlx.worker.MLXWorker

Exceptions and behavior

Class MLXWorker declares 21 direct member(s). No direct raise statement appears in this definition.

View source #L23-L278.

vllm_mlx.worker.MLXWorker.__init__ · method 
vllm_mlx.worker.MLXWorker.__init__(vllm_config: 'VllmConfig', local_rank: int, rank: int, distributed_init_method: str, is_driver_worker: bool = False) -> None

Initialize MLX worker.

Parameters

Name Type Required Default Description
vllm_config 'VllmConfig' yes none Complete vLLM configuration
local_rank int yes none Local device index (usually 0 for single GPU)
rank int yes none Global rank in distributed setup
distributed_init_method str yes none Distributed initialization method
is_driver_worker bool no False Whether this worker handles driver responsibilities

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.__init__ updates self.vllm_config, self.model_config, self.cache_config, self.parallel_config; calls torch.device, logger.info. No direct raise statement appears in this definition.

View source #L36-L75.

vllm_mlx.worker.MLXWorker.init_device · method 
vllm_mlx.worker.MLXWorker.init_device() -> None

Initialize MLX device and verify it's working.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.init_device updates self.model_runner; calls mx.default_device, logger.info, get_mlx_device_info, MLXModelRunner; can raise ImportError. Directly raised exceptions: ImportError.

View source #L77-L103.

vllm_mlx.worker.MLXWorker.load_model · method 
vllm_mlx.worker.MLXWorker.load_model() -> None

Load model using mlx-lm.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.load_model calls RuntimeError, self.model_runner.load_model, logger.info; can raise RuntimeError. Directly raised exceptions: RuntimeError.

View source #L105-L111.

vllm_mlx.worker.MLXWorker.determine_available_memory · method 
vllm_mlx.worker.MLXWorker.determine_available_memory() -> int

Determine available memory for KV cache.

Parameters

This callable has no explicit inputs.

Returns

  • Type: int
  • Direct return expressions: available; 4 * 1024 * 1024 * 1024

Exceptions and behavior

Method MLXWorker.determine_available_memory calls subprocess.run, int, result.stdout.strip, logger.info; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L113-L143.

vllm_mlx.worker.MLXWorker.initialize_cache · method 
vllm_mlx.worker.MLXWorker.initialize_cache(num_gpu_blocks: int, num_cpu_blocks: int) -> None

Initialize KV cache with the given size.

Parameters

Name Type Required Default Description
num_gpu_blocks int yes none Required positional or keyword input.
num_cpu_blocks int yes none Required positional or keyword input.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.initialize_cache updates self.cache_config.num_gpu_blocks, self.cache_config.num_cpu_blocks; calls self.model_runner.initialize_cache, logger.info. No direct raise statement appears in this definition.

View source #L145-L153.

vllm_mlx.worker.MLXWorker.get_kv_cache_spec · method 
vllm_mlx.worker.MLXWorker.get_kv_cache_spec() -> dict

Get KV cache specification.

Parameters

This callable has no explicit inputs.

Returns

  • Type: dict
  • Direct return expressions: self.model_runner.get_kv_cache_spec(); {}

Exceptions and behavior

Method MLXWorker.get_kv_cache_spec calls self.model_runner.get_kv_cache_spec; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L155-L159.

vllm_mlx.worker.MLXWorker.compile_or_warm_up_model · method 
vllm_mlx.worker.MLXWorker.compile_or_warm_up_model() -> None

Warm up model for inference.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.compile_or_warm_up_model calls self.model_runner.warm_up, logger.info. No direct raise statement appears in this definition.

View source #L161-L165.

vllm_mlx.worker.MLXWorker.execute_model · method 
vllm_mlx.worker.MLXWorker.execute_model(scheduler_output: 'SchedulerOutput') -> 'ModelRunnerOutput | None'

Execute model inference for the given scheduler output.

Parameters

Name Type Required Default Description
scheduler_output 'SchedulerOutput' yes none Contains requests to process

Returns

  • Type: 'ModelRunnerOutput | None'
  • Direct return expressions: self.model_runner.execute_model(scheduler_output)

Exceptions and behavior

Method MLXWorker.execute_model calls RuntimeError, self.model_runner.execute_model; can raise RuntimeError; returns self.model_runner.execute_model(scheduler_output). Directly raised exceptions: RuntimeError.

View source #L167-L183.

vllm_mlx.worker.MLXWorker.get_model · method 
vllm_mlx.worker.MLXWorker.get_model() -> not annotated

Get the underlying model.

Parameters

This callable has no explicit inputs.

Returns

  • Type: not annotated
  • Direct return expressions: self.model_runner.model; None

Exceptions and behavior

Method MLXWorker.get_model has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L185-L189.

vllm_mlx.worker.MLXWorker.check_health · method 
vllm_mlx.worker.MLXWorker.check_health() -> None

Check worker health.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.check_health calls mx.array, mx.sum(test).item, mx.sum, RuntimeError; can raise RuntimeError. Directly raised exceptions: RuntimeError.

View source #L191-L200.

vllm_mlx.worker.MLXWorker.shutdown · method 
vllm_mlx.worker.MLXWorker.shutdown() -> None

Clean up resources.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.shutdown updates self.model, self.tokenizer, self.model_runner; calls logger.info, mx.clear_cache, gc.collect. No direct raise statement appears in this definition.

View source #L202-L219.

vllm_mlx.worker.MLXWorker.add_lora · method 
vllm_mlx.worker.MLXWorker.add_lora(lora_request) -> bool

Report that dynamically adding a LoRA adapter is unsupported.

Parameters

Name Type Required Default Description
lora_request not annotated yes none Required positional or keyword input.

Returns

  • Type: bool
  • Direct return expressions: False

Exceptions and behavior

Method MLXWorker.add_lora calls logger.warning; returns False. No direct raise statement appears in this definition.

View source #L222-L226.

vllm_mlx.worker.MLXWorker.remove_lora · method 
vllm_mlx.worker.MLXWorker.remove_lora(lora_id: int) -> bool

Report that dynamically removing a LoRA adapter is unsupported.

Parameters

Name Type Required Default Description
lora_id int yes none Required positional or keyword input.

Returns

  • Type: bool
  • Direct return expressions: False

Exceptions and behavior

Method MLXWorker.remove_lora returns False. No direct raise statement appears in this definition.

View source #L228-L231.

vllm_mlx.worker.MLXWorker.pin_lora · method 
vllm_mlx.worker.MLXWorker.pin_lora(lora_id: int) -> bool

Report that pinning a LoRA adapter is unsupported.

Parameters

Name Type Required Default Description
lora_id int yes none Required positional or keyword input.

Returns

  • Type: bool
  • Direct return expressions: False

Exceptions and behavior

Method MLXWorker.pin_lora returns False. No direct raise statement appears in this definition.

View source #L233-L236.

vllm_mlx.worker.MLXWorker.list_loras · method 
vllm_mlx.worker.MLXWorker.list_loras() -> set[int]

Return the empty set because runtime LoRA adapters are unsupported.

Parameters

This callable has no explicit inputs.

Returns

  • Type: set[int]
  • Direct return expressions: set()

Exceptions and behavior

Method MLXWorker.list_loras calls set; returns set(). No direct raise statement appears in this definition.

View source #L238-L241.

vllm_mlx.worker.MLXWorker.sleep · method 
vllm_mlx.worker.MLXWorker.sleep(level: int = 1) -> None

Leave the worker active because MLX unified memory has no sleep mode.

Parameters

Name Type Required Default Description
level int no 1 Optional positional or keyword input; defaults to 1.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.sleep calls logger.debug. No direct raise statement appears in this definition.

View source #L244-L247.

vllm_mlx.worker.MLXWorker.wake_up · method 
vllm_mlx.worker.MLXWorker.wake_up(tags: list[str] | None = None) -> None

Perform no work because the MLX worker never enters sleep mode.

Parameters

Name Type Required Default Description
tags list[str] \| None no None Optional positional or keyword input; defaults to None.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.wake_up calls logger.debug. No direct raise statement appears in this definition.

View source #L249-L252.

vllm_mlx.worker.MLXWorker.vocab_size · method 
vllm_mlx.worker.MLXWorker.vocab_size() -> int

Get vocabulary size.

Parameters

This callable has no explicit inputs.

Returns

  • Type: int
  • Direct return expressions: self.model_config.get_vocab_size()

Exceptions and behavior

Method MLXWorker.vocab_size calls self.model_config.get_vocab_size; returns self.model_config.get_vocab_size(). No direct raise statement appears in this definition.

View source #L255-L257.

vllm_mlx.worker.MLXWorker.get_cache_block_size_bytes · method 
vllm_mlx.worker.MLXWorker.get_cache_block_size_bytes() -> int

Get size of a cache block in bytes.

Parameters

This callable has no explicit inputs.

Returns

  • Type: int
  • Direct return expressions: self.model_runner.get_cache_block_size_bytes(); 2 * block_size * num_layers * num_heads * head_size * 2

Exceptions and behavior

Method MLXWorker.get_cache_block_size_bytes calls self.model_runner.get_cache_block_size_bytes, self.model_config.get_head_size, self.model_config.get_num_kv_heads, self.model_config.get_num_layers; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L259-L271.

vllm_mlx.worker.MLXWorker.profile · method 
vllm_mlx.worker.MLXWorker.profile(is_start: bool = True) -> None

Profiling (not yet implemented for MLX).

Parameters

Name Type Required Default Description
is_start bool no True Optional positional or keyword input; defaults to True.

Returns

  • Type: None

Exceptions and behavior

Method MLXWorker.profile calls logger.debug. No direct raise statement appears in this definition.

View source #L273-L275.

vllm_mlx.worker.MLXWorker.__repr__ · method 
vllm_mlx.worker.MLXWorker.__repr__() -> str

Method MLXWorker.__repr__ returns f'<MLXWorker rank={self.rank} local_rank={self.local_rank}>'.

Parameters

This callable has no explicit inputs.

Returns

  • Type: str
  • Direct return expressions: f'<MLXWorker rank={self.rank} local_rank={self.local_rank}>'

Exceptions and behavior

Method MLXWorker.__repr__ returns f'<MLXWorker rank={self.rank} local_rank={self.local_rank}>'. No direct raise statement appears in this definition.

View source #L277-L278.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
MLXWorker class MLXWorker(vllm_config: 'VllmConfig', local_rank: int, rank: int, distributed_init_method: str, is_driver_worker: bool = False) Worker implementation for MLX-based inference on Apple Silicon. #L23-L278
MLXWorker.__init__ method MLXWorker.__init__(vllm_config: 'VllmConfig', local_rank: int, rank: int, distributed_init_method: str, is_driver_worker: bool = False) -> None Initialize MLX worker. #L36-L75
MLXWorker.init_device method MLXWorker.init_device() -> None Initialize MLX device and verify it's working. #L77-L103
MLXWorker.load_model method MLXWorker.load_model() -> None Load model using mlx-lm. #L105-L111
MLXWorker.determine_available_memory method MLXWorker.determine_available_memory() -> int Determine available memory for KV cache. #L113-L143
MLXWorker.initialize_cache method MLXWorker.initialize_cache(num_gpu_blocks: int, num_cpu_blocks: int) -> None Initialize KV cache with the given size. #L145-L153
MLXWorker.get_kv_cache_spec method MLXWorker.get_kv_cache_spec() -> dict Get KV cache specification. #L155-L159
MLXWorker.compile_or_warm_up_model method MLXWorker.compile_or_warm_up_model() -> None Warm up model for inference. #L161-L165
MLXWorker.execute_model method MLXWorker.execute_model(scheduler_output: 'SchedulerOutput') -> 'ModelRunnerOutput \| None' Execute model inference for the given scheduler output. #L167-L183
MLXWorker.get_model method MLXWorker.get_model() -> not annotated Get the underlying model. #L185-L189
MLXWorker.check_health method MLXWorker.check_health() -> None Check worker health. #L191-L200
MLXWorker.shutdown method MLXWorker.shutdown() -> None Clean up resources. #L202-L219
MLXWorker.add_lora method MLXWorker.add_lora(lora_request) -> bool Report that dynamically adding a LoRA adapter is unsupported. #L222-L226
MLXWorker.remove_lora method MLXWorker.remove_lora(lora_id: int) -> bool Report that dynamically removing a LoRA adapter is unsupported. #L228-L231
MLXWorker.pin_lora method MLXWorker.pin_lora(lora_id: int) -> bool Report that pinning a LoRA adapter is unsupported. #L233-L236
MLXWorker.list_loras method MLXWorker.list_loras() -> set[int] Return the empty set because runtime LoRA adapters are unsupported. #L238-L241
MLXWorker.sleep method MLXWorker.sleep(level: int = 1) -> None Leave the worker active because MLX unified memory has no sleep mode. #L244-L247
MLXWorker.wake_up method MLXWorker.wake_up(tags: list[str] \| None = None) -> None Perform no work because the MLX worker never enters sleep mode. #L249-L252
MLXWorker.vocab_size method MLXWorker.vocab_size() -> int Get vocabulary size. #L255-L257
MLXWorker.get_cache_block_size_bytes method MLXWorker.get_cache_block_size_bytes() -> int Get size of a cache block in bytes. #L259-L271
MLXWorker.profile method MLXWorker.profile(is_start: bool = True) -> None Profiling (not yet implemented for MLX). #L273-L275
MLXWorker.__repr__ method MLXWorker.__repr__() -> str Method MLXWorker.__repr__ returns f'<MLXWorker rank={self.rank} local_rank={self.local_rank}>'. #L277-L278