# `vllm_mlx.model_runner`
MLX Model Runner for vLLM.
[View the complete module source at #L1-L476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L1-L476).
## API details
Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.
::: vllm_mlx.model_runner
options:
members:
- logger
- SamplerOutput
- MLXModelRunnerOutput
- MLXModelRunner
filters: []
show_if_no_docstring: true
## Complete contract reference
Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.
vllm_mlx.model_runner.SamplerOutput · class
```python
vllm_mlx.model_runner.SamplerOutput(token_ids: list[int], logprobs: list[dict] | None = None)
```
Output from sampling.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `token_ids` | `list[int]` | `yes` | `none` | Required constructor field. |
| `logprobs` | `list[dict] \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
**Returns**
- Constructs: `vllm_mlx.model_runner.SamplerOutput`
**Exceptions and behavior**
Class `SamplerOutput` declares 0 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L29-L33](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L29-L33).
vllm_mlx.model_runner.MLXModelRunnerOutput · class
```python
vllm_mlx.model_runner.MLXModelRunnerOutput(req_id_to_token_ids: dict[str, list[int]], req_id_to_logprobs: dict[str, list[dict]] | None = None, num_tokens_generated: int = 0, generation_time_s: float = 0.0)
```
Output from MLX model runner, compatible with vLLM's ModelRunnerOutput.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `req_id_to_token_ids` | `dict[str, list[int]]` | `yes` | `none` | Required constructor field. |
| `req_id_to_logprobs` | `dict[str, list[dict]] \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `num_tokens_generated` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `generation_time_s` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. |
**Returns**
- Constructs: `vllm_mlx.model_runner.MLXModelRunnerOutput`
**Exceptions and behavior**
Class `MLXModelRunnerOutput` declares 0 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L37-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L37-L50).
vllm_mlx.model_runner.MLXModelRunner · class
```python
vllm_mlx.model_runner.MLXModelRunner(vllm_config: 'VllmConfig', enable_optimizations: bool = True)
```
Model runner that uses mlx-lm for inference.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `vllm_config` | `'VllmConfig'` | `yes` | `none` | vLLM configuration |
| `enable_optimizations` | `bool` | `no` | `True` | Whether to enable low-level optimizations |
**Returns**
- Constructs: `vllm_mlx.model_runner.MLXModelRunner`
**Exceptions and behavior**
Class `MLXModelRunner` declares 16 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L53-L476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L53-L476).
vllm_mlx.model_runner.MLXModelRunner.__init__ · method
```python
vllm_mlx.model_runner.MLXModelRunner.__init__(vllm_config: 'VllmConfig', enable_optimizations: bool = True) -> not annotated
```
Initialize MLX model runner.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `vllm_config` | `'VllmConfig'` | `yes` | `none` | vLLM configuration |
| `enable_optimizations` | `bool` | `no` | `True` | Whether to enable low-level optimizations |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Method `MLXModelRunner.__init__` updates `self.vllm_config`, `self.model_config`, `self.cache_config`, `self.scheduler_config`; calls `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L69-L104](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L69-L104).
vllm_mlx.model_runner.MLXModelRunner.load_model · method
```python
vllm_mlx.model_runner.MLXModelRunner.load_model() -> None
```
Load model using mlx-lm with optimizations.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Method `MLXModelRunner.load_model` updates `self.model`, `self.tokenizer`, `self._loaded`; calls `logger.info`, `time.time`, `load`, `self._create_default_sampler`; can raise `ImportError`; returns `None`.
Directly raised exceptions: `ImportError`.
[View source #L106-L145](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L106-L145).
vllm_mlx.model_runner.MLXModelRunner._apply_optimizations · method
```python
vllm_mlx.model_runner.MLXModelRunner._apply_optimizations() -> None
```
Apply low-level optimizations for maximum performance.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `MLXModelRunner._apply_optimizations` updates `self._hardware_info`; calls `detect_hardware`, `logger.info`, `configure_memory_optimization`, `self._setup_compiled_forward`.
No direct `raise` statement appears in this definition.
[View source #L147-L168](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L147-L168).
vllm_mlx.model_runner.MLXModelRunner._setup_compiled_forward · method
```python
vllm_mlx.model_runner.MLXModelRunner._setup_compiled_forward() -> None
```
Setup compiled forward pass using mx.compile() for kernel fusion.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Method `MLXModelRunner._setup_compiled_forward` updates `self._compiled_forward`; calls `hasattr`, `mx.compile`, `logger.info`, `logger.warning`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L170-L193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L170-L193).
vllm_mlx.model_runner.MLXModelRunner._create_default_sampler · method
```python
vllm_mlx.model_runner.MLXModelRunner._create_default_sampler() -> None
```
Create default sampler for generation.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `MLXModelRunner._create_default_sampler` updates `self._sampler`; calls `make_sampler`, `logger.warning`.
No direct `raise` statement appears in this definition.
[View source #L195-L205](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L195-L205).
vllm_mlx.model_runner.MLXModelRunner.initialize_cache · method
```python
vllm_mlx.model_runner.MLXModelRunner.initialize_cache(num_blocks: int) -> None
```
Initialize KV cache.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `num_blocks` | `int` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `MLXModelRunner.initialize_cache` updates `self._num_cache_blocks`; calls `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L207-L210](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L207-L210).
vllm_mlx.model_runner.MLXModelRunner.get_kv_cache_spec · method
```python
vllm_mlx.model_runner.MLXModelRunner.get_kv_cache_spec() -> dict
```
Get KV cache specification.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `dict`
- Direct return expressions: `{'num_blocks': self._num_cache_blocks, 'block_size': self.cache_config.block_size}`
**Exceptions and behavior**
Method `MLXModelRunner.get_kv_cache_spec` returns `{'num_blocks': self._num_cache_blocks, 'block_size': self.cache_config.block_size}`.
No direct `raise` statement appears in this definition.
[View source #L215-L220](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L215-L220).
vllm_mlx.model_runner.MLXModelRunner.get_cache_block_size_bytes · method
```python
vllm_mlx.model_runner.MLXModelRunner.get_cache_block_size_bytes() -> int
```
Calculate cache block size in bytes.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `int`
- Direct return expressions: `0`; `2 * block_size * num_layers * num_kv_heads * head_size * 2`
**Exceptions and behavior**
Method `MLXModelRunner.get_cache_block_size_bytes` calls `getattr`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L222-L240](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L222-L240).
vllm_mlx.model_runner.MLXModelRunner.warm_up · method
```python
vllm_mlx.model_runner.MLXModelRunner.warm_up() -> None
```
Warm up model with a test generation.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `MLXModelRunner.warm_up` calls `self.load_model`, `logger.info`, `generate`, `logger.warning`.
No direct `raise` statement appears in this definition.
[View source #L242-L263](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L242-L263).
vllm_mlx.model_runner.MLXModelRunner.execute_model · method
```python
vllm_mlx.model_runner.MLXModelRunner.execute_model(scheduler_output: 'SchedulerOutput') -> MLXModelRunnerOutput
```
Execute model inference for scheduled requests.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `scheduler_output` | `'SchedulerOutput'` | `yes` | `none` | Contains requests to process |
**Returns**
- Type: `MLXModelRunnerOutput`
- Direct return expressions: `MLXModelRunnerOutput(req_id_to_token_ids=req_id_to_token_ids, num_tokens_generated=total_tokens, generation_time_s=gene…`
**Exceptions and behavior**
Method `MLXModelRunner.execute_model` calls `RuntimeError`, `time.time`, `self._generate_for_request`, `len`; can raise `RuntimeError`; returns `MLXModelRunnerOutput(req_id_to_token_ids=req_id_to_token_ids, num_tokens_generated=total_tokens, generation_time_s=gene…`.
Directly raised exceptions: `RuntimeError`.
[View source #L265-L315](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L265-L315).
vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking · method
```python
vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking(input_ids: mx.array, cache: Optional[Any] = None) -> tuple[mx.array, Any]
```
Process prompt with optimal chunking for L2 cache efficiency.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `input_ids` | `mx.array` | `yes` | `none` | Input token IDs [1, seq_len] |
| `cache` | `Optional[Any]` | `no` | `None` | Optional existing KV cache |
**Returns**
- Type: `tuple[mx.array, Any]`
- Direct return expressions: `forward_fn(input_ids, cache=cache)`; `(logits, cache)`
**Exceptions and behavior**
Method `MLXModelRunner._prefill_with_chunking` calls `len`, `get_optimal_prefill_size`, `input_ids.reshape`, `forward_fn`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L317-L362](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L317-L362).
vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size · nested function
```python
vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size(seq_len) -> not annotated
```
Nested Function `MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size` calls `min`; returns `min(512, seq_len)`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `seq_len` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `min(512, seq_len)`
**Exceptions and behavior**
Nested Function `MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size` calls `min`; returns `min(512, seq_len)`.
No direct `raise` statement appears in this definition.
[View source #L339-L340](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L339-L340).
vllm_mlx.model_runner.MLXModelRunner._generate_for_request · method
```python
vllm_mlx.model_runner.MLXModelRunner._generate_for_request(prompt_token_ids: list[int], sampling_params: Any, max_tokens: int = 1) -> list[int]
```
Generate tokens for a single request.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt_token_ids` | `list[int]` | `yes` | `none` | Input token IDs |
| `sampling_params` | `Any` | `yes` | `none` | Sampling parameters |
| `max_tokens` | `int` | `no` | `1` | Maximum tokens to generate |
**Returns**
- Type: `list[int]`
- Direct return expressions: `generated_ids`; `[]`
**Exceptions and behavior**
Method `MLXModelRunner._generate_for_request` calls `getattr`, `make_sampler`, `mx.array`, `generate_step`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L364-L418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L364-L418).
vllm_mlx.model_runner.MLXModelRunner._continue_generation · method
```python
vllm_mlx.model_runner.MLXModelRunner._continue_generation(req_id: str) -> list[int]
```
Continue generation for an existing request.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `req_id` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `list[int]`
- Direct return expressions: `[]`
**Exceptions and behavior**
Method `MLXModelRunner._continue_generation` returns `[]`.
No direct `raise` statement appears in this definition.
[View source #L420-L428](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L420-L428).
vllm_mlx.model_runner.MLXModelRunner.decode_tokens · method
```python
vllm_mlx.model_runner.MLXModelRunner.decode_tokens(token_ids: list[int]) -> str
```
Decode token IDs to text.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `token_ids` | `list[int]` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `str`
- Direct return expressions: `''`; `self.tokenizer.decode(token_ids)`
**Exceptions and behavior**
Method `MLXModelRunner.decode_tokens` calls `self.tokenizer.decode`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L430-L434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L430-L434).
vllm_mlx.model_runner.MLXModelRunner.get_model_info · method
```python
vllm_mlx.model_runner.MLXModelRunner.get_model_info() -> dict
```
Get information about the loaded model and optimizations.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `dict`
- Direct return expressions: `info`
**Exceptions and behavior**
Method `MLXModelRunner.get_model_info` calls `getattr`, `info.update`; returns `info`.
No direct `raise` statement appears in this definition.
[View source #L436-L471](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L436-L471).
vllm_mlx.model_runner.MLXModelRunner.__repr__ · method
```python
vllm_mlx.model_runner.MLXModelRunner.__repr__() -> str
```
Method `MLXModelRunner.__repr__` returns `f''`.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `str`
- Direct return expressions: `f''`
**Exceptions and behavior**
Method `MLXModelRunner.__repr__` returns `f''`.
No direct `raise` statement appears in this definition.
[View source #L473-L476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L473-L476).
## Complete symbol map
This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.
| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`SamplerOutput`](#contract-vllm_mlx.model_runner.SamplerOutput) | class | `SamplerOutput(token_ids: list[int], logprobs: list[dict] \| None = None)` | Output from sampling. | [#L29-L33](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L29-L33) |
| [`MLXModelRunnerOutput`](#contract-vllm_mlx.model_runner.MLXModelRunnerOutput) | class | `MLXModelRunnerOutput(req_id_to_token_ids: dict[str, list[int]], req_id_to_logprobs: dict[str, list[dict]] \| None = None, num_tokens_generated: int = 0, generation_time_s: float = 0.0)` | Output from MLX model runner, compatible with vLLM's ModelRunnerOutput. | [#L37-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L37-L50) |
| [`MLXModelRunner`](#contract-vllm_mlx.model_runner.MLXModelRunner) | class | `MLXModelRunner(vllm_config: 'VllmConfig', enable_optimizations: bool = True)` | Model runner that uses mlx-lm for inference. | [#L53-L476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L53-L476) |
| [`MLXModelRunner.__init__`](#contract-vllm_mlx.model_runner.MLXModelRunner.__init__) | method | `MLXModelRunner.__init__(vllm_config: 'VllmConfig', enable_optimizations: bool = True) -> not annotated` | Initialize MLX model runner. | [#L69-L104](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L69-L104) |
| [`MLXModelRunner.load_model`](#contract-vllm_mlx.model_runner.MLXModelRunner.load_model) | method | `MLXModelRunner.load_model() -> None` | Load model using mlx-lm with optimizations. | [#L106-L145](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L106-L145) |
| [`MLXModelRunner._apply_optimizations`](#contract-vllm_mlx.model_runner.MLXModelRunner._apply_optimizations) | method | `MLXModelRunner._apply_optimizations() -> None` | Apply low-level optimizations for maximum performance. | [#L147-L168](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L147-L168) |
| [`MLXModelRunner._setup_compiled_forward`](#contract-vllm_mlx.model_runner.MLXModelRunner._setup_compiled_forward) | method | `MLXModelRunner._setup_compiled_forward() -> None` | Setup compiled forward pass using mx.compile() for kernel fusion. | [#L170-L193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L170-L193) |
| [`MLXModelRunner._create_default_sampler`](#contract-vllm_mlx.model_runner.MLXModelRunner._create_default_sampler) | method | `MLXModelRunner._create_default_sampler() -> None` | Create default sampler for generation. | [#L195-L205](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L195-L205) |
| [`MLXModelRunner.initialize_cache`](#contract-vllm_mlx.model_runner.MLXModelRunner.initialize_cache) | method | `MLXModelRunner.initialize_cache(num_blocks: int) -> None` | Initialize KV cache. | [#L207-L210](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L207-L210) |
| [`MLXModelRunner.get_kv_cache_spec`](#contract-vllm_mlx.model_runner.MLXModelRunner.get_kv_cache_spec) | method | `MLXModelRunner.get_kv_cache_spec() -> dict` | Get KV cache specification. | [#L215-L220](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L215-L220) |
| [`MLXModelRunner.get_cache_block_size_bytes`](#contract-vllm_mlx.model_runner.MLXModelRunner.get_cache_block_size_bytes) | method | `MLXModelRunner.get_cache_block_size_bytes() -> int` | Calculate cache block size in bytes. | [#L222-L240](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L222-L240) |
| [`MLXModelRunner.warm_up`](#contract-vllm_mlx.model_runner.MLXModelRunner.warm_up) | method | `MLXModelRunner.warm_up() -> None` | Warm up model with a test generation. | [#L242-L263](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L242-L263) |
| [`MLXModelRunner.execute_model`](#contract-vllm_mlx.model_runner.MLXModelRunner.execute_model) | method | `MLXModelRunner.execute_model(scheduler_output: 'SchedulerOutput') -> MLXModelRunnerOutput` | Execute model inference for scheduled requests. | [#L265-L315](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L265-L315) |
| [`MLXModelRunner._prefill_with_chunking`](#contract-vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking) | method | `MLXModelRunner._prefill_with_chunking(input_ids: mx.array, cache: Optional[Any] = None) -> tuple[mx.array, Any]` | Process prompt with optimal chunking for L2 cache efficiency. | [#L317-L362](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L317-L362) |
| [`MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size`](#contract-vllm_mlx.model_runner.MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size) | nested function | `MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size(seq_len) -> not annotated` | Nested Function `MLXModelRunner._prefill_with_chunking.get_optimal_prefill_size` calls `min`; returns `min(512, seq_len)`. | [#L339-L340](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L339-L340) |
| [`MLXModelRunner._generate_for_request`](#contract-vllm_mlx.model_runner.MLXModelRunner._generate_for_request) | method | `MLXModelRunner._generate_for_request(prompt_token_ids: list[int], sampling_params: Any, max_tokens: int = 1) -> list[int]` | Generate tokens for a single request. | [#L364-L418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L364-L418) |
| [`MLXModelRunner._continue_generation`](#contract-vllm_mlx.model_runner.MLXModelRunner._continue_generation) | method | `MLXModelRunner._continue_generation(req_id: str) -> list[int]` | Continue generation for an existing request. | [#L420-L428](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L420-L428) |
| [`MLXModelRunner.decode_tokens`](#contract-vllm_mlx.model_runner.MLXModelRunner.decode_tokens) | method | `MLXModelRunner.decode_tokens(token_ids: list[int]) -> str` | Decode token IDs to text. | [#L430-L434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L430-L434) |
| [`MLXModelRunner.get_model_info`](#contract-vllm_mlx.model_runner.MLXModelRunner.get_model_info) | method | `MLXModelRunner.get_model_info() -> dict` | Get information about the loaded model and optimizations. | [#L436-L471](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L436-L471) |
| [`MLXModelRunner.__repr__`](#contract-vllm_mlx.model_runner.MLXModelRunner.__repr__) | method | `MLXModelRunner.__repr__() -> str` | Method `MLXModelRunner.__repr__` returns `f''`. | [#L473-L476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/model_runner.py#L473-L476) |