# `vllm_mlx.engine_core`

Engine Core for vllm-mlx continuous batching.

[View the complete module source at #L1-L794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L1-L794).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.engine_core
    options:
      members:
        - logger
        - _is_stream_thread_error
        - EngineConfig
        - EngineCore
        - AsyncEngineCore
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.engine_core._is_stream_thread_error" markdown="1">
<summary><code>vllm_mlx.engine_core._is_stream_thread_error</code> · function</summary>

```python
vllm_mlx.engine_core._is_stream_thread_error(error: Exception) -> bool
```

True when MLX reports stream ownership mismatch across threads.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `error` | `Exception` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `'no Stream(' in message or 'no Stream(gpu' in message`

**Exceptions and behavior**

Function `_is_stream_thread_error` calls `str`; returns `'no Stream(' in message or 'no Stream(gpu' in message`.
No direct `raise` statement appears in this definition.

[View source #L33-L36](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L33-L36).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineConfig" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineConfig</code> · class</summary>

```python
vllm_mlx.engine_core.EngineConfig(model_name: str = '', scheduler_config: Optional[SchedulerConfig] = None, step_interval: float = 0.001, stream_interval: int = 1, gpu_memory_utilization: float = 0.9)
```

Configuration for the engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. |
| `scheduler_config` | `Optional[SchedulerConfig]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `step_interval` | `float` | `no` | `0.001` | Optional constructor field; defaults to `0.001`. |
| `stream_interval` | `int` | `no` | `1` | Optional constructor field; defaults to `1`. |
| `gpu_memory_utilization` | `float` | `no` | `0.9` | Optional constructor field; defaults to `0.9`. |

**Returns**

- Constructs: `vllm_mlx.engine_core.EngineConfig`

**Exceptions and behavior**

Class `EngineConfig` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L40-L47](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L40-L47).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore</code> · class</summary>

```python
vllm_mlx.engine_core.EngineCore(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None, engine_id: Optional[str] = None, force_model_ownership: bool = True)
```

Core engine for vllm-mlx inference with continuous batching.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The MLX model |
| `tokenizer` | `Any` | `yes` | `none` | The tokenizer |
| `config` | `Optional[EngineConfig]` | `no` | `None` | Engine configuration |
| `engine_id` | `Optional[str]` | `no` | `None` | Optional unique ID for this engine (auto-generated if None) |
| `force_model_ownership` | `bool` | `no` | `True` | If True (default), forcibly take model ownership from any existing engine. If False, raises ModelOwnershipError if model is in use. |

**Returns**

- Constructs: `vllm_mlx.engine_core.EngineCore`

**Exceptions and behavior**

Class `EngineCore` declares 21 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L50-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L50-L698).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.__init__" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.__init__</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.__init__(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None, engine_id: Optional[str] = None, force_model_ownership: bool = True) -> not annotated
```

Initialize the engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The MLX model |
| `tokenizer` | `Any` | `yes` | `none` | The tokenizer |
| `config` | `Optional[EngineConfig]` | `no` | `None` | Engine configuration |
| `engine_id` | `Optional[str]` | `no` | `None` | Optional unique ID for this engine (auto-generated if None) |
| `force_model_ownership` | `bool` | `no` | `True` | If True (default), forcibly take model ownership from any existing engine. If False, raises ModelOwnershipError if model is in use. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `EngineCore.__init__` updates `self.model`, `self.tokenizer`, `self.config`, `self._engine_id`; calls `EngineConfig`, `str`, `uuid.uuid4`, `get_registry`.
No direct `raise` statement appears in this definition.

[View source #L58-L114](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L58-L114).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.start" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.start</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.start() -> None
```

Start the engine loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `EngineCore.start` updates `self._running`, `self._start_time`, `self._task`; calls `time.time`, `asyncio.create_task`, `self._engine_loop`, `logger.info`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L116-L124](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L116-L124).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.stop" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.stop</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.stop() -> None
```

Stop the engine loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `EngineCore.stop` updates `self._running`, `self._task`; calls `self._task.cancel`, `self.scheduler._close_batch_generator`, `logger.info`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L126-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L126-L140).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.is_running" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.is_running</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.is_running() -> bool
```

Check if engine is running.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `self._running`

**Exceptions and behavior**

Method `EngineCore.is_running` returns `self._running`.
No direct `raise` statement appears in this definition.

[View source #L142-L144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L142-L144).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore._engine_loop() -> None
```

Main engine loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `EngineCore._engine_loop` calls `asyncio.get_running_loop`, `ThreadPoolExecutor`, `mx.device_info().get`, `mx.device_info`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L146-L334](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L146-L334).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._bind_worker_streams_once" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._bind_worker_streams_once</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._bind_worker_streams_once() -> None
```

Nested Function `EngineCore._engine_loop._bind_worker_streams_once` calls `bind_generation_streams`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._bind_worker_streams_once` calls `bind_generation_streams`.
No direct `raise` statement appears in this definition.

[View source #L160-L164](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L160-L164).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._bind_model_streams_once" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._bind_model_streams_once</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._bind_model_streams_once() -> None
```

Nested Function `EngineCore._engine_loop._bind_model_streams_once` calls `bind_generation_streams`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._bind_model_streams_once` calls `bind_generation_streams`.
No direct `raise` statement appears in this definition.

[View source #L166-L170](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L166-L170).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._step_on_worker" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._step_on_worker</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._step_on_worker() -> not annotated
```

Nested Function `EngineCore._engine_loop._step_on_worker` updates `self._steps_executed`; calls `_bind_worker_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `output`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._step_on_worker` updates `self._steps_executed`; calls `_bind_worker_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`.
No direct `raise` statement appears in this definition.

[View source #L172-L190](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L172-L190).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._step_on_model_thread" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._step_on_model_thread</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._step_on_model_thread() -> not annotated
```

Nested Function `EngineCore._engine_loop._step_on_model_thread` updates `self._steps_executed`; calls `_bind_model_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `output`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._step_on_model_thread` updates `self._steps_executed`; calls `_bind_model_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`.
No direct `raise` statement appears in this definition.

[View source #L192-L210](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L192-L210).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._recover_stream_thread_error_on_worker" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._recover_stream_thread_error_on_worker</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._recover_stream_thread_error_on_worker() -> None
```

Nested Function `EngineCore._engine_loop._recover_stream_thread_error_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._recover_from_cache_error`, `self.scheduler._reschedule_running_requests`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._recover_stream_thread_error_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._recover_from_cache_error`, `self.scheduler._reschedule_running_requests`.
No direct `raise` statement appears in this definition.

[View source #L212-L215](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L212-L215).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._clear_cache_on_worker" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._clear_cache_on_worker</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._clear_cache_on_worker() -> None
```

Nested Function `EngineCore._engine_loop._clear_cache_on_worker` calls `_bind_worker_streams_once`, `mx.clear_cache`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._clear_cache_on_worker` calls `_bind_worker_streams_once`, `mx.clear_cache`.
No direct `raise` statement appears in this definition.

[View source #L217-L219](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L217-L219).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._engine_loop._close_batch_generator_on_worker" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._engine_loop._close_batch_generator_on_worker</code> · nested function</summary>

```python
vllm_mlx.engine_core.EngineCore._engine_loop._close_batch_generator_on_worker() -> None
```

Nested Function `EngineCore._engine_loop._close_batch_generator_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._close_batch_generator`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `EngineCore._engine_loop._close_batch_generator_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._close_batch_generator`.
No direct `raise` statement appears in this definition.

[View source #L221-L223](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L221-L223).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.add_request" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.add_request</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.add_request(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, images: Optional[List[Any]] = None, videos: Optional[List[Any]] = None, prefix_boundary: int = 0) -> str
```

Add a request for processing.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `Union[str, List[int]]` | `yes` | `none` | Input prompt (string or token IDs) |
| `sampling_params` | `Optional[SamplingParams]` | `no` | `None` | Generation parameters |
| `request_id` | `Optional[str]` | `no` | `None` | Optional custom request ID |
| `images` | `Optional[List[Any]]` | `no` | `None` | Optional images for multimodal |
| `videos` | `Optional[List[Any]]` | `no` | `None` | Optional videos for multimodal |
| `prefix_boundary` | `int` | `no` | `0` | Token count for shared prefix (for cache) |

**Returns**

- Type: `str`
- Direct return expressions: `request_id`

**Exceptions and behavior**

Method `EngineCore.add_request` calls `str`, `uuid.uuid4`, `SamplingParams`, `Request`; returns `request_id`.
No direct `raise` statement appears in this definition.

[View source #L336-L384](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L336-L384).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.abort_request" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.abort_request</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.abort_request(request_id: str) -> bool
```

Abort a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `result`

**Exceptions and behavior**

Method `EngineCore.abort_request` calls `self.scheduler.abort_request`, `self._cleanup_request`; returns `result`.
No direct `raise` statement appears in this definition.

[View source #L386-L390](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L386-L390).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._cleanup_request" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._cleanup_request</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore._cleanup_request(request_id: str) -> None
```

Clean up request tracking.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `EngineCore._cleanup_request` calls `self._output_collectors.pop`, `collector.clear`, `self._stream_states.pop`, `self._finished_events.pop`.
No direct `raise` statement appears in this definition.

[View source #L392-L399](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L392-L399).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.stream_outputs" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.stream_outputs</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.stream_outputs(request_id: str, timeout: Optional[float] = None) -> AsyncIterator[RequestOutput]
```

Stream outputs for a request with low-latency non-blocking pattern.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | The request ID |
| `timeout` | `Optional[float]` | `no` | `None` | Optional timeout in seconds |

**Returns**

- Type: `AsyncIterator[RequestOutput]`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Method `EngineCore.stream_outputs` calls `_time.monotonic`, `self._output_collectors.get`, `logger.warning`, `logger.info`; awaits asynchronous work; yields values incrementally; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L401-L488](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L401-L488).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.generate" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.generate</code> · method</summary>

```python
async vllm_mlx.engine_core.EngineCore.generate(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, **kwargs) -> RequestOutput
```

Generate a complete response (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `Union[str, List[int]]` | `yes` | `none` | Input prompt |
| `sampling_params` | `Optional[SamplingParams]` | `no` | `None` | Generation parameters |
| `request_id` | `Optional[str]` | `no` | `None` | Optional request ID |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `RequestOutput`
- Direct return expressions: `final_output`

**Exceptions and behavior**

Method `EngineCore.generate` calls `self.add_request`, `self._finished_events.get`, `RuntimeError`, `event.wait`; awaits asynchronous work; can raise `RuntimeError`; returns `final_output`.
Directly raised exceptions: `RuntimeError`.

[View source #L490-L552](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L490-L552).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.generate_batch_sync" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.generate_batch_sync</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.generate_batch_sync(prompts: List[Union[str, List[int]]], sampling_params: Optional[SamplingParams] = None) -> List[RequestOutput]
```

Generate responses synchronously for maximum throughput.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompts` | `List[Union[str, List[int]]]` | `yes` | `none` | List of input prompts |
| `sampling_params` | `Optional[SamplingParams]` | `no` | `None` | Generation parameters (same for all) |

**Returns**

- Type: `List[RequestOutput]`
- Direct return expressions: `[results[rid] for rid in request_ids]`

**Exceptions and behavior**

Method `EngineCore.generate_batch_sync` calls `SamplingParams`, `str`, `uuid_module.uuid4`, `Request`; returns `[results[rid] for rid in request_ids]`.
No direct `raise` statement appears in this definition.

[View source #L554-L609](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L554-L609).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.get_stats" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.get_stats</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.get_stats() -> Dict[str, Any]
```

Get engine statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, Any]`
- Direct return expressions: `{'running': self._running, 'uptime_seconds': uptime, 'steps_executed': self._steps_executed, 'active_requests': len(sel…`

**Exceptions and behavior**

Method `EngineCore.get_stats` calls `self.scheduler.get_stats`, `time.time`, `len`, `self.scheduler.get_running_requests_info`; returns `{'running': self._running, 'uptime_seconds': uptime, 'steps_executed': self._steps_executed, 'active_requests': len(sel…`.
No direct `raise` statement appears in this definition.

[View source #L611-L624](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L611-L624).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.get_cache_stats" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.get_cache_stats</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.get_cache_stats() -> Optional[Dict[str, Any]]
```

Get prefix cache statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Optional[Dict[str, Any]]`
- Direct return expressions: `self.scheduler.get_cache_stats()`

**Exceptions and behavior**

Method `EngineCore.get_cache_stats` calls `self.scheduler.get_cache_stats`; returns `self.scheduler.get_cache_stats()`.
No direct `raise` statement appears in this definition.

[View source #L626-L628](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L626-L628).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.save_cache_to_disk" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.save_cache_to_disk</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.save_cache_to_disk(cache_dir: str) -> bool
```

Save prefix cache to disk.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `self.scheduler.save_cache_to_disk(cache_dir)`

**Exceptions and behavior**

Method `EngineCore.save_cache_to_disk` calls `self.scheduler.save_cache_to_disk`; returns `self.scheduler.save_cache_to_disk(cache_dir)`.
No direct `raise` statement appears in this definition.

[View source #L630-L632](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L630-L632).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.load_cache_from_disk" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.load_cache_from_disk</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.load_cache_from_disk(cache_dir: str) -> int
```

Load prefix cache from disk.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `int`
- Direct return expressions: `self.scheduler.load_cache_from_disk(cache_dir)`

**Exceptions and behavior**

Method `EngineCore.load_cache_from_disk` calls `self.scheduler.load_cache_from_disk`; returns `self.scheduler.load_cache_from_disk(cache_dir)`.
No direct `raise` statement appears in this definition.

[View source #L634-L636](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L634-L636).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.clear_runtime_caches" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.clear_runtime_caches</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.clear_runtime_caches() -> Dict[str, Any] | None
```

Clear scheduler-managed runtime caches.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, Any] | None`
- Direct return expressions: `self.scheduler.clear_runtime_caches()`

**Exceptions and behavior**

Method `EngineCore.clear_runtime_caches` calls `self.scheduler.clear_runtime_caches`; returns `self.scheduler.clear_runtime_caches()`.
No direct `raise` statement appears in this definition.

[View source #L638-L640](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L638-L640).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.clear_prefix_cache" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.clear_prefix_cache</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.clear_prefix_cache() -> None
```

Clear the prefix cache (delegates to scheduler).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `EngineCore.clear_prefix_cache` calls `hasattr`, `self.scheduler.clear_prefix_cache`.
No direct `raise` statement appears in this definition.

[View source #L642-L645](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L642-L645).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore._release_model" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore._release_model</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore._release_model() -> None
```

Release model ownership.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `EngineCore._release_model` updates `self._owns_model`; calls `get_registry`, `registry.release`, `logger.debug`.
No direct `raise` statement appears in this definition.

[View source #L647-L653](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L647-L653).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.close" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.close</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.close() -> None
```

Explicitly close the engine and release resources.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `EngineCore.close` updates `self._owns_model`, `self._closed`; calls `get_registry`, `registry.release`, `logger.debug`, `self.scheduler.deep_reset`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L655-L685](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L655-L685).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.__del__" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.__del__</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.__del__() -> not annotated
```

Cleanup on destruction.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `EngineCore.__del__` calls `self._release_model`.
No direct `raise` statement appears in this definition.

[View source #L687-L693](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L687-L693).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.EngineCore.engine_id" markdown="1">
<summary><code>vllm_mlx.engine_core.EngineCore.engine_id</code> · method</summary>

```python
vllm_mlx.engine_core.EngineCore.engine_id() -> str
```

Get the engine ID.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`
- Direct return expressions: `self._engine_id`

**Exceptions and behavior**

Method `EngineCore.engine_id` returns `self._engine_id`.
No direct `raise` statement appears in this definition.

[View source #L696-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L696-L698).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore</code> · class</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None)
```

Async context manager wrapper for EngineCore.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `tokenizer` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `config` | `Optional[EngineConfig]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Constructs: `vllm_mlx.engine_core.AsyncEngineCore`

**Exceptions and behavior**

Class `AsyncEngineCore` declares 14 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L701-L794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L701-L794).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.__init__" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.__init__</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.__init__(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None) -> not annotated
```

Method `AsyncEngineCore.__init__` updates `self.engine`; calls `EngineCore`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `tokenizer` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `config` | `Optional[EngineConfig]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `AsyncEngineCore.__init__` updates `self.engine`; calls `EngineCore`.
No direct `raise` statement appears in this definition.

[View source #L712-L718](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L712-L718).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.__aenter__" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.__aenter__</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.__aenter__() -> 'AsyncEngineCore'
```

Method `AsyncEngineCore.__aenter__` calls `self.engine.start`; awaits asynchronous work; returns `self`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `'AsyncEngineCore'`
- Direct return expressions: `self`

**Exceptions and behavior**

Method `AsyncEngineCore.__aenter__` calls `self.engine.start`; awaits asynchronous work; returns `self`.
No direct `raise` statement appears in this definition.

[View source #L720-L722](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L720-L722).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.__aexit__" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.__aexit__</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.__aexit__(*args) -> None
```

Method `AsyncEngineCore.__aexit__` calls `self.engine.stop`; awaits asynchronous work.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `*args` | `not annotated` | `no` | `none` | Additional variadic positional inputs accepted by this callable. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `AsyncEngineCore.__aexit__` calls `self.engine.stop`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L724-L725](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L724-L725).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.start" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.start</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.start() -> None
```

Start engine (creates task in current loop).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `AsyncEngineCore.start` updates `self._start_task`; calls `asyncio.create_task`, `self.engine.start`.
No direct `raise` statement appears in this definition.

[View source #L727-L729](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L727-L729).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.stop" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.stop</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.stop() -> None
```

Stop the engine.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `AsyncEngineCore.stop` calls `self.engine.stop`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L731-L733](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L731-L733).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.add_request" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.add_request</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.add_request(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, **kwargs) -> str
```

Add a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `Union[str, List[int]]` | `yes` | `none` | Required positional or keyword input. |
| `sampling_params` | `Optional[SamplingParams]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `request_id` | `Optional[str]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `str`
- Direct return expressions: `await self.engine.add_request(prompt=prompt, sampling_params=sampling_params, request_id=request_id, **kwargs)`

**Exceptions and behavior**

Method `AsyncEngineCore.add_request` calls `self.engine.add_request`; awaits asynchronous work; returns `await self.engine.add_request(prompt=prompt, sampling_params=sampling_params, request_id=request_id, **kwargs)`.
No direct `raise` statement appears in this definition.

[View source #L735-L748](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L735-L748).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.abort_request" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.abort_request</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.abort_request(request_id: str) -> bool
```

Abort a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `await self.engine.abort_request(request_id)`

**Exceptions and behavior**

Method `AsyncEngineCore.abort_request` calls `self.engine.abort_request`; awaits asynchronous work; returns `await self.engine.abort_request(request_id)`.
No direct `raise` statement appears in this definition.

[View source #L750-L752](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L750-L752).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.stream_outputs" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.stream_outputs</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.stream_outputs(request_id: str, timeout: Optional[float] = None) -> AsyncIterator[RequestOutput]
```

Stream outputs.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
| `timeout` | `Optional[float]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `AsyncIterator[RequestOutput]`
- Yields values incrementally.

**Exceptions and behavior**

Method `AsyncEngineCore.stream_outputs` calls `self.engine.stream_outputs`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L754-L761](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L754-L761).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.generate" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.generate</code> · method</summary>

```python
async vllm_mlx.engine_core.AsyncEngineCore.generate(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, **kwargs) -> RequestOutput
```

Generate complete response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `Union[str, List[int]]` | `yes` | `none` | Required positional or keyword input. |
| `sampling_params` | `Optional[SamplingParams]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `RequestOutput`
- Direct return expressions: `await self.engine.generate(prompt=prompt, sampling_params=sampling_params, **kwargs)`

**Exceptions and behavior**

Method `AsyncEngineCore.generate` calls `self.engine.generate`; awaits asynchronous work; returns `await self.engine.generate(prompt=prompt, sampling_params=sampling_params, **kwargs)`.
No direct `raise` statement appears in this definition.

[View source #L763-L774](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L763-L774).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.get_stats" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.get_stats</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.get_stats() -> Dict[str, Any]
```

Get engine stats.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, Any]`
- Direct return expressions: `self.engine.get_stats()`

**Exceptions and behavior**

Method `AsyncEngineCore.get_stats` calls `self.engine.get_stats`; returns `self.engine.get_stats()`.
No direct `raise` statement appears in this definition.

[View source #L776-L778](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L776-L778).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.get_cache_stats" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.get_cache_stats</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.get_cache_stats() -> Optional[Dict[str, Any]]
```

Get prefix cache statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Optional[Dict[str, Any]]`
- Direct return expressions: `self.engine.get_cache_stats()`

**Exceptions and behavior**

Method `AsyncEngineCore.get_cache_stats` calls `self.engine.get_cache_stats`; returns `self.engine.get_cache_stats()`.
No direct `raise` statement appears in this definition.

[View source #L780-L782](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L780-L782).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.save_cache_to_disk" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.save_cache_to_disk</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.save_cache_to_disk(cache_dir: str) -> bool
```

Save prefix cache to disk.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `self.engine.save_cache_to_disk(cache_dir)`

**Exceptions and behavior**

Method `AsyncEngineCore.save_cache_to_disk` calls `self.engine.save_cache_to_disk`; returns `self.engine.save_cache_to_disk(cache_dir)`.
No direct `raise` statement appears in this definition.

[View source #L784-L786](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L784-L786).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.load_cache_from_disk" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.load_cache_from_disk</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.load_cache_from_disk(cache_dir: str) -> int
```

Load prefix cache from disk.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `int`
- Direct return expressions: `self.engine.load_cache_from_disk(cache_dir)`

**Exceptions and behavior**

Method `AsyncEngineCore.load_cache_from_disk` calls `self.engine.load_cache_from_disk`; returns `self.engine.load_cache_from_disk(cache_dir)`.
No direct `raise` statement appears in this definition.

[View source #L788-L790](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L788-L790).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine_core.AsyncEngineCore.clear_runtime_caches" markdown="1">
<summary><code>vllm_mlx.engine_core.AsyncEngineCore.clear_runtime_caches</code> · method</summary>

```python
vllm_mlx.engine_core.AsyncEngineCore.clear_runtime_caches() -> Dict[str, Any] | None
```

Clear scheduler-managed runtime caches.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, Any] | None`
- Direct return expressions: `self.engine.clear_runtime_caches()`

**Exceptions and behavior**

Method `AsyncEngineCore.clear_runtime_caches` calls `self.engine.clear_runtime_caches`; returns `self.engine.clear_runtime_caches()`.
No direct `raise` statement appears in this definition.

[View source #L792-L794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L792-L794).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`_is_stream_thread_error`](#contract-vllm_mlx.engine_core._is_stream_thread_error) | function | `_is_stream_thread_error(error: Exception) -> bool` | True when MLX reports stream ownership mismatch across threads. | [#L33-L36](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L33-L36) |
| [`EngineConfig`](#contract-vllm_mlx.engine_core.EngineConfig) | class | `EngineConfig(model_name: str = '', scheduler_config: Optional[SchedulerConfig] = None, step_interval: float = 0.001, stream_interval: int = 1, gpu_memory_utilization: float = 0.9)` | Configuration for the engine. | [#L40-L47](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L40-L47) |
| [`EngineCore`](#contract-vllm_mlx.engine_core.EngineCore) | class | `EngineCore(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None, engine_id: Optional[str] = None, force_model_ownership: bool = True)` | Core engine for vllm-mlx inference with continuous batching. | [#L50-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L50-L698) |
| [`EngineCore.__init__`](#contract-vllm_mlx.engine_core.EngineCore.__init__) | method | `EngineCore.__init__(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None, engine_id: Optional[str] = None, force_model_ownership: bool = True) -> not annotated` | Initialize the engine. | [#L58-L114](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L58-L114) |
| [`EngineCore.start`](#contract-vllm_mlx.engine_core.EngineCore.start) | method | `async EngineCore.start() -> None` | Start the engine loop. | [#L116-L124](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L116-L124) |
| [`EngineCore.stop`](#contract-vllm_mlx.engine_core.EngineCore.stop) | method | `async EngineCore.stop() -> None` | Stop the engine loop. | [#L126-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L126-L140) |
| [`EngineCore.is_running`](#contract-vllm_mlx.engine_core.EngineCore.is_running) | method | `EngineCore.is_running() -> bool` | Check if engine is running. | [#L142-L144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L142-L144) |
| [`EngineCore._engine_loop`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop) | method | `async EngineCore._engine_loop() -> None` | Main engine loop. | [#L146-L334](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L146-L334) |
| [`EngineCore._engine_loop._bind_worker_streams_once`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._bind_worker_streams_once) | nested function | `EngineCore._engine_loop._bind_worker_streams_once() -> None` | Nested Function `EngineCore._engine_loop._bind_worker_streams_once` calls `bind_generation_streams`. | [#L160-L164](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L160-L164) |
| [`EngineCore._engine_loop._bind_model_streams_once`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._bind_model_streams_once) | nested function | `EngineCore._engine_loop._bind_model_streams_once() -> None` | Nested Function `EngineCore._engine_loop._bind_model_streams_once` calls `bind_generation_streams`. | [#L166-L170](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L166-L170) |
| [`EngineCore._engine_loop._step_on_worker`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._step_on_worker) | nested function | `EngineCore._engine_loop._step_on_worker() -> not annotated` | Nested Function `EngineCore._engine_loop._step_on_worker` updates `self._steps_executed`; calls `_bind_worker_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`. | [#L172-L190](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L172-L190) |
| [`EngineCore._engine_loop._step_on_model_thread`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._step_on_model_thread) | nested function | `EngineCore._engine_loop._step_on_model_thread() -> not annotated` | Nested Function `EngineCore._engine_loop._step_on_model_thread` updates `self._steps_executed`; calls `_bind_model_streams_once`, `self.scheduler.step`, `mx.get_active_memory`, `mx.clear_cache`; returns `output`. | [#L192-L210](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L192-L210) |
| [`EngineCore._engine_loop._recover_stream_thread_error_on_worker`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._recover_stream_thread_error_on_worker) | nested function | `EngineCore._engine_loop._recover_stream_thread_error_on_worker() -> None` | Nested Function `EngineCore._engine_loop._recover_stream_thread_error_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._recover_from_cache_error`, `self.scheduler._reschedule_running_requests`. | [#L212-L215](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L212-L215) |
| [`EngineCore._engine_loop._clear_cache_on_worker`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._clear_cache_on_worker) | nested function | `EngineCore._engine_loop._clear_cache_on_worker() -> None` | Nested Function `EngineCore._engine_loop._clear_cache_on_worker` calls `_bind_worker_streams_once`, `mx.clear_cache`. | [#L217-L219](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L217-L219) |
| [`EngineCore._engine_loop._close_batch_generator_on_worker`](#contract-vllm_mlx.engine_core.EngineCore._engine_loop._close_batch_generator_on_worker) | nested function | `EngineCore._engine_loop._close_batch_generator_on_worker() -> None` | Nested Function `EngineCore._engine_loop._close_batch_generator_on_worker` calls `_bind_worker_streams_once`, `self.scheduler._close_batch_generator`. | [#L221-L223](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L221-L223) |
| [`EngineCore.add_request`](#contract-vllm_mlx.engine_core.EngineCore.add_request) | method | `async EngineCore.add_request(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, images: Optional[List[Any]] = None, videos: Optional[List[Any]] = None, prefix_boundary: int = 0) -> str` | Add a request for processing. | [#L336-L384](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L336-L384) |
| [`EngineCore.abort_request`](#contract-vllm_mlx.engine_core.EngineCore.abort_request) | method | `async EngineCore.abort_request(request_id: str) -> bool` | Abort a request. | [#L386-L390](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L386-L390) |
| [`EngineCore._cleanup_request`](#contract-vllm_mlx.engine_core.EngineCore._cleanup_request) | method | `EngineCore._cleanup_request(request_id: str) -> None` | Clean up request tracking. | [#L392-L399](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L392-L399) |
| [`EngineCore.stream_outputs`](#contract-vllm_mlx.engine_core.EngineCore.stream_outputs) | method | `async EngineCore.stream_outputs(request_id: str, timeout: Optional[float] = None) -> AsyncIterator[RequestOutput]` | Stream outputs for a request with low-latency non-blocking pattern. | [#L401-L488](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L401-L488) |
| [`EngineCore.generate`](#contract-vllm_mlx.engine_core.EngineCore.generate) | method | `async EngineCore.generate(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, **kwargs) -> RequestOutput` | Generate a complete response (non-streaming). | [#L490-L552](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L490-L552) |
| [`EngineCore.generate_batch_sync`](#contract-vllm_mlx.engine_core.EngineCore.generate_batch_sync) | method | `EngineCore.generate_batch_sync(prompts: List[Union[str, List[int]]], sampling_params: Optional[SamplingParams] = None) -> List[RequestOutput]` | Generate responses synchronously for maximum throughput. | [#L554-L609](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L554-L609) |
| [`EngineCore.get_stats`](#contract-vllm_mlx.engine_core.EngineCore.get_stats) | method | `EngineCore.get_stats() -> Dict[str, Any]` | Get engine statistics. | [#L611-L624](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L611-L624) |
| [`EngineCore.get_cache_stats`](#contract-vllm_mlx.engine_core.EngineCore.get_cache_stats) | method | `EngineCore.get_cache_stats() -> Optional[Dict[str, Any]]` | Get prefix cache statistics. | [#L626-L628](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L626-L628) |
| [`EngineCore.save_cache_to_disk`](#contract-vllm_mlx.engine_core.EngineCore.save_cache_to_disk) | method | `EngineCore.save_cache_to_disk(cache_dir: str) -> bool` | Save prefix cache to disk. | [#L630-L632](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L630-L632) |
| [`EngineCore.load_cache_from_disk`](#contract-vllm_mlx.engine_core.EngineCore.load_cache_from_disk) | method | `EngineCore.load_cache_from_disk(cache_dir: str) -> int` | Load prefix cache from disk. | [#L634-L636](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L634-L636) |
| [`EngineCore.clear_runtime_caches`](#contract-vllm_mlx.engine_core.EngineCore.clear_runtime_caches) | method | `EngineCore.clear_runtime_caches() -> Dict[str, Any] \| None` | Clear scheduler-managed runtime caches. | [#L638-L640](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L638-L640) |
| [`EngineCore.clear_prefix_cache`](#contract-vllm_mlx.engine_core.EngineCore.clear_prefix_cache) | method | `EngineCore.clear_prefix_cache() -> None` | Clear the prefix cache (delegates to scheduler). | [#L642-L645](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L642-L645) |
| [`EngineCore._release_model`](#contract-vllm_mlx.engine_core.EngineCore._release_model) | method | `EngineCore._release_model() -> None` | Release model ownership. | [#L647-L653](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L647-L653) |
| [`EngineCore.close`](#contract-vllm_mlx.engine_core.EngineCore.close) | method | `EngineCore.close() -> None` | Explicitly close the engine and release resources. | [#L655-L685](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L655-L685) |
| [`EngineCore.__del__`](#contract-vllm_mlx.engine_core.EngineCore.__del__) | method | `EngineCore.__del__() -> not annotated` | Cleanup on destruction. | [#L687-L693](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L687-L693) |
| [`EngineCore.engine_id`](#contract-vllm_mlx.engine_core.EngineCore.engine_id) | method | `EngineCore.engine_id() -> str` | Get the engine ID. | [#L696-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L696-L698) |
| [`AsyncEngineCore`](#contract-vllm_mlx.engine_core.AsyncEngineCore) | class | `AsyncEngineCore(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None)` | Async context manager wrapper for EngineCore. | [#L701-L794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L701-L794) |
| [`AsyncEngineCore.__init__`](#contract-vllm_mlx.engine_core.AsyncEngineCore.__init__) | method | `AsyncEngineCore.__init__(model: Any, tokenizer: Any, config: Optional[EngineConfig] = None) -> not annotated` | Method `AsyncEngineCore.__init__` updates `self.engine`; calls `EngineCore`. | [#L712-L718](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L712-L718) |
| [`AsyncEngineCore.__aenter__`](#contract-vllm_mlx.engine_core.AsyncEngineCore.__aenter__) | method | `async AsyncEngineCore.__aenter__() -> 'AsyncEngineCore'` | Method `AsyncEngineCore.__aenter__` calls `self.engine.start`; awaits asynchronous work; returns `self`. | [#L720-L722](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L720-L722) |
| [`AsyncEngineCore.__aexit__`](#contract-vllm_mlx.engine_core.AsyncEngineCore.__aexit__) | method | `async AsyncEngineCore.__aexit__(*args) -> None` | Method `AsyncEngineCore.__aexit__` calls `self.engine.stop`; awaits asynchronous work. | [#L724-L725](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L724-L725) |
| [`AsyncEngineCore.start`](#contract-vllm_mlx.engine_core.AsyncEngineCore.start) | method | `AsyncEngineCore.start() -> None` | Start engine (creates task in current loop). | [#L727-L729](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L727-L729) |
| [`AsyncEngineCore.stop`](#contract-vllm_mlx.engine_core.AsyncEngineCore.stop) | method | `async AsyncEngineCore.stop() -> None` | Stop the engine. | [#L731-L733](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L731-L733) |
| [`AsyncEngineCore.add_request`](#contract-vllm_mlx.engine_core.AsyncEngineCore.add_request) | method | `async AsyncEngineCore.add_request(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, request_id: Optional[str] = None, **kwargs) -> str` | Add a request. | [#L735-L748](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L735-L748) |
| [`AsyncEngineCore.abort_request`](#contract-vllm_mlx.engine_core.AsyncEngineCore.abort_request) | method | `async AsyncEngineCore.abort_request(request_id: str) -> bool` | Abort a request. | [#L750-L752](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L750-L752) |
| [`AsyncEngineCore.stream_outputs`](#contract-vllm_mlx.engine_core.AsyncEngineCore.stream_outputs) | method | `async AsyncEngineCore.stream_outputs(request_id: str, timeout: Optional[float] = None) -> AsyncIterator[RequestOutput]` | Stream outputs. | [#L754-L761](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L754-L761) |
| [`AsyncEngineCore.generate`](#contract-vllm_mlx.engine_core.AsyncEngineCore.generate) | method | `async AsyncEngineCore.generate(prompt: Union[str, List[int]], sampling_params: Optional[SamplingParams] = None, **kwargs) -> RequestOutput` | Generate complete response. | [#L763-L774](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L763-L774) |
| [`AsyncEngineCore.get_stats`](#contract-vllm_mlx.engine_core.AsyncEngineCore.get_stats) | method | `AsyncEngineCore.get_stats() -> Dict[str, Any]` | Get engine stats. | [#L776-L778](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L776-L778) |
| [`AsyncEngineCore.get_cache_stats`](#contract-vllm_mlx.engine_core.AsyncEngineCore.get_cache_stats) | method | `AsyncEngineCore.get_cache_stats() -> Optional[Dict[str, Any]]` | Get prefix cache statistics. | [#L780-L782](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L780-L782) |
| [`AsyncEngineCore.save_cache_to_disk`](#contract-vllm_mlx.engine_core.AsyncEngineCore.save_cache_to_disk) | method | `AsyncEngineCore.save_cache_to_disk(cache_dir: str) -> bool` | Save prefix cache to disk. | [#L784-L786](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L784-L786) |
| [`AsyncEngineCore.load_cache_from_disk`](#contract-vllm_mlx.engine_core.AsyncEngineCore.load_cache_from_disk) | method | `AsyncEngineCore.load_cache_from_disk(cache_dir: str) -> int` | Load prefix cache from disk. | [#L788-L790](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L788-L790) |
| [`AsyncEngineCore.clear_runtime_caches`](#contract-vllm_mlx.engine_core.AsyncEngineCore.clear_runtime_caches) | method | `AsyncEngineCore.clear_runtime_caches() -> Dict[str, Any] \| None` | Clear scheduler-managed runtime caches. | [#L792-L794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine_core.py#L792-L794) |