# `vllm_mlx.engine.base`

Base engine interface for vllm-mlx inference.

[View the complete module source at #L1-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L1-L288).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.engine.base
    options:
      members:
        - logger
        - GenerationOutput
        - EngineBusy
        - suspend_cancellation
        - run_blocking_startup_work
        - cleanup_startup_cancellation
        - BaseEngine
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.engine.base.GenerationOutput" markdown="1">
<summary><code>vllm_mlx.engine.base.GenerationOutput</code> · class</summary>

```python
vllm_mlx.engine.base.GenerationOutput(text: str, tokens: list[int] = field(default_factory=list), prompt_tokens: int = 0, completion_tokens: int = 0, finish_reason: str | None = 'stop', mtp_drafts: int = 0, mtp_accepted: int = 0, new_text: str = '', finished: bool = True, mtp_drafts: int = 0, mtp_accepted: int = 0)
```

Output from generation.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text` | `str` | `yes` | `none` | Required constructor field. |
| `tokens` | `list[int]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `prompt_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `completion_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `finish_reason` | `str \| None` | `no` | `'stop'` | Optional constructor field; defaults to `'stop'`. |
| `mtp_drafts` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `mtp_accepted` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `new_text` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. |
| `finished` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `mtp_drafts` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `mtp_accepted` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |

**Returns**

- Constructs: `vllm_mlx.engine.base.GenerationOutput`

**Exceptions and behavior**

Class `GenerationOutput` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L18-L37](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L18-L37).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.EngineBusy" markdown="1">
<summary><code>vllm_mlx.engine.base.EngineBusy</code> · class</summary>

```python
vllm_mlx.engine.base.EngineBusy()
```

Raised when a serialized engine route is already serving a request.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Constructs: `vllm_mlx.engine.base.EngineBusy`

**Exceptions and behavior**

Class `EngineBusy` derives from `RuntimeError` and declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L40-L43](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L40-L43).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.suspend_cancellation" markdown="1">
<summary><code>vllm_mlx.engine.base.suspend_cancellation</code> · function</summary>

```python
vllm_mlx.engine.base.suspend_cancellation() -> not annotated
```

Temporarily clear task cancellation so cleanup can finish deterministically.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Function `suspend_cancellation` calls `asyncio.current_task`, `getattr`, `cancelling`, `range`; yields values incrementally; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L47-L67](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L47-L67).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.run_blocking_startup_work" markdown="1">
<summary><code>vllm_mlx.engine.base.run_blocking_startup_work</code> · function</summary>

```python
async vllm_mlx.engine.base.run_blocking_startup_work(work: Callable[[], Any]) -> None
```

Run blocking startup work off-loop without leaking cancellation races.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `work` | `Callable[[], Any]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `run_blocking_startup_work` calls `asyncio.create_task`, `asyncio.to_thread`, `asyncio.shield`, `suspend_cancellation`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L70-L84](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L70-L84).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.cleanup_startup_cancellation" markdown="1">
<summary><code>vllm_mlx.engine.base.cleanup_startup_cancellation</code> · function</summary>

```python
async vllm_mlx.engine.base.cleanup_startup_cancellation(cleanup: Callable[[], Awaitable[None]]) -> None
```

Run startup cleanup without letting cleanup failures replace cancellation.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cleanup` | `Callable[[], Awaitable[None]]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `cleanup_startup_cancellation` calls `suspend_cancellation`, `cleanup`, `isinstance`, `logger.error`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L87-L98](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L87-L98).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine</code> · class</summary>

```python
vllm_mlx.engine.base.BaseEngine()
```

Abstract base class for inference engines.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Constructs: `vllm_mlx.engine.base.BaseEngine`

**Exceptions and behavior**

Class `BaseEngine` derives from `ABC` and declares 16 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L101-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L101-L288).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.model_name" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.model_name</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.model_name() -> str
```

Get the model name.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`

**Exceptions and behavior**

Method `BaseEngine.model_name` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L111-L113](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L111-L113).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.is_mllm" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.is_mllm</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.is_mllm() -> bool
```

Check if this is a multimodal model.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`

**Exceptions and behavior**

Method `BaseEngine.is_mllm` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L117-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L117-L119).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.tokenizer" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.tokenizer</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.tokenizer() -> Any
```

Get the tokenizer.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Any`

**Exceptions and behavior**

Method `BaseEngine.tokenizer` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L123-L125](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L123-L125).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format() -> bool
```

Whether to preserve native tool message format.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `getattr(self, '_preserve_native_tool_format', False)`

**Exceptions and behavior**

Method `BaseEngine.preserve_native_tool_format` calls `getattr`; returns `getattr(self, '_preserve_native_tool_format', False)`.
No direct `raise` statement appears in this definition.

[View source #L128-L135](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L128-L135).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format(value: bool) -> None
```

Enable or disable preservation of model-native tool messages.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `value` | `bool` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `BaseEngine.preserve_native_tool_format` updates `self._preserve_native_tool_format`.
No direct `raise` statement appears in this definition.

[View source #L138-L141](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L138-L141).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.prepare_for_start" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.prepare_for_start</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.prepare_for_start() -> None
```

Run blocking startup work before async engine start.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `BaseEngine.prepare_for_start` returns `None`.
No direct `raise` statement appears in this definition.

[View source #L143-L150](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L143-L150).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.start" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.start</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.start() -> None
```

Start the engine (load model if not loaded).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `BaseEngine.start` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L153-L155](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L153-L155).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.stop" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.stop</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.stop() -> None
```

Stop the engine and cleanup resources.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `BaseEngine.stop` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L158-L160](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L158-L160).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.generate" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.generate</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] | None = None, **kwargs) -> GenerationOutput
```

Generate a complete response (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Input text |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `stop` | `list[str] \| None` | `no` | `None` | Stop sequences |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `GenerationOutput`

**Exceptions and behavior**

Method `BaseEngine.generate` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L163-L186](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L163-L186).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.stream_generate" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.stream_generate</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.stream_generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Stream generation token by token.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Input text |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `stop` | `list[str] \| None` | `no` | `None` | Stop sequences |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`

**Exceptions and behavior**

Method `BaseEngine.stream_generate` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L189-L212](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L189-L212).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.chat" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.chat</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] | None = None, images: list[str] | None = None, videos: list[str] | None = None, **kwargs) -> GenerationOutput
```

Chat completion (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | List of chat messages |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `tools` | `list[dict] \| None` | `no` | `None` | Optional tool definitions |
| `images` | `list[str] \| None` | `no` | `None` | Optional image URLs/paths |
| `videos` | `list[str] \| None` | `no` | `None` | Optional video URLs/paths |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `GenerationOutput`

**Exceptions and behavior**

Method `BaseEngine.chat` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L215-L242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L215-L242).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.stream_chat" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.stream_chat</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.stream_chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] | None = None, images: list[str] | None = None, videos: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Stream chat completion token by token.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | List of chat messages |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `tools` | `list[dict] \| None` | `no` | `None` | Optional tool definitions |
| `images` | `list[str] \| None` | `no` | `None` | Optional image URLs/paths |
| `videos` | `list[str] \| None` | `no` | `None` | Optional video URLs/paths |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`

**Exceptions and behavior**

Method `BaseEngine.stream_chat` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.

[View source #L245-L272](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L245-L272).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.get_stats" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.get_stats</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.get_stats() -> dict[str, Any]
```

Get engine statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any]`
- Direct return expressions: `{}`

**Exceptions and behavior**

Method `BaseEngine.get_stats` returns `{}`.
No direct `raise` statement appears in this definition.

[View source #L274-L276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L274-L276).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.get_cache_stats" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.get_cache_stats</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.get_cache_stats() -> dict[str, Any] | None
```

Get cache statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any] | None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `BaseEngine.get_cache_stats` returns `None`.
No direct `raise` statement appears in this definition.

[View source #L278-L280](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L278-L280).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.clear_runtime_caches" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.clear_runtime_caches</code> · method</summary>

```python
vllm_mlx.engine.base.BaseEngine.clear_runtime_caches() -> dict[str, Any] | None
```

Clear engine-managed runtime caches.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any] | None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `BaseEngine.clear_runtime_caches` returns `None`.
No direct `raise` statement appears in this definition.

[View source #L282-L284](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L282-L284).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.base.BaseEngine.abort_request" markdown="1">
<summary><code>vllm_mlx.engine.base.BaseEngine.abort_request</code> · method</summary>

```python
async vllm_mlx.engine.base.BaseEngine.abort_request(request_id: str) -> bool
```

Abort an active or queued request when the engine supports it.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`

**Exceptions and behavior**

Method `BaseEngine.abort_request` returns `False`.
No direct `raise` statement appears in this definition.

[View source #L286-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L286-L288).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`GenerationOutput`](#contract-vllm_mlx.engine.base.GenerationOutput) | class | `GenerationOutput(text: str, tokens: list[int] = field(default_factory=list), prompt_tokens: int = 0, completion_tokens: int = 0, finish_reason: str \| None = 'stop', mtp_drafts: int = 0, mtp_accepted: int = 0, new_text: str = '', finished: bool = True, mtp_drafts: int = 0, mtp_accepted: int = 0)` | Output from generation. | [#L18-L37](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L18-L37) |
| [`EngineBusy`](#contract-vllm_mlx.engine.base.EngineBusy) | class | `EngineBusy()` | Raised when a serialized engine route is already serving a request. | [#L40-L43](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L40-L43) |
| [`suspend_cancellation`](#contract-vllm_mlx.engine.base.suspend_cancellation) | function | `suspend_cancellation() -> not annotated` | Temporarily clear task cancellation so cleanup can finish deterministically. | [#L47-L67](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L47-L67) |
| [`run_blocking_startup_work`](#contract-vllm_mlx.engine.base.run_blocking_startup_work) | function | `async run_blocking_startup_work(work: Callable[[], Any]) -> None` | Run blocking startup work off-loop without leaking cancellation races. | [#L70-L84](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L70-L84) |
| [`cleanup_startup_cancellation`](#contract-vllm_mlx.engine.base.cleanup_startup_cancellation) | function | `async cleanup_startup_cancellation(cleanup: Callable[[], Awaitable[None]]) -> None` | Run startup cleanup without letting cleanup failures replace cancellation. | [#L87-L98](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L87-L98) |
| [`BaseEngine`](#contract-vllm_mlx.engine.base.BaseEngine) | class | `BaseEngine()` | Abstract base class for inference engines. | [#L101-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L101-L288) |
| [`BaseEngine.model_name`](#contract-vllm_mlx.engine.base.BaseEngine.model_name) | method | `BaseEngine.model_name() -> str` | Get the model name. | [#L111-L113](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L111-L113) |
| [`BaseEngine.is_mllm`](#contract-vllm_mlx.engine.base.BaseEngine.is_mllm) | method | `BaseEngine.is_mllm() -> bool` | Check if this is a multimodal model. | [#L117-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L117-L119) |
| [`BaseEngine.tokenizer`](#contract-vllm_mlx.engine.base.BaseEngine.tokenizer) | method | `BaseEngine.tokenizer() -> Any` | Get the tokenizer. | [#L123-L125](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L123-L125) |
| [`BaseEngine.preserve_native_tool_format`](#contract-vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format) | method | `BaseEngine.preserve_native_tool_format() -> bool` | Whether to preserve native tool message format. | [#L128-L135](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L128-L135) |
| [`BaseEngine.preserve_native_tool_format`](#contract-vllm_mlx.engine.base.BaseEngine.preserve_native_tool_format) | method | `BaseEngine.preserve_native_tool_format(value: bool) -> None` | Enable or disable preservation of model-native tool messages. | [#L138-L141](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L138-L141) |
| [`BaseEngine.prepare_for_start`](#contract-vllm_mlx.engine.base.BaseEngine.prepare_for_start) | method | `BaseEngine.prepare_for_start() -> None` | Run blocking startup work before async engine start. | [#L143-L150](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L143-L150) |
| [`BaseEngine.start`](#contract-vllm_mlx.engine.base.BaseEngine.start) | method | `async BaseEngine.start() -> None` | Start the engine (load model if not loaded). | [#L153-L155](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L153-L155) |
| [`BaseEngine.stop`](#contract-vllm_mlx.engine.base.BaseEngine.stop) | method | `async BaseEngine.stop() -> None` | Stop the engine and cleanup resources. | [#L158-L160](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L158-L160) |
| [`BaseEngine.generate`](#contract-vllm_mlx.engine.base.BaseEngine.generate) | method | `async BaseEngine.generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] \| None = None, **kwargs) -> GenerationOutput` | Generate a complete response (non-streaming). | [#L163-L186](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L163-L186) |
| [`BaseEngine.stream_generate`](#contract-vllm_mlx.engine.base.BaseEngine.stream_generate) | method | `async BaseEngine.stream_generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Stream generation token by token. | [#L189-L212](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L189-L212) |
| [`BaseEngine.chat`](#contract-vllm_mlx.engine.base.BaseEngine.chat) | method | `async BaseEngine.chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] \| None = None, images: list[str] \| None = None, videos: list[str] \| None = None, **kwargs) -> GenerationOutput` | Chat completion (non-streaming). | [#L215-L242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L215-L242) |
| [`BaseEngine.stream_chat`](#contract-vllm_mlx.engine.base.BaseEngine.stream_chat) | method | `async BaseEngine.stream_chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] \| None = None, images: list[str] \| None = None, videos: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Stream chat completion token by token. | [#L245-L272](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L245-L272) |
| [`BaseEngine.get_stats`](#contract-vllm_mlx.engine.base.BaseEngine.get_stats) | method | `BaseEngine.get_stats() -> dict[str, Any]` | Get engine statistics. | [#L274-L276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L274-L276) |
| [`BaseEngine.get_cache_stats`](#contract-vllm_mlx.engine.base.BaseEngine.get_cache_stats) | method | `BaseEngine.get_cache_stats() -> dict[str, Any] \| None` | Get cache statistics. | [#L278-L280](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L278-L280) |
| [`BaseEngine.clear_runtime_caches`](#contract-vllm_mlx.engine.base.BaseEngine.clear_runtime_caches) | method | `BaseEngine.clear_runtime_caches() -> dict[str, Any] \| None` | Clear engine-managed runtime caches. | [#L282-L284](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L282-L284) |
| [`BaseEngine.abort_request`](#contract-vllm_mlx.engine.base.BaseEngine.abort_request) | method | `async BaseEngine.abort_request(request_id: str) -> bool` | Abort an active or queued request when the engine supports it. | [#L286-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/base.py#L286-L288) |