# `vllm_mlx.engine.simple`

Simple engine for maximum single-user throughput.

[View the complete module source at #L1-L2912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1-L2912).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.engine.simple
    options:
      members:
        - _in_tracker
        - logger
        - _bind_worker_generation_streams
        - _seed_logits_processors
        - _sample_with_processors
        - _processors_can_retire
        - _processors_retired
        - _SpecPrefillCancelled
        - SimpleEngine
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.engine.simple._bind_worker_generation_streams" markdown="1">
<summary><code>vllm_mlx.engine.simple._bind_worker_generation_streams</code> · function</summary>

```python
vllm_mlx.engine.simple._bind_worker_generation_streams() -> None
```

Rebind mlx generation streams inside the current worker thread.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_bind_worker_generation_streams` calls `bind_generation_streams`.
No direct `raise` statement appears in this definition.

[View source #L48-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L48-L50).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._seed_logits_processors" markdown="1">
<summary><code>vllm_mlx.engine.simple._seed_logits_processors</code> · function</summary>

```python
vllm_mlx.engine.simple._seed_logits_processors(seed_tokens: mx.array | None, processors: list[Any] | None) -> list[Any] | None
```

Wrap logits processors so continuation decode sees the full prompt.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `seed_tokens` | `mx.array \| None` | `yes` | `none` | Required positional or keyword input. |
| `processors` | `list[Any] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[Any] | None`
- Direct return expressions: `None`; `list(processors)`; `[_wrap(processor) for processor in processors]`

**Exceptions and behavior**

Function `_seed_logits_processors` calls `list`, `_wrap`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L53-L77](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L53-L77).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._seed_logits_processors._wrap" markdown="1">
<summary><code>vllm_mlx.engine.simple._seed_logits_processors._wrap</code> · nested function</summary>

```python
vllm_mlx.engine.simple._seed_logits_processors._wrap(processor) -> not annotated
```

Nested Function `_seed_logits_processors._wrap` returns `_seeded`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `processor` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `_seeded`

**Exceptions and behavior**

Nested Function `_seed_logits_processors._wrap` returns `_seeded`.
No direct `raise` statement appears in this definition.

[View source #L63-L75](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L63-L75).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._seed_logits_processors._wrap._seeded" markdown="1">
<summary><code>vllm_mlx.engine.simple._seed_logits_processors._wrap._seeded</code> · nested function</summary>

```python
vllm_mlx.engine.simple._seed_logits_processors._wrap._seeded(tokens, logits) -> not annotated
```

Nested Function `_seed_logits_processors._wrap._seeded` calls `isinstance`, `mx.array`, `mx.concatenate`, `processor`; returns `processor(merged, logits)`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `logits` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `processor(merged, logits)`

**Exceptions and behavior**

Nested Function `_seed_logits_processors._wrap._seeded` calls `isinstance`, `mx.array`, `mx.concatenate`, `processor`; returns `processor(merged, logits)`.
No direct `raise` statement appears in this definition.

[View source #L64-L73](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L64-L73).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._sample_with_processors" markdown="1">
<summary><code>vllm_mlx.engine.simple._sample_with_processors</code> · function</summary>

```python
vllm_mlx.engine.simple._sample_with_processors(tokens: mx.array | None, logits: mx.array, sampler: Any, logits_processors: list[Any] | None) -> tuple[mx.array, mx.array]
```

Sample a token while honoring any active logits processors.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tokens` | `mx.array \| None` | `yes` | `none` | Required positional or keyword input. |
| `logits` | `mx.array` | `yes` | `none` | Required positional or keyword input. |
| `sampler` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `logits_processors` | `list[Any] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[mx.array, mx.array]`
- Direct return expressions: `(tok, logprobs)`

**Exceptions and behavior**

Function `_sample_with_processors` calls `processor`, `logits.squeeze`, `mx.logsumexp`, `sampler`; returns `(tok, logprobs)`.
No direct `raise` statement appears in this definition.

[View source #L80-L97](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L80-L97).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._processors_can_retire" markdown="1">
<summary><code>vllm_mlx.engine.simple._processors_can_retire</code> · function</summary>

```python
vllm_mlx.engine.simple._processors_can_retire(processors: list[Any] | None) -> bool
```

True when any processor advertises a retire-to-content transition.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `processors` | `list[Any] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `bool(processors) and any((isinstance(getattr(p, 'is_retired', None), bool) for p in processors))`

**Exceptions and behavior**

Function `_processors_can_retire` calls `os.getenv`, `bool`, `any`, `isinstance`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L100-L106](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L100-L106).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._processors_retired" markdown="1">
<summary><code>vllm_mlx.engine.simple._processors_retired</code> · function</summary>

```python
vllm_mlx.engine.simple._processors_retired(processors: list[Any] | None) -> bool
```

True when any retire-capable processor has entered its retired state.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `processors` | `list[Any] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `bool(processors) and any((getattr(p, 'is_retired', False) is True for p in processors))`

**Exceptions and behavior**

Function `_processors_retired` calls `os.getenv`, `bool`, `any`, `getattr`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L109-L115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L109-L115).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple._SpecPrefillCancelled" markdown="1">
<summary><code>vllm_mlx.engine.simple._SpecPrefillCancelled</code> · class</summary>

```python
vllm_mlx.engine.simple._SpecPrefillCancelled()
```

Cooperative cancellation sentinel for blocking SpecPrefill workers.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Constructs: `vllm_mlx.engine.simple._SpecPrefillCancelled`

**Exceptions and behavior**

Class `_SpecPrefillCancelled` derives from `Exception` and declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L118-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L118-L119).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine</code> · class</summary>

```python
vllm_mlx.engine.simple.SimpleEngine(model_name: str, trust_remote_code: bool = False, enable_cache: bool = True, force_mllm: bool = False, mtp: bool = False, mtp_num_draft_tokens: int = 1, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str | None = None, max_kv_size: int = 0, mllm_draft_model: str | None = None, mllm_draft_kind: str | None = None, mllm_draft_block_size: int | None = None)
```

Simple engine for direct model calls.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `yes` | `none` | HuggingFace model name or local path |
| `trust_remote_code` | `bool` | `no` | `False` | Whether to trust remote code |
| `enable_cache` | `bool` | `no` | `True` | Enable VLM cache for multimodal models |
| `force_mllm` | `bool` | `no` | `False` | Force loading as MLLM even if not auto-detected |
| `mtp` | `bool` | `no` | `False` | Enable native MTP speculative decoding (model must have MTP head) |
| `mtp_num_draft_tokens` | `int` | `no` | `1` | Draft tokens per speculative MTP step |
| `prefill_step_size` | `int` | `no` | `2048` | Chunk size for prompt prefill processing (default: 2048) |
| `specprefill_enabled` | `bool` | `no` | `False` | Enable SpecPrefill (attention-based sparse prefill) |
| `specprefill_threshold` | `int` | `no` | `8192` | Minimum suffix tokens to trigger SpecPrefill |
| `specprefill_keep_pct` | `float` | `no` | `0.3` | Fraction of tokens to keep (default: 0.3) |
| `specprefill_backbone_pct` | `float` | `no` | `0.0` | Fraction of chunks to reserve for evenly spaced coverage (default: 0.0) |
| `specprefill_draft_model` | `str \| None` | `no` | `None` | Path to small draft model for importance scoring |
| `max_kv_size` | `int` | `no` | `0` | Maximum KV cache size per sequence (0 = unbounded) |
| `mllm_draft_model` | `str \| None` | `no` | `None` | Optional MLLM speculative draft/assistant model path |
| `mllm_draft_kind` | `str \| None` | `no` | `None` | Optional mlx-vlm draft kind, for example "mtp" |
| `mllm_draft_block_size` | `int \| None` | `no` | `None` | Optional speculative block size for mlx-vlm |

**Returns**

- Constructs: `vllm_mlx.engine.simple.SimpleEngine`

**Exceptions and behavior**

Class `SimpleEngine` derives from `BaseEngine` and declares 31 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L122-L2912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L122-L2912).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.__init__" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.__init__</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.__init__(model_name: str, trust_remote_code: bool = False, enable_cache: bool = True, force_mllm: bool = False, mtp: bool = False, mtp_num_draft_tokens: int = 1, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str | None = None, max_kv_size: int = 0, mllm_draft_model: str | None = None, mllm_draft_kind: str | None = None, mllm_draft_block_size: int | None = None) -> not annotated
```

Initialize the simple engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `yes` | `none` | HuggingFace model name or local path |
| `trust_remote_code` | `bool` | `no` | `False` | Whether to trust remote code |
| `enable_cache` | `bool` | `no` | `True` | Enable VLM cache for multimodal models |
| `force_mllm` | `bool` | `no` | `False` | Force loading as MLLM even if not auto-detected |
| `mtp` | `bool` | `no` | `False` | Enable native MTP speculative decoding (model must have MTP head) |
| `mtp_num_draft_tokens` | `int` | `no` | `1` | Draft tokens per speculative MTP step |
| `prefill_step_size` | `int` | `no` | `2048` | Chunk size for prompt prefill processing (default: 2048) |
| `specprefill_enabled` | `bool` | `no` | `False` | Enable SpecPrefill (attention-based sparse prefill) |
| `specprefill_threshold` | `int` | `no` | `8192` | Minimum suffix tokens to trigger SpecPrefill |
| `specprefill_keep_pct` | `float` | `no` | `0.3` | Fraction of tokens to keep (default: 0.3) |
| `specprefill_backbone_pct` | `float` | `no` | `0.0` | Fraction of chunks to reserve for evenly spaced coverage (default: 0.0) |
| `specprefill_draft_model` | `str \| None` | `no` | `None` | Path to small draft model for importance scoring |
| `max_kv_size` | `int` | `no` | `0` | Maximum KV cache size per sequence (0 = unbounded) |
| `mllm_draft_model` | `str \| None` | `no` | `None` | Optional MLLM speculative draft/assistant model path |
| `mllm_draft_kind` | `str \| None` | `no` | `None` | Optional mlx-vlm draft kind, for example "mtp" |
| `mllm_draft_block_size` | `int \| None` | `no` | `None` | Optional speculative block size for mlx-vlm |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `SimpleEngine.__init__` updates `self._model_name`, `self._created_at`, `self._trust_remote_code`, `self._enable_cache`; calls `time.time`, `is_mllm_model`, `deque`, `asyncio.Lock`.
No direct `raise` statement appears in this definition.

[View source #L130-L257](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L130-L257).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._clone_cache_state" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._clone_cache_state</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._clone_cache_state(value: Any) -> Any
```

Copy cache state containers without duplicating immutable MLX arrays.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `value` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `Any`
- Direct return expressions: `tuple((SimpleEngine._clone_cache_state(v) for v in value))`; `[SimpleEngine._clone_cache_state(v) for v in value]`; `value`

**Exceptions and behavior**

Method `SimpleEngine._clone_cache_state` calls `isinstance`, `tuple`, `SimpleEngine._clone_cache_state`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L260-L266](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L260-L266).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._snapshot_prompt_cache" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._snapshot_prompt_cache</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._snapshot_prompt_cache(prompt_cache: list[Any]) -> list[Any]
```

Capture cache states without aliasing mutable state containers.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt_cache` | `list[Any]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[Any]`
- Direct return expressions: `[cls._clone_cache_state(c.state) for c in prompt_cache]`

**Exceptions and behavior**

Method `SimpleEngine._snapshot_prompt_cache` calls `cls._clone_cache_state`; returns `[cls._clone_cache_state(c.state) for c in prompt_cache]`.
No direct `raise` statement appears in this definition.

[View source #L269-L271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L269-L271).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._restore_prompt_cache" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._restore_prompt_cache</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._restore_prompt_cache(prompt_cache: list[Any], snapshot: list[Any]) -> None
```

Restore cache states without letting decode mutate the saved snapshot.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt_cache` | `list[Any]` | `yes` | `none` | Required positional or keyword input. |
| `snapshot` | `list[Any]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `SimpleEngine._restore_prompt_cache` calls `enumerate`, `cls._clone_cache_state`.
No direct `raise` statement appears in this definition.

[View source #L274-L279](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L274-L279).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._iter_cache_state_arrays" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._iter_cache_state_arrays</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._iter_cache_state_arrays(value: Any) -> not annotated
```

Method `SimpleEngine._iter_cache_state_arrays` calls `isinstance`, `SimpleEngine._iter_cache_state_arrays`, `hasattr`; yields values incrementally.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `value` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._iter_cache_state_arrays` calls `isinstance`, `SimpleEngine._iter_cache_state_arrays`, `hasattr`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L282-L287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L282-L287).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._eval_cache_snapshot" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._eval_cache_snapshot</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._eval_cache_snapshot(snapshot: list[Any]) -> None
```

Method `SimpleEngine._eval_cache_snapshot` calls `list`, `cls._iter_cache_state_arrays`, `mx.eval`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `snapshot` | `list[Any]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `SimpleEngine._eval_cache_snapshot` calls `list`, `cls._iter_cache_state_arrays`, `mx.eval`.
No direct `raise` statement appears in this definition.

[View source #L290-L293](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L290-L293).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._cache_class_is_system_snapshot_safe" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._cache_class_is_system_snapshot_safe</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._cache_class_is_system_snapshot_safe(cache_entry: Any) -> bool
```

Method `SimpleEngine._cache_class_is_system_snapshot_safe` calls `isinstance`, `type`; has 2 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_entry` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `isinstance(cache_entry, (KVCache, ArraysCache))`; `cache_type in {'KVCache', 'ArraysCache'}`

**Exceptions and behavior**

Method `SimpleEngine._cache_class_is_system_snapshot_safe` calls `isinstance`, `type`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L296-L303](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L296-L303).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._probe_system_kv_cache_support" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._probe_system_kv_cache_support</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._probe_system_kv_cache_support(model: Any, route: str) -> bool
```

Method `SimpleEngine._probe_system_kv_cache_support` calls `make_prompt_cache`, `bool`, `all`, `cls._cache_class_is_system_snapshot_safe`; has 2 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `route` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `supported`; `False`

**Exceptions and behavior**

Method `SimpleEngine._probe_system_kv_cache_support` calls `make_prompt_cache`, `bool`, `all`, `cls._cache_class_is_system_snapshot_safe`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L306-L331](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L306-L331).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.model_name" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.model_name</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.model_name() -> str
```

Get the model name.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`
- Direct return expressions: `self._model_name`

**Exceptions and behavior**

Method `SimpleEngine.model_name` returns `self._model_name`.
No direct `raise` statement appears in this definition.

[View source #L334-L336](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L334-L336).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.is_mllm" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.is_mllm</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.is_mllm() -> bool
```

Check if this is a multimodal model.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `self._is_mllm`

**Exceptions and behavior**

Method `SimpleEngine.is_mllm` returns `self._is_mllm`.
No direct `raise` statement appears in this definition.

[View source #L339-L341](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L339-L341).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.tokenizer" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.tokenizer</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.tokenizer() -> Any
```

Get the tokenizer.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Any`
- Direct return expressions: `None`; `getattr(self._model, 'processor', None)`; `self._model.tokenizer`

**Exceptions and behavior**

Method `SimpleEngine.tokenizer` calls `getattr`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L344-L350](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L344-L350).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._generation_lock_holder_summary" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._generation_lock_holder_summary</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._generation_lock_holder_summary() -> str
```

Method `SimpleEngine._generation_lock_holder_summary` calls `time.time`, `self._active_requests.items`, `info.get`, `round`; has 2 explicit return paths.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`
- Direct return expressions: `'none'`; `','.join(holders)`

**Exceptions and behavior**

Method `SimpleEngine._generation_lock_holder_summary` calls `time.time`, `self._active_requests.items`, `info.get`, `round`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L352-L371](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L352-L371).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._acquire_generation_slot" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._acquire_generation_slot</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._acquire_generation_slot(request_id: str) -> not annotated
```

Admission control for SimpleEngine's serialized MLX route.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._acquire_generation_slot` updates `self._generation_busy_rejections`, `self._generation_waiters`; calls `self._generation_lock.locked`, `EngineBusy`, `self._generation_lock_holder_summary`; yields values incrementally; can raise `EngineBusy`.
Directly raised exceptions: `EngineBusy`.

[View source #L374-L398](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L374-L398).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.prepare_for_start" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.prepare_for_start</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.prepare_for_start() -> None
```

Load the backing model off the serving event loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `SimpleEngine.prepare_for_start` updates `self._model`; calls `MLXMultimodalLM`, `MLXLanguageModel`, `self._model.load`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L400-L427](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L400-L427).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._uses_default_prepare_for_start" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._uses_default_prepare_for_start</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._uses_default_prepare_for_start() -> bool
```

Return True when prepare_for_start is the class implementation.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `method is SimpleEngine.prepare_for_start`

**Exceptions and behavior**

Method `SimpleEngine._uses_default_prepare_for_start` calls `getattr`; returns `method is SimpleEngine.prepare_for_start`.
No direct `raise` statement appears in this definition.

[View source #L429-L432](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L429-L432).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.start" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.start</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.start() -> None
```

Start the engine (load model if not loaded).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `SimpleEngine.start` updates `self._loaded`, `self._supports_system_kv_cache`, `self._text_model`, `self._text_tokenizer`; calls `self._uses_default_prepare_for_start`, `self.prepare_for_start`, `run_blocking_startup_work`, `logger.warning`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L434-L595](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L434-L595).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.stop" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.stop</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.stop() -> None
```

Stop the engine and cleanup resources.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `SimpleEngine.stop` updates `self._model`, `self._text_model`, `self._text_tokenizer`, `self._draft_model`; calls `self._system_kv_cache.clear`, `logger.info`.
No direct `raise` statement appears in this definition.

[View source #L597-L608](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L597-L608).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._should_route_text_through_text_model" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._should_route_text_through_text_model</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._should_route_text_through_text_model(*, mllm_draft_requested: bool = False) -> bool
```

Return whether text-only MLLM requests may use mlx_lm TextModel.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `mllm_draft_requested` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. |

**Returns**

- Type: `bool`
- Direct return expressions: `not (mllm_draft_requested and self._mllm_draft_model_path is not None)`

**Exceptions and behavior**

Method `SimpleEngine._should_route_text_through_text_model` returns `not (mllm_draft_requested and self._mllm_draft_model_path is not None)`.
No direct `raise` statement appears in this definition.

[View source #L610-L614](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L610-L614).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized(func, /, *args, request_id: str | None = None, on_cancel = None, **kwargs) -> not annotated
```

Run a blocking MLX operation under the generation lock.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `func` | `not annotated` | `yes` | `none` | Required positional-only input. |
| `*args` | `not annotated` | `no` | `none` | Additional variadic positional inputs accepted by this callable. |
| `request_id` | `str \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `on_cancel` | `not annotated` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `await asyncio.shield(task)`

**Exceptions and behavior**

Method `SimpleEngine._run_blocking_serialized` calls `id`, `self._acquire_generation_slot`, `time.time`, `asyncio.create_task`; awaits asynchronous work; returns `await asyncio.shield(task)`.
No direct `raise` statement appears in this definition.

[View source #L616-L666](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L616-L666).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized.run_bound" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized.run_bound</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized.run_bound() -> not annotated
```

Nested Function `SimpleEngine._run_blocking_serialized.run_bound` calls `_bind_worker_generation_streams`, `func`; returns `func(*args, **kwargs)`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `func(*args, **kwargs)`

**Exceptions and behavior**

Nested Function `SimpleEngine._run_blocking_serialized.run_bound` calls `_bind_worker_generation_streams`, `func`; returns `func(*args, **kwargs)`.
No direct `raise` statement appears in this definition.

[View source #L644-L646](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L644-L646).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.generate" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.generate</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] | None = None, **kwargs) -> GenerationOutput
```

Generate a complete response (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Input text |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `stop` | `list[str] \| None` | `no` | `None` | Stop sequences |
| `**kwargs` | `not annotated` | `no` | `none` | Additional parameters forwarded to stream_generate, including per-request `specprefill` / `specprefill_keep_pct` |

**Returns**

- Type: `GenerationOutput`
- Direct return expressions: `GenerationOutput(text='', finish_reason='stop')`; `GenerationOutput(text=text, tokens=list(last_output.tokens), prompt_tokens=last_output.prompt_tokens, completion_tokens…`

**Exceptions and behavior**

Method `SimpleEngine.generate` calls `self.start`, `self.stream_generate`, `GenerationOutput`, `clean_output_text`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L668-L730](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L668-L730).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._track_request_stream" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._track_request_stream</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._track_request_stream(source_gen: AsyncIterator[GenerationOutput], *, max_tokens: int = 0) -> AsyncIterator[GenerationOutput]
```

Yield-through wrapper that records per-request live state and final ``prompt_tokens``/``completion_tokens`` counters.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `source_gen` | `AsyncIterator[GenerationOutput]` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `no` | `0` | Optional keyword-only input; defaults to `0`. |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._track_request_stream` updates `self._num_running`, `self._total_requests_processed`, `self._total_prompt_tokens`, `self._total_completion_tokens`; calls `_in_tracker.get`, `_in_tracker.set`, `str`, `uuid.uuid4`; yields values incrementally; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L732-L817](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L732-L817).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.stream_generate" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.stream_generate</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.stream_generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Public stream-generate wrapper with request stats tracking.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `no` | `256` | Optional positional or keyword input; defaults to `256`. |
| `temperature` | `float` | `no` | `0.7` | Optional positional or keyword input; defaults to `0.7`. |
| `top_p` | `float` | `no` | `0.9` | Optional positional or keyword input; defaults to `0.9`. |
| `stop` | `list[str] \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine.stream_generate` calls `self._track_request_stream`, `self._stream_generate_impl`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L819-L840](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L819-L840).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_impl" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_impl</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_generate_impl(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Stream generation token by token.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Input text |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `stop` | `list[str] \| None` | `no` | `None` | Stop sequences |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._stream_generate_impl` calls `self.start`, `kwargs.pop`, `str`, `id`; awaits asynchronous work; yields values incrementally; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L842-L1012](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L842-L1012).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.chat" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.chat</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] | None = None, images: list[str] | None = None, videos: list[str] | None = None, **kwargs) -> GenerationOutput
```

Chat completion (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | List of chat messages |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `tools` | `list[dict] \| None` | `no` | `None` | Optional tool definitions |
| `images` | `list[str] \| None` | `no` | `None` | Optional image URLs/paths |
| `videos` | `list[str] \| None` | `no` | `None` | Optional video URLs/paths |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `GenerationOutput`
- Direct return expressions: `await aggregate_stream_chat()`; `GenerationOutput(text=text, prompt_tokens=output.prompt_tokens, completion_tokens=output.completion_tokens, finish_reas…`; `GenerationOutput(text=text, tokens=output.tokens, prompt_tokens=prompt_token_count, completion_tokens=len(output.tokens…`

**Exceptions and behavior**

Method `SimpleEngine.chat` calls `self.start`, `dict`, `kwargs.pop`, `aggregate_stream_chat`; awaits asynchronous work; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1014-L1144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1014-L1144).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.chat.aggregate_stream_chat" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.chat.aggregate_stream_chat</code> · nested function</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.chat.aggregate_stream_chat() -> GenerationOutput
```

Nested Function `SimpleEngine.chat.aggregate_stream_chat` calls `GenerationOutput`, `self.stream_chat`, `clean_output_text`, `list`; returns `GenerationOutput(text=text, tokens=list(final_output.tokens), prompt_tokens=final_output.prompt_tokens, completion_toke…`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `GenerationOutput`
- Direct return expressions: `GenerationOutput(text=text, tokens=list(final_output.tokens), prompt_tokens=final_output.prompt_tokens, completion_toke…`

**Exceptions and behavior**

Nested Function `SimpleEngine.chat.aggregate_stream_chat` calls `GenerationOutput`, `self.stream_chat`, `clean_output_text`, `list`; returns `GenerationOutput(text=text, tokens=list(final_output.tokens), prompt_tokens=final_output.prompt_tokens, completion_toke…`.
No direct `raise` statement appears in this definition.

[View source #L1046-L1069](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1046-L1069).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.stream_chat" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.stream_chat</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine.stream_chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] | None = None, images: list[str] | None = None, videos: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Public stream-chat wrapper with request stats tracking.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `no` | `256` | Optional positional or keyword input; defaults to `256`. |
| `temperature` | `float` | `no` | `0.7` | Optional positional or keyword input; defaults to `0.7`. |
| `top_p` | `float` | `no` | `0.9` | Optional positional or keyword input; defaults to `0.9`. |
| `tools` | `list[dict] \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `images` | `list[str] \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `videos` | `list[str] \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine.stream_chat` calls `self._track_request_stream`, `self._stream_chat_impl`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L1146-L1171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1146-L1171).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] | None = None, images: list[str] | None = None, videos: list[str] | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Stream chat completion token by token.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | List of chat messages |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `tools` | `list[dict] \| None` | `no` | `None` | Optional tool definitions |
| `images` | `list[str] \| None` | `no` | `None` | Optional image URLs/paths |
| `videos` | `list[str] \| None` | `no` | `None` | Optional video URLs/paths |
| `**kwargs` | `not annotated` | `no` | `none` | Additional model-specific parameters |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._stream_chat_impl` calls `self.start`, `dict`, `kwargs.pop`, `bool`; awaits asynchronous work; yields values incrementally; can raise `payload`; returns `None`.
Directly raised exceptions: `payload`.

[View source #L1173-L1794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1173-L1794).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.mllm_call_kwargs" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.mllm_call_kwargs</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.mllm_call_kwargs() -> dict
```

Nested Function `SimpleEngine._stream_chat_impl.mllm_call_kwargs` calls `dict`; returns `local_kwargs`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict`
- Direct return expressions: `local_kwargs`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl.mllm_call_kwargs` calls `dict`; returns `local_kwargs`.
No direct `raise` statement appears in this definition.

[View source #L1236-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1236-L1242).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.run_native_video" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.run_native_video</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.run_native_video() -> not annotated
```

Nested Function `SimpleEngine._stream_chat_impl.run_native_video` calls `mllm_call_kwargs`, `list`, `self._model.stream_chat`; returns `list(self._model.stream_chat(messages=messages, max_tokens=max_tokens, temperature=temperature, tools=template_tools, *…`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `list(self._model.stream_chat(messages=messages, max_tokens=max_tokens, temperature=temperature, tools=template_tools, *…`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl.run_native_video` calls `mllm_call_kwargs`, `list`, `self._model.stream_chat`; returns `list(self._model.stream_chat(messages=messages, max_tokens=max_tokens, temperature=temperature, tools=template_tools, *…`.
No direct `raise` statement appears in this definition.

[View source #L1299-L1309](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1299-L1309).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._to_msg_dict" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._to_msg_dict</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._to_msg_dict(m: Any) -> dict[str, Any]
```

Nested Function `SimpleEngine._stream_chat_impl._to_msg_dict` calls `isinstance`, `hasattr`, `m.model_dump`, `m.dict`; has 4 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `m` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict[str, Any]`
- Direct return expressions: `m`; `m.model_dump()`; `m.dict()`; `{'role': getattr(m, 'role', None), 'content': getattr(m, 'content', '')}`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._to_msg_dict` calls `isinstance`, `hasattr`, `m.model_dump`, `m.dict`; has 4 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1499-L1509](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1499-L1509).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._with_user" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._with_user</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._with_user(user_content: str) -> list[dict[str, Any]]
```

Nested Function `SimpleEngine._stream_chat_impl._with_user` calls `dict`, `msgs[-1].get`; returns `msgs`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `user_content` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[dict[str, Any]]`
- Direct return expressions: `msgs`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._with_user` calls `dict`, `msgs[-1].get`; returns `msgs`.
No direct `raise` statement appears in this definition.

[View source #L1519-L1525](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1519-L1525).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_response" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_response</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_response(resp: Any) -> None
```

Nested Function `SimpleEngine._stream_chat_impl._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resp` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L1609-L1612](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1609-L1612).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_done" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_done</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_done() -> None
```

Nested Function `SimpleEngine._stream_chat_impl._emit_done` calls `loop.call_soon_threadsafe`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._emit_done` calls `loop.call_soon_threadsafe`.
No direct `raise` statement appears in this definition.

[View source #L1614-L1615](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1614-L1615).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_error" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_error</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_error(exc: BaseException) -> None
```

Nested Function `SimpleEngine._stream_chat_impl._emit_error` calls `loop.call_soon_threadsafe`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `exc` | `BaseException` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._emit_error` calls `loop.call_soon_threadsafe`.
No direct `raise` statement appears in this definition.

[View source #L1617-L1618](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1617-L1618).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._run_with_cache" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._run_with_cache</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._run_with_cache() -> None
```

Nested Function `SimpleEngine._stream_chat_impl._run_with_cache` calls `make_sampler`, `make_prompt_cache`, `self._restore_prompt_cache`, `self._system_kv_cache.move_to_end`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._run_with_cache` calls `make_sampler`, `make_prompt_cache`, `self._restore_prompt_cache`, `self._system_kv_cache.move_to_end`.
No direct `raise` statement appears in this definition.

[View source #L1620-L1705](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1620-L1705).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._produce_responses" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._produce_responses</code> · nested function</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._produce_responses() -> None
```

Nested Function `SimpleEngine._stream_chat_impl._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_chat_impl._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1707-L1718](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1707-L1718).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill(prompt: str, tokens: list[int], max_tokens: int, temperature: float, top_p: float, stop: list[str] | None = None, specprefill_keep_pct: float | None = None, specprefill_backbone_pct: float | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

SpecPrefill path for non-MTP models (Nemotron, GPT-OSS, etc).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |
| `tokens` | `list[int]` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |
| `temperature` | `float` | `yes` | `none` | Required positional or keyword input. |
| `top_p` | `float` | `yes` | `none` | Required positional or keyword input. |
| `stop` | `list[str] \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `specprefill_keep_pct` | `float \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `specprefill_backbone_pct` | `float \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._stream_generate_specprefill` calls `len`, `Event`, `self._run_blocking_serialized`, `enumerate`; awaits asynchronous work; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L1796-L2000](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1796-L2000).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._request_cancel" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._request_cancel</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._request_cancel() -> None
```

Nested Function `SimpleEngine._stream_generate_specprefill._request_cancel` calls `cancel_requested.set`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_specprefill._request_cancel` calls `cancel_requested.set`.
No direct `raise` statement appears in this definition.

[View source #L1821-L1822](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1821-L1822).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._cancel_check" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._cancel_check</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._cancel_check() -> None
```

Nested Function `SimpleEngine._stream_generate_specprefill._cancel_check` calls `cancel_requested.is_set`, `_SpecPrefillCancelled`; can raise `_SpecPrefillCancelled`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_specprefill._cancel_check` calls `cancel_requested.is_set`, `_SpecPrefillCancelled`; can raise `_SpecPrefillCancelled`.
Directly raised exceptions: `_SpecPrefillCancelled`.

[View source #L1824-L1826](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1824-L1826).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_all" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_all</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_all() -> not annotated
```

Nested Function `SimpleEngine._stream_generate_specprefill._run_all` calls `_run_specprefill`, `logger.error`, `_run_normal`; has 2 explicit return paths.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `_run_specprefill()`; `_run_normal()`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_specprefill._run_all` calls `_run_specprefill`, `logger.error`, `_run_normal`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1828-L1835](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1828-L1835).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_specprefill" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_specprefill</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_specprefill() -> not annotated
```

Score tokens, sparse prefill, generate autoregressively.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `results`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_specprefill._run_specprefill` calls `make_prompt_cache`, `time.monotonic`, `score_tokens`, `_cancel_check`; returns `results`.
No direct `raise` statement appears in this definition.

[View source #L1837-L1939](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1837-L1939).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_normal" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_normal</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_normal() -> not annotated
```

Fallback: normal generation without specprefill.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `results`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_specprefill._run_normal` calls `self._model.stream_generate`, `_cancel_check`, `hasattr`, `str`; returns `results`.
No direct `raise` statement appears in this definition.

[View source #L1941-L1962](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1941-L1962).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text</code> · method</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_generate_text(messages: list[dict[str, Any]], max_tokens: int, temperature: float, top_p: float, tools: list | None = None, **kwargs) -> AsyncIterator[GenerationOutput]
```

Text-only generation via mlx_lm TextModel.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict[str, Any]]` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |
| `temperature` | `float` | `yes` | `none` | Required positional or keyword input. |
| `top_p` | `float` | `yes` | `none` | Required positional or keyword input. |
| `tools` | `list \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `AsyncIterator[GenerationOutput]`
- Yields values incrementally.

**Exceptions and behavior**

Method `SimpleEngine._stream_generate_text` calls `kwargs.pop`, `dict`, `threading.Event`, `os.environ.get`; awaits asynchronous work; yields values incrementally; can raise `payload`.
Directly raised exceptions: `payload`.

[View source #L2002-L2734](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2002-L2734).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text.make_cache_with_snapshot" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text.make_cache_with_snapshot</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text.make_cache_with_snapshot(text_model, system_kv_snapshot, _max_kv_size = self._max_kv_size) -> not annotated
```

Nested Function `SimpleEngine._stream_generate_text.make_cache_with_snapshot` calls `make_prompt_cache`, `SimpleEngine._restore_prompt_cache`, `mx.array`; returns `(backbone_cache, prompt_to_send)`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text_model` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `system_kv_snapshot` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `_max_kv_size` | `not annotated` | `no` | `self._max_kv_size` | Optional positional or keyword input; defaults to `self._max_kv_size`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `(backbone_cache, prompt_to_send)`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text.make_cache_with_snapshot` calls `make_prompt_cache`, `SimpleEngine._restore_prompt_cache`, `mx.array`; returns `(backbone_cache, prompt_to_send)`.
No direct `raise` statement appears in this definition.

[View source #L2159-L2176](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2159-L2176).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_response" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_response</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_response(resp: Any) -> None
```

Nested Function `SimpleEngine._stream_generate_text._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resp` | `Any` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L2272-L2275](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2272-L2275).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_done" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_done</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_done() -> None
```

Nested Function `SimpleEngine._stream_generate_text._emit_done` calls `loop.call_soon_threadsafe`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._emit_done` calls `loop.call_soon_threadsafe`.
No direct `raise` statement appears in this definition.

[View source #L2277-L2278](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2277-L2278).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_error" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_error</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_error(exc: BaseException) -> None
```

Nested Function `SimpleEngine._stream_generate_text._emit_error` calls `loop.call_soon_threadsafe`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `exc` | `BaseException` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._emit_error` calls `loop.call_soon_threadsafe`.
No direct `raise` statement appears in this definition.

[View source #L2280-L2281](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2280-L2281).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._seed_from_last_response" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._seed_from_last_response</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._seed_from_last_response(prompt_cache, last_resp) -> not annotated
```

Nested Function `SimpleEngine._stream_generate_text._seed_from_last_response` calls `getattr`, `cache_module.trim_prompt_cache`, `mx.array`, `self._text_tokenizer.encode`; has 2 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `last_resp` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `mx.array([last_tok], dtype=mx.uint32)`; `mx.array(self._text_tokenizer.encode(getattr(last_resp, 'text', '')), dtype=mx.uint32)`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._seed_from_last_response` calls `getattr`, `cache_module.trim_prompt_cache`, `mx.array`, `self._text_tokenizer.encode`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2283-L2291](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2283-L2291).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._resume_after_processor_retirement" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._resume_after_processor_retirement</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._resume_after_processor_retirement(model, prompt_cache, prompt, remaining_tokens: int) -> None
```

Nested Function `SimpleEngine._stream_generate_text._resume_after_processor_retirement` calls `dict`, `hasattr`, `model.make_mtp_cache`, `mlx_stream_generate`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `remaining_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._resume_after_processor_retirement` calls `dict`, `hasattr`, `model.make_mtp_cache`, `mlx_stream_generate`.
No direct `raise` statement appears in this definition.

[View source #L2293-L2320](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2293-L2320).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_all" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_all</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_all() -> not annotated
```

Nested Function `SimpleEngine._stream_generate_text._run_all` calls `_processors_can_retire`, `hasattr`, `logger.info`, `make_prompt_cache`; returns `None`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._run_all` calls `_processors_can_retire`, `hasattr`, `logger.info`, `make_prompt_cache`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L2323-L2485](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2323-L2485).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_specprefill" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_specprefill</code> · nested function</summary>

```python
vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_specprefill(model, bc, use_mtp) -> not annotated
```

Score tokens, sparse prefill, then continue on the standard decode path.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `bc` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `use_mtp` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._run_specprefill` calls `make_prompt_cache`, `time.monotonic`, `score_tokens`, `select_chunks`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L2487-L2664](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2487-L2664).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._produce_responses" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._produce_responses</code> · nested function</summary>

```python
async vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._produce_responses() -> None
```

Nested Function `SimpleEngine._stream_generate_text._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `SimpleEngine._stream_generate_text._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L2666-L2677](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2666-L2677).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.get_stats" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.get_stats</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.get_stats() -> dict[str, Any]
```

Get engine statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any]`
- Direct return expressions: `stats`

**Exceptions and behavior**

Method `SimpleEngine.get_stats` calls `sum`, `time.time`, `self._active_requests.values`, `dict`; returns `stats`.
No direct `raise` statement appears in this definition.

[View source #L2736-L2858](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2736-L2858).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.get_cache_stats" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.get_cache_stats</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.get_cache_stats() -> dict[str, Any] | None
```

Get cache statistics for the system-prompt KV LRU plus, when the model is multimodal, the MLLM's own cache stats.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any] | None`
- Direct return expressions: `result or None`

**Exceptions and behavior**

Method `SimpleEngine.get_cache_stats` calls `dict`, `round`, `len`, `self._model.get_cache_stats`; returns `result or None`.
No direct `raise` statement appears in this definition.

[View source #L2860-L2878](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2860-L2878).

</details>

<details class="api-contract" id="contract-vllm_mlx.engine.simple.SimpleEngine.clear_runtime_caches" markdown="1">
<summary><code>vllm_mlx.engine.simple.SimpleEngine.clear_runtime_caches</code> · method</summary>

```python
vllm_mlx.engine.simple.SimpleEngine.clear_runtime_caches() -> dict[str, Any] | None
```

Clear engine-managed runtime caches.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any] | None`
- Direct return expressions: `result or None`

**Exceptions and behavior**

Method `SimpleEngine.clear_runtime_caches` calls `len`, `any`, `self._system_kv_cache_stats.values`, `self._system_kv_cache.clear`; returns `result or None`.
No direct `raise` statement appears in this definition.

[View source #L2880-L2912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2880-L2912).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`_bind_worker_generation_streams`](#contract-vllm_mlx.engine.simple._bind_worker_generation_streams) | function | `_bind_worker_generation_streams() -> None` | Rebind mlx generation streams inside the current worker thread. | [#L48-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L48-L50) |
| [`_seed_logits_processors`](#contract-vllm_mlx.engine.simple._seed_logits_processors) | function | `_seed_logits_processors(seed_tokens: mx.array \| None, processors: list[Any] \| None) -> list[Any] \| None` | Wrap logits processors so continuation decode sees the full prompt. | [#L53-L77](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L53-L77) |
| [`_seed_logits_processors._wrap`](#contract-vllm_mlx.engine.simple._seed_logits_processors._wrap) | nested function | `_seed_logits_processors._wrap(processor) -> not annotated` | Nested Function `_seed_logits_processors._wrap` returns `_seeded`. | [#L63-L75](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L63-L75) |
| [`_seed_logits_processors._wrap._seeded`](#contract-vllm_mlx.engine.simple._seed_logits_processors._wrap._seeded) | nested function | `_seed_logits_processors._wrap._seeded(tokens, logits) -> not annotated` | Nested Function `_seed_logits_processors._wrap._seeded` calls `isinstance`, `mx.array`, `mx.concatenate`, `processor`; returns `processor(merged, logits)`. | [#L64-L73](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L64-L73) |
| [`_sample_with_processors`](#contract-vllm_mlx.engine.simple._sample_with_processors) | function | `_sample_with_processors(tokens: mx.array \| None, logits: mx.array, sampler: Any, logits_processors: list[Any] \| None) -> tuple[mx.array, mx.array]` | Sample a token while honoring any active logits processors. | [#L80-L97](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L80-L97) |
| [`_processors_can_retire`](#contract-vllm_mlx.engine.simple._processors_can_retire) | function | `_processors_can_retire(processors: list[Any] \| None) -> bool` | True when any processor advertises a retire-to-content transition. | [#L100-L106](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L100-L106) |
| [`_processors_retired`](#contract-vllm_mlx.engine.simple._processors_retired) | function | `_processors_retired(processors: list[Any] \| None) -> bool` | True when any retire-capable processor has entered its retired state. | [#L109-L115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L109-L115) |
| [`_SpecPrefillCancelled`](#contract-vllm_mlx.engine.simple._SpecPrefillCancelled) | class | `_SpecPrefillCancelled()` | Cooperative cancellation sentinel for blocking SpecPrefill workers. | [#L118-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L118-L119) |
| [`SimpleEngine`](#contract-vllm_mlx.engine.simple.SimpleEngine) | class | `SimpleEngine(model_name: str, trust_remote_code: bool = False, enable_cache: bool = True, force_mllm: bool = False, mtp: bool = False, mtp_num_draft_tokens: int = 1, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str \| None = None, max_kv_size: int = 0, mllm_draft_model: str \| None = None, mllm_draft_kind: str \| None = None, mllm_draft_block_size: int \| None = None)` | Simple engine for direct model calls. | [#L122-L2912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L122-L2912) |
| [`SimpleEngine.__init__`](#contract-vllm_mlx.engine.simple.SimpleEngine.__init__) | method | `SimpleEngine.__init__(model_name: str, trust_remote_code: bool = False, enable_cache: bool = True, force_mllm: bool = False, mtp: bool = False, mtp_num_draft_tokens: int = 1, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str \| None = None, max_kv_size: int = 0, mllm_draft_model: str \| None = None, mllm_draft_kind: str \| None = None, mllm_draft_block_size: int \| None = None) -> not annotated` | Initialize the simple engine. | [#L130-L257](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L130-L257) |
| [`SimpleEngine._clone_cache_state`](#contract-vllm_mlx.engine.simple.SimpleEngine._clone_cache_state) | method | `SimpleEngine._clone_cache_state(value: Any) -> Any` | Copy cache state containers without duplicating immutable MLX arrays. | [#L260-L266](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L260-L266) |
| [`SimpleEngine._snapshot_prompt_cache`](#contract-vllm_mlx.engine.simple.SimpleEngine._snapshot_prompt_cache) | method | `SimpleEngine._snapshot_prompt_cache(prompt_cache: list[Any]) -> list[Any]` | Capture cache states without aliasing mutable state containers. | [#L269-L271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L269-L271) |
| [`SimpleEngine._restore_prompt_cache`](#contract-vllm_mlx.engine.simple.SimpleEngine._restore_prompt_cache) | method | `SimpleEngine._restore_prompt_cache(prompt_cache: list[Any], snapshot: list[Any]) -> None` | Restore cache states without letting decode mutate the saved snapshot. | [#L274-L279](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L274-L279) |
| [`SimpleEngine._iter_cache_state_arrays`](#contract-vllm_mlx.engine.simple.SimpleEngine._iter_cache_state_arrays) | method | `SimpleEngine._iter_cache_state_arrays(value: Any) -> not annotated` | Method `SimpleEngine._iter_cache_state_arrays` calls `isinstance`, `SimpleEngine._iter_cache_state_arrays`, `hasattr`; yields values incrementally. | [#L282-L287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L282-L287) |
| [`SimpleEngine._eval_cache_snapshot`](#contract-vllm_mlx.engine.simple.SimpleEngine._eval_cache_snapshot) | method | `SimpleEngine._eval_cache_snapshot(snapshot: list[Any]) -> None` | Method `SimpleEngine._eval_cache_snapshot` calls `list`, `cls._iter_cache_state_arrays`, `mx.eval`. | [#L290-L293](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L290-L293) |
| [`SimpleEngine._cache_class_is_system_snapshot_safe`](#contract-vllm_mlx.engine.simple.SimpleEngine._cache_class_is_system_snapshot_safe) | method | `SimpleEngine._cache_class_is_system_snapshot_safe(cache_entry: Any) -> bool` | Method `SimpleEngine._cache_class_is_system_snapshot_safe` calls `isinstance`, `type`; has 2 explicit return paths. | [#L296-L303](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L296-L303) |
| [`SimpleEngine._probe_system_kv_cache_support`](#contract-vllm_mlx.engine.simple.SimpleEngine._probe_system_kv_cache_support) | method | `SimpleEngine._probe_system_kv_cache_support(model: Any, route: str) -> bool` | Method `SimpleEngine._probe_system_kv_cache_support` calls `make_prompt_cache`, `bool`, `all`, `cls._cache_class_is_system_snapshot_safe`; has 2 explicit return paths. | [#L306-L331](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L306-L331) |
| [`SimpleEngine.model_name`](#contract-vllm_mlx.engine.simple.SimpleEngine.model_name) | method | `SimpleEngine.model_name() -> str` | Get the model name. | [#L334-L336](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L334-L336) |
| [`SimpleEngine.is_mllm`](#contract-vllm_mlx.engine.simple.SimpleEngine.is_mllm) | method | `SimpleEngine.is_mllm() -> bool` | Check if this is a multimodal model. | [#L339-L341](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L339-L341) |
| [`SimpleEngine.tokenizer`](#contract-vllm_mlx.engine.simple.SimpleEngine.tokenizer) | method | `SimpleEngine.tokenizer() -> Any` | Get the tokenizer. | [#L344-L350](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L344-L350) |
| [`SimpleEngine._generation_lock_holder_summary`](#contract-vllm_mlx.engine.simple.SimpleEngine._generation_lock_holder_summary) | method | `SimpleEngine._generation_lock_holder_summary() -> str` | Method `SimpleEngine._generation_lock_holder_summary` calls `time.time`, `self._active_requests.items`, `info.get`, `round`; has 2 explicit return paths. | [#L352-L371](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L352-L371) |
| [`SimpleEngine._acquire_generation_slot`](#contract-vllm_mlx.engine.simple.SimpleEngine._acquire_generation_slot) | method | `async SimpleEngine._acquire_generation_slot(request_id: str) -> not annotated` | Admission control for SimpleEngine's serialized MLX route. | [#L374-L398](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L374-L398) |
| [`SimpleEngine.prepare_for_start`](#contract-vllm_mlx.engine.simple.SimpleEngine.prepare_for_start) | method | `SimpleEngine.prepare_for_start() -> None` | Load the backing model off the serving event loop. | [#L400-L427](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L400-L427) |
| [`SimpleEngine._uses_default_prepare_for_start`](#contract-vllm_mlx.engine.simple.SimpleEngine._uses_default_prepare_for_start) | method | `SimpleEngine._uses_default_prepare_for_start() -> bool` | Return True when prepare_for_start is the class implementation. | [#L429-L432](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L429-L432) |
| [`SimpleEngine.start`](#contract-vllm_mlx.engine.simple.SimpleEngine.start) | method | `async SimpleEngine.start() -> None` | Start the engine (load model if not loaded). | [#L434-L595](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L434-L595) |
| [`SimpleEngine.stop`](#contract-vllm_mlx.engine.simple.SimpleEngine.stop) | method | `async SimpleEngine.stop() -> None` | Stop the engine and cleanup resources. | [#L597-L608](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L597-L608) |
| [`SimpleEngine._should_route_text_through_text_model`](#contract-vllm_mlx.engine.simple.SimpleEngine._should_route_text_through_text_model) | method | `SimpleEngine._should_route_text_through_text_model(*, mllm_draft_requested: bool = False) -> bool` | Return whether text-only MLLM requests may use mlx_lm TextModel. | [#L610-L614](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L610-L614) |
| [`SimpleEngine._run_blocking_serialized`](#contract-vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized) | method | `async SimpleEngine._run_blocking_serialized(func, /, *args, request_id: str \| None = None, on_cancel = None, **kwargs) -> not annotated` | Run a blocking MLX operation under the generation lock. | [#L616-L666](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L616-L666) |
| [`SimpleEngine._run_blocking_serialized.run_bound`](#contract-vllm_mlx.engine.simple.SimpleEngine._run_blocking_serialized.run_bound) | nested function | `SimpleEngine._run_blocking_serialized.run_bound() -> not annotated` | Nested Function `SimpleEngine._run_blocking_serialized.run_bound` calls `_bind_worker_generation_streams`, `func`; returns `func(*args, **kwargs)`. | [#L644-L646](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L644-L646) |
| [`SimpleEngine.generate`](#contract-vllm_mlx.engine.simple.SimpleEngine.generate) | method | `async SimpleEngine.generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] \| None = None, **kwargs) -> GenerationOutput` | Generate a complete response (non-streaming). | [#L668-L730](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L668-L730) |
| [`SimpleEngine._track_request_stream`](#contract-vllm_mlx.engine.simple.SimpleEngine._track_request_stream) | method | `async SimpleEngine._track_request_stream(source_gen: AsyncIterator[GenerationOutput], *, max_tokens: int = 0) -> AsyncIterator[GenerationOutput]` | Yield-through wrapper that records per-request live state and final ``prompt_tokens``/``completion_tokens`` counters. | [#L732-L817](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L732-L817) |
| [`SimpleEngine.stream_generate`](#contract-vllm_mlx.engine.simple.SimpleEngine.stream_generate) | method | `async SimpleEngine.stream_generate(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Public stream-generate wrapper with request stats tracking. | [#L819-L840](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L819-L840) |
| [`SimpleEngine._stream_generate_impl`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_impl) | method | `async SimpleEngine._stream_generate_impl(prompt: str, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, stop: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Stream generation token by token. | [#L842-L1012](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L842-L1012) |
| [`SimpleEngine.chat`](#contract-vllm_mlx.engine.simple.SimpleEngine.chat) | method | `async SimpleEngine.chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] \| None = None, images: list[str] \| None = None, videos: list[str] \| None = None, **kwargs) -> GenerationOutput` | Chat completion (non-streaming). | [#L1014-L1144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1014-L1144) |
| [`SimpleEngine.chat.aggregate_stream_chat`](#contract-vllm_mlx.engine.simple.SimpleEngine.chat.aggregate_stream_chat) | nested function | `async SimpleEngine.chat.aggregate_stream_chat() -> GenerationOutput` | Nested Function `SimpleEngine.chat.aggregate_stream_chat` calls `GenerationOutput`, `self.stream_chat`, `clean_output_text`, `list`; returns `GenerationOutput(text=text, tokens=list(final_output.tokens), prompt_tokens=final_output.prompt_tokens, completion_toke…`. | [#L1046-L1069](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1046-L1069) |
| [`SimpleEngine.stream_chat`](#contract-vllm_mlx.engine.simple.SimpleEngine.stream_chat) | method | `async SimpleEngine.stream_chat(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] \| None = None, images: list[str] \| None = None, videos: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Public stream-chat wrapper with request stats tracking. | [#L1146-L1171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1146-L1171) |
| [`SimpleEngine._stream_chat_impl`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl) | method | `async SimpleEngine._stream_chat_impl(messages: list[dict[str, Any]], max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, tools: list[dict] \| None = None, images: list[str] \| None = None, videos: list[str] \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Stream chat completion token by token. | [#L1173-L1794](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1173-L1794) |
| [`SimpleEngine._stream_chat_impl.mllm_call_kwargs`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.mllm_call_kwargs) | nested function | `SimpleEngine._stream_chat_impl.mllm_call_kwargs() -> dict` | Nested Function `SimpleEngine._stream_chat_impl.mllm_call_kwargs` calls `dict`; returns `local_kwargs`. | [#L1236-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1236-L1242) |
| [`SimpleEngine._stream_chat_impl.run_native_video`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl.run_native_video) | nested function | `SimpleEngine._stream_chat_impl.run_native_video() -> not annotated` | Nested Function `SimpleEngine._stream_chat_impl.run_native_video` calls `mllm_call_kwargs`, `list`, `self._model.stream_chat`; returns `list(self._model.stream_chat(messages=messages, max_tokens=max_tokens, temperature=temperature, tools=template_tools, *…`. | [#L1299-L1309](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1299-L1309) |
| [`SimpleEngine._stream_chat_impl._to_msg_dict`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._to_msg_dict) | nested function | `SimpleEngine._stream_chat_impl._to_msg_dict(m: Any) -> dict[str, Any]` | Nested Function `SimpleEngine._stream_chat_impl._to_msg_dict` calls `isinstance`, `hasattr`, `m.model_dump`, `m.dict`; has 4 explicit return paths. | [#L1499-L1509](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1499-L1509) |
| [`SimpleEngine._stream_chat_impl._with_user`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._with_user) | nested function | `SimpleEngine._stream_chat_impl._with_user(user_content: str) -> list[dict[str, Any]]` | Nested Function `SimpleEngine._stream_chat_impl._with_user` calls `dict`, `msgs[-1].get`; returns `msgs`. | [#L1519-L1525](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1519-L1525) |
| [`SimpleEngine._stream_chat_impl._emit_response`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_response) | nested function | `SimpleEngine._stream_chat_impl._emit_response(resp: Any) -> None` | Nested Function `SimpleEngine._stream_chat_impl._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`. | [#L1609-L1612](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1609-L1612) |
| [`SimpleEngine._stream_chat_impl._emit_done`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_done) | nested function | `SimpleEngine._stream_chat_impl._emit_done() -> None` | Nested Function `SimpleEngine._stream_chat_impl._emit_done` calls `loop.call_soon_threadsafe`. | [#L1614-L1615](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1614-L1615) |
| [`SimpleEngine._stream_chat_impl._emit_error`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._emit_error) | nested function | `SimpleEngine._stream_chat_impl._emit_error(exc: BaseException) -> None` | Nested Function `SimpleEngine._stream_chat_impl._emit_error` calls `loop.call_soon_threadsafe`. | [#L1617-L1618](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1617-L1618) |
| [`SimpleEngine._stream_chat_impl._run_with_cache`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._run_with_cache) | nested function | `SimpleEngine._stream_chat_impl._run_with_cache() -> None` | Nested Function `SimpleEngine._stream_chat_impl._run_with_cache` calls `make_sampler`, `make_prompt_cache`, `self._restore_prompt_cache`, `self._system_kv_cache.move_to_end`. | [#L1620-L1705](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1620-L1705) |
| [`SimpleEngine._stream_chat_impl._produce_responses`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_chat_impl._produce_responses) | nested function | `async SimpleEngine._stream_chat_impl._produce_responses() -> None` | Nested Function `SimpleEngine._stream_chat_impl._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work. | [#L1707-L1718](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1707-L1718) |
| [`SimpleEngine._stream_generate_specprefill`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill) | method | `async SimpleEngine._stream_generate_specprefill(prompt: str, tokens: list[int], max_tokens: int, temperature: float, top_p: float, stop: list[str] \| None = None, specprefill_keep_pct: float \| None = None, specprefill_backbone_pct: float \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | SpecPrefill path for non-MTP models (Nemotron, GPT-OSS, etc). | [#L1796-L2000](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1796-L2000) |
| [`SimpleEngine._stream_generate_specprefill._request_cancel`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._request_cancel) | nested function | `SimpleEngine._stream_generate_specprefill._request_cancel() -> None` | Nested Function `SimpleEngine._stream_generate_specprefill._request_cancel` calls `cancel_requested.set`. | [#L1821-L1822](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1821-L1822) |
| [`SimpleEngine._stream_generate_specprefill._cancel_check`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._cancel_check) | nested function | `SimpleEngine._stream_generate_specprefill._cancel_check() -> None` | Nested Function `SimpleEngine._stream_generate_specprefill._cancel_check` calls `cancel_requested.is_set`, `_SpecPrefillCancelled`; can raise `_SpecPrefillCancelled`. | [#L1824-L1826](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1824-L1826) |
| [`SimpleEngine._stream_generate_specprefill._run_all`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_all) | nested function | `SimpleEngine._stream_generate_specprefill._run_all() -> not annotated` | Nested Function `SimpleEngine._stream_generate_specprefill._run_all` calls `_run_specprefill`, `logger.error`, `_run_normal`; has 2 explicit return paths. | [#L1828-L1835](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1828-L1835) |
| [`SimpleEngine._stream_generate_specprefill._run_specprefill`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_specprefill) | nested function | `SimpleEngine._stream_generate_specprefill._run_specprefill() -> not annotated` | Score tokens, sparse prefill, generate autoregressively. | [#L1837-L1939](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1837-L1939) |
| [`SimpleEngine._stream_generate_specprefill._run_normal`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_specprefill._run_normal) | nested function | `SimpleEngine._stream_generate_specprefill._run_normal() -> not annotated` | Fallback: normal generation without specprefill. | [#L1941-L1962](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L1941-L1962) |
| [`SimpleEngine._stream_generate_text`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text) | method | `async SimpleEngine._stream_generate_text(messages: list[dict[str, Any]], max_tokens: int, temperature: float, top_p: float, tools: list \| None = None, **kwargs) -> AsyncIterator[GenerationOutput]` | Text-only generation via mlx_lm TextModel. | [#L2002-L2734](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2002-L2734) |
| [`SimpleEngine._stream_generate_text.make_cache_with_snapshot`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text.make_cache_with_snapshot) | nested function | `SimpleEngine._stream_generate_text.make_cache_with_snapshot(text_model, system_kv_snapshot, _max_kv_size = self._max_kv_size) -> not annotated` | Nested Function `SimpleEngine._stream_generate_text.make_cache_with_snapshot` calls `make_prompt_cache`, `SimpleEngine._restore_prompt_cache`, `mx.array`; returns `(backbone_cache, prompt_to_send)`. | [#L2159-L2176](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2159-L2176) |
| [`SimpleEngine._stream_generate_text._emit_response`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_response) | nested function | `SimpleEngine._stream_generate_text._emit_response(resp: Any) -> None` | Nested Function `SimpleEngine._stream_generate_text._emit_response` calls `abort_event.is_set`, `loop.call_soon_threadsafe`; returns `None`. | [#L2272-L2275](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2272-L2275) |
| [`SimpleEngine._stream_generate_text._emit_done`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_done) | nested function | `SimpleEngine._stream_generate_text._emit_done() -> None` | Nested Function `SimpleEngine._stream_generate_text._emit_done` calls `loop.call_soon_threadsafe`. | [#L2277-L2278](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2277-L2278) |
| [`SimpleEngine._stream_generate_text._emit_error`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._emit_error) | nested function | `SimpleEngine._stream_generate_text._emit_error(exc: BaseException) -> None` | Nested Function `SimpleEngine._stream_generate_text._emit_error` calls `loop.call_soon_threadsafe`. | [#L2280-L2281](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2280-L2281) |
| [`SimpleEngine._stream_generate_text._seed_from_last_response`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._seed_from_last_response) | nested function | `SimpleEngine._stream_generate_text._seed_from_last_response(prompt_cache, last_resp) -> not annotated` | Nested Function `SimpleEngine._stream_generate_text._seed_from_last_response` calls `getattr`, `cache_module.trim_prompt_cache`, `mx.array`, `self._text_tokenizer.encode`; has 2 explicit return paths. | [#L2283-L2291](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2283-L2291) |
| [`SimpleEngine._stream_generate_text._resume_after_processor_retirement`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._resume_after_processor_retirement) | nested function | `SimpleEngine._stream_generate_text._resume_after_processor_retirement(model, prompt_cache, prompt, remaining_tokens: int) -> None` | Nested Function `SimpleEngine._stream_generate_text._resume_after_processor_retirement` calls `dict`, `hasattr`, `model.make_mtp_cache`, `mlx_stream_generate`. | [#L2293-L2320](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2293-L2320) |
| [`SimpleEngine._stream_generate_text._run_all`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_all) | nested function | `SimpleEngine._stream_generate_text._run_all() -> not annotated` | Nested Function `SimpleEngine._stream_generate_text._run_all` calls `_processors_can_retire`, `hasattr`, `logger.info`, `make_prompt_cache`; returns `None`. | [#L2323-L2485](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2323-L2485) |
| [`SimpleEngine._stream_generate_text._run_specprefill`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._run_specprefill) | nested function | `SimpleEngine._stream_generate_text._run_specprefill(model, bc, use_mtp) -> not annotated` | Score tokens, sparse prefill, then continue on the standard decode path. | [#L2487-L2664](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2487-L2664) |
| [`SimpleEngine._stream_generate_text._produce_responses`](#contract-vllm_mlx.engine.simple.SimpleEngine._stream_generate_text._produce_responses) | nested function | `async SimpleEngine._stream_generate_text._produce_responses() -> None` | Nested Function `SimpleEngine._stream_generate_text._produce_responses` calls `self._run_blocking_serialized`, `_emit_error`, `_emit_done`; awaits asynchronous work. | [#L2666-L2677](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2666-L2677) |
| [`SimpleEngine.get_stats`](#contract-vllm_mlx.engine.simple.SimpleEngine.get_stats) | method | `SimpleEngine.get_stats() -> dict[str, Any]` | Get engine statistics. | [#L2736-L2858](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2736-L2858) |
| [`SimpleEngine.get_cache_stats`](#contract-vllm_mlx.engine.simple.SimpleEngine.get_cache_stats) | method | `SimpleEngine.get_cache_stats() -> dict[str, Any] \| None` | Get cache statistics for the system-prompt KV LRU plus, when the model is multimodal, the MLLM's own cache stats. | [#L2860-L2878](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2860-L2878) |
| [`SimpleEngine.clear_runtime_caches`](#contract-vllm_mlx.engine.simple.SimpleEngine.clear_runtime_caches) | method | `SimpleEngine.clear_runtime_caches() -> dict[str, Any] \| None` | Clear engine-managed runtime caches. | [#L2880-L2912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/engine/simple.py#L2880-L2912) |