# `vllm_mlx.lifecycle`

Model lifecycle / residency management for vllm-mlx.

[View the complete module source at #L1-L493](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L1-L493).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.lifecycle
    options:
      members:
        - ResidentState
        - ModelSpec
        - ResidentModel
        - ResidencyManager
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidentState" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidentState</code> · class</summary>

```python
vllm_mlx.lifecycle.ResidentState()
```

Runtime residency state for a configured model.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Constructs: `vllm_mlx.lifecycle.ResidentState`

**Exceptions and behavior**

Class `ResidentState` derives from `str`, `Enum` and declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L17-L24](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L17-L24).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ModelSpec" markdown="1">
<summary><code>vllm_mlx.lifecycle.ModelSpec</code> · class</summary>

```python
vllm_mlx.lifecycle.ModelSpec(model_key: str, model_name: str, use_batching: bool = False, scheduler_config: Any | None = None, stream_interval: int = 1, max_tokens: int = 32768, force_mllm: bool = False, mtp: bool = False, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str | None = None)
```

Immutable engine construction inputs for a resident model.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required constructor field. |
| `model_name` | `str` | `yes` | `none` | Required constructor field. |
| `use_batching` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `scheduler_config` | `Any \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `stream_interval` | `int` | `no` | `1` | Optional constructor field; defaults to `1`. |
| `max_tokens` | `int` | `no` | `32768` | Optional constructor field; defaults to `32768`. |
| `force_mllm` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `mtp` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `prefill_step_size` | `int` | `no` | `2048` | Optional constructor field; defaults to `2048`. |
| `specprefill_enabled` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `specprefill_threshold` | `int` | `no` | `8192` | Optional constructor field; defaults to `8192`. |
| `specprefill_keep_pct` | `float` | `no` | `0.3` | Optional constructor field; defaults to `0.3`. |
| `specprefill_backbone_pct` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. |
| `specprefill_draft_model` | `str \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |

**Returns**

- Constructs: `vllm_mlx.lifecycle.ModelSpec`

**Exceptions and behavior**

Class `ModelSpec` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L28-L44](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L28-L44).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidentModel" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidentModel</code> · class</summary>

```python
vllm_mlx.lifecycle.ResidentModel(spec: ModelSpec, state: ResidentState = ResidentState.UNLOADED, engine: BaseEngine | None = None, active_requests: int = 0, last_used_at: float | None = None, loaded_at: float | None = None, last_error: str | None = None, estimated_memory_bytes: int | None = None, _load_waiters: int = field(default=0, repr=False), _load_waiter_task: asyncio.Task[BaseEngine] | None = field(default=None, repr=False), _prepare_task: asyncio.Task[None] | None = field(default=None, repr=False), _abandoned_loading_task: asyncio.Task[BaseEngine] | None = field(default=None, repr=False), _loading_task: asyncio.Task[BaseEngine] | None = field(default=None, repr=False), _unloading_task: asyncio.Task[bool] | None = field(default=None, repr=False))
```

Runtime state for a single resident model.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required constructor field. |
| `state` | `ResidentState` | `no` | `ResidentState.UNLOADED` | Optional constructor field; defaults to `ResidentState.UNLOADED`. |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `active_requests` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `last_used_at` | `float \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `loaded_at` | `float \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `last_error` | `str \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `estimated_memory_bytes` | `int \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `_load_waiters` | `int` | `no` | `field(default=0, repr=False)` | Optional constructor field; defaults to `field(default=0, repr=False)`. |
| `_load_waiter_task` | `asyncio.Task[BaseEngine] \| None` | `no` | `field(default=None, repr=False)` | Optional constructor field; defaults to `field(default=None, repr=False)`. |
| `_prepare_task` | `asyncio.Task[None] \| None` | `no` | `field(default=None, repr=False)` | Optional constructor field; defaults to `field(default=None, repr=False)`. |
| `_abandoned_loading_task` | `asyncio.Task[BaseEngine] \| None` | `no` | `field(default=None, repr=False)` | Optional constructor field; defaults to `field(default=None, repr=False)`. |
| `_loading_task` | `asyncio.Task[BaseEngine] \| None` | `no` | `field(default=None, repr=False)` | Optional constructor field; defaults to `field(default=None, repr=False)`. |
| `_unloading_task` | `asyncio.Task[bool] \| None` | `no` | `field(default=None, repr=False)` | Optional constructor field; defaults to `field(default=None, repr=False)`. |

**Returns**

- Constructs: `vllm_mlx.lifecycle.ResidentModel`

**Exceptions and behavior**

Class `ResidentModel` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L48-L66](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L48-L66).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager</code> · class</summary>

```python
vllm_mlx.lifecycle.ResidencyManager(engine_factory: Callable[[ModelSpec], Awaitable[BaseEngine]], *, on_engine_loaded: Callable[[ModelSpec, BaseEngine], Awaitable[None] | None] | None = None, on_engine_unloading: Callable[[ModelSpec, BaseEngine], Awaitable[None] | None] | None = None, time_fn: Callable[[], float] | None = None, auto_unload_idle_seconds: float = 0)
```

Single-flight lifecycle manager for resident models.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine_factory` | `Callable[[ModelSpec], Awaitable[BaseEngine]]` | `yes` | `none` | Required positional or keyword input. |
| `on_engine_loaded` | `Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `on_engine_unloading` | `Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `time_fn` | `Callable[[], float] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `auto_unload_idle_seconds` | `float` | `no` | `0` | Optional keyword-only input; defaults to `0`. |

**Returns**

- Constructs: `vllm_mlx.lifecycle.ResidencyManager`

**Exceptions and behavior**

Class `ResidencyManager` declares 16 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L69-L493](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L69-L493).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.__init__" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.__init__</code> · method</summary>

```python
vllm_mlx.lifecycle.ResidencyManager.__init__(engine_factory: Callable[[ModelSpec], Awaitable[BaseEngine]], *, on_engine_loaded: Callable[[ModelSpec, BaseEngine], Awaitable[None] | None] | None = None, on_engine_unloading: Callable[[ModelSpec, BaseEngine], Awaitable[None] | None] | None = None, time_fn: Callable[[], float] | None = None, auto_unload_idle_seconds: float = 0) -> None
```

Method `ResidencyManager.__init__` updates `self._engine_factory`, `self._on_engine_loaded`, `self._on_engine_unloading`, `self._time_fn`; calls `__import__`, `asyncio.Lock`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine_factory` | `Callable[[ModelSpec], Awaitable[BaseEngine]]` | `yes` | `none` | Required positional or keyword input. |
| `on_engine_loaded` | `Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `on_engine_unloading` | `Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `time_fn` | `Callable[[], float] \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `auto_unload_idle_seconds` | `float` | `no` | `0` | Optional keyword-only input; defaults to `0`. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `ResidencyManager.__init__` updates `self._engine_factory`, `self._on_engine_loaded`, `self._on_engine_unloading`, `self._time_fn`; calls `__import__`, `asyncio.Lock`.
No direct `raise` statement appears in this definition.

[View source #L72-L91](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L72-L91).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.register_model" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.register_model</code> · method</summary>

```python
vllm_mlx.lifecycle.ResidencyManager.register_model(spec: ModelSpec) -> str
```

Register a model spec, or replace a dormant resident entry.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `spec.model_key`

**Exceptions and behavior**

Method `ResidencyManager.register_model` calls `self._residents.get`, `RuntimeError`, `ResidentModel`; can raise `RuntimeError`; returns `spec.model_key`.
Directly raised exceptions: `RuntimeError`.

[View source #L93-L111](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L93-L111).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.get_engine" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.get_engine</code> · method</summary>

```python
vllm_mlx.lifecycle.ResidencyManager.get_engine(model_key: str) -> BaseEngine | None
```

Get the currently loaded engine, if any.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine | None`
- Direct return expressions: `self._resident(model_key).engine`

**Exceptions and behavior**

Method `ResidencyManager.get_engine` calls `self._resident`; returns `self._resident(model_key).engine`.
No direct `raise` statement appears in this definition.

[View source #L113-L115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L113-L115).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.get_status" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.get_status</code> · method</summary>

```python
vllm_mlx.lifecycle.ResidencyManager.get_status(model_key: str) -> dict[str, Any]
```

Return a serializable snapshot of resident state.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict[str, Any]`
- Direct return expressions: `{'model_key': resident.spec.model_key, 'model_name': resident.spec.model_name, 'state': resident.state.value, 'active_r…`

**Exceptions and behavior**

Method `ResidencyManager.get_status` calls `self._resident`; returns `{'model_key': resident.spec.model_key, 'model_name': resident.spec.model_name, 'state': resident.state.value, 'active_r…`.
No direct `raise` statement appears in this definition.

[View source #L117-L130](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L117-L130).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.ensure_loaded" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.ensure_loaded</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager.ensure_loaded(model_key: str) -> BaseEngine
```

Load and start a resident engine if needed.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `resident.engine`; `await asyncio.shield(task)`

**Exceptions and behavior**

Method `ResidencyManager.ensure_loaded` calls `self._resident`, `asyncio.create_task`, `self._load_engine`, `asyncio.shield`; awaits asynchronous work; can raise `RuntimeError`; has 2 explicit return paths.
Directly raised exceptions: `RuntimeError`.

[View source #L132-L184](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L132-L184).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.acquire" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.acquire</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager.acquire(model_key: str, *, count_activity: bool = True) -> BaseEngine
```

Acquire a resident engine for request processing.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `engine`

**Exceptions and behavior**

Method `ResidencyManager.acquire` calls `self.ensure_loaded`, `self._resident`, `self._time_fn`; awaits asynchronous work; returns `engine`.
No direct `raise` statement appears in this definition.

[View source #L186-L206](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L186-L206).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.release" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.release</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager.release(model_key: str, *, count_activity: bool = True) -> None
```

Release a previously acquired resident engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `ResidencyManager.release` calls `self._resident`, `self._time_fn`.
No direct `raise` statement appears in this definition.

[View source #L208-L215](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L208-L215).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.unload_if_idle" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.unload_if_idle</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager.unload_if_idle(model_key: str) -> bool
```

Unload a resident engine if it has been idle past the threshold.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `await asyncio.shield(unloading_task)`

**Exceptions and behavior**

Method `ResidencyManager.unload_if_idle` calls `self._resident`, `self._time_fn`, `asyncio.create_task`, `self._unload_engine`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L217-L253](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L217-L253).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager.shutdown" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager.shutdown</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager.shutdown() -> None
```

Stop all loaded residents.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `ResidencyManager.shutdown` calls `list`, `self._residents.keys`, `self._resident`, `resident._loading_task.cancel`; awaits asynchronous work; can raise `RuntimeError`.
Directly raised exceptions: `RuntimeError`.

[View source #L255-L312](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L255-L312).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._load_engine" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._load_engine</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._load_engine(resident: ResidentModel) -> BaseEngine
```

Create and start a resident engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resident` | `ResidentModel` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `engine`

**Exceptions and behavior**

Method `ResidencyManager._load_engine` calls `self._engine_factory`, `self._prepare_engine_start`, `engine.start`, `self._run_hook`; awaits asynchronous work; can raise `asyncio.CancelledError`; returns `engine`.
Directly raised exceptions: `asyncio.CancelledError`.

[View source #L314-L354](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L314-L354).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._unload_engine" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._unload_engine</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._unload_engine(resident: ResidentModel) -> bool
```

Stop and drop a resident engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resident` | `ResidentModel` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `True`

**Exceptions and behavior**

Method `ResidencyManager._unload_engine` calls `self._run_hook`, `engine.stop`, `str`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L356-L388](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L356-L388).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._resident" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._resident</code> · method</summary>

```python
vllm_mlx.lifecycle.ResidencyManager._resident(model_key: str) -> ResidentModel
```

Method `ResidencyManager._resident` calls `KeyError`; can raise `KeyError`; returns `self._residents[model_key]`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `ResidentModel`
- Direct return expressions: `self._residents[model_key]`

**Exceptions and behavior**

Method `ResidencyManager._resident` calls `KeyError`; can raise `KeyError`; returns `self._residents[model_key]`.
Directly raised exceptions: `KeyError`.

[View source #L390-L394](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L390-L394).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._run_hook" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._run_hook</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._run_hook(hook: Callable[[ModelSpec, BaseEngine], Awaitable[None] | None] | None, spec: ModelSpec, engine: BaseEngine) -> None
```

Method `ResidencyManager._run_hook` calls `hook`, `inspect.isawaitable`; awaits asynchronous work; returns `None`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `hook` | `Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None` | `yes` | `none` | Required positional or keyword input. |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `ResidencyManager._run_hook` calls `hook`, `inspect.isawaitable`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L396-L407](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L396-L407).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._prepare_engine_start" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._prepare_engine_start</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._prepare_engine_start(resident: ResidentModel, engine: BaseEngine) -> None
```

Run blocking startup work away from the serving event loop.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resident` | `ResidentModel` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `ResidencyManager._prepare_engine_start` calls `getattr`, `callable`, `uses_default_prepare`, `prepare_for_start`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L409-L445](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L409-L445).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._cleanup_cancelled_load" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._cleanup_cancelled_load</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._cleanup_cancelled_load(resident: ResidentModel, engine: BaseEngine | None) -> None
```

Stop a partially loaded engine and unwind resident state.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `resident` | `ResidentModel` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `ResidencyManager._cleanup_cancelled_load` calls `suspend_cancellation`, `suppress`, `engine.stop`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L447-L465](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L447-L465).

</details>

<details class="api-contract" id="contract-vllm_mlx.lifecycle.ResidencyManager._release_load_waiter" markdown="1">
<summary><code>vllm_mlx.lifecycle.ResidencyManager._release_load_waiter</code> · method</summary>

```python
async vllm_mlx.lifecycle.ResidencyManager._release_load_waiter(model_key: str, task: asyncio.Task[BaseEngine]) -> None
```

Drop one waiter from a shared load, canceling abandoned solo loads.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_key` | `str` | `yes` | `none` | Required positional or keyword input. |
| `task` | `asyncio.Task[BaseEngine]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `ResidencyManager._release_load_waiter` calls `self._resident`, `task.done`, `suspend_cancellation`, `task_to_cancel.cancel`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L467-L493](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L467-L493).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`ResidentState`](#contract-vllm_mlx.lifecycle.ResidentState) | class | `ResidentState()` | Runtime residency state for a configured model. | [#L17-L24](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L17-L24) |
| [`ModelSpec`](#contract-vllm_mlx.lifecycle.ModelSpec) | class | `ModelSpec(model_key: str, model_name: str, use_batching: bool = False, scheduler_config: Any \| None = None, stream_interval: int = 1, max_tokens: int = 32768, force_mllm: bool = False, mtp: bool = False, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str \| None = None)` | Immutable engine construction inputs for a resident model. | [#L28-L44](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L28-L44) |
| [`ResidentModel`](#contract-vllm_mlx.lifecycle.ResidentModel) | class | `ResidentModel(spec: ModelSpec, state: ResidentState = ResidentState.UNLOADED, engine: BaseEngine \| None = None, active_requests: int = 0, last_used_at: float \| None = None, loaded_at: float \| None = None, last_error: str \| None = None, estimated_memory_bytes: int \| None = None, _load_waiters: int = field(default=0, repr=False), _load_waiter_task: asyncio.Task[BaseEngine] \| None = field(default=None, repr=False), _prepare_task: asyncio.Task[None] \| None = field(default=None, repr=False), _abandoned_loading_task: asyncio.Task[BaseEngine] \| None = field(default=None, repr=False), _loading_task: asyncio.Task[BaseEngine] \| None = field(default=None, repr=False), _unloading_task: asyncio.Task[bool] \| None = field(default=None, repr=False))` | Runtime state for a single resident model. | [#L48-L66](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L48-L66) |
| [`ResidencyManager`](#contract-vllm_mlx.lifecycle.ResidencyManager) | class | `ResidencyManager(engine_factory: Callable[[ModelSpec], Awaitable[BaseEngine]], *, on_engine_loaded: Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None = None, on_engine_unloading: Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None = None, time_fn: Callable[[], float] \| None = None, auto_unload_idle_seconds: float = 0)` | Single-flight lifecycle manager for resident models. | [#L69-L493](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L69-L493) |
| [`ResidencyManager.__init__`](#contract-vllm_mlx.lifecycle.ResidencyManager.__init__) | method | `ResidencyManager.__init__(engine_factory: Callable[[ModelSpec], Awaitable[BaseEngine]], *, on_engine_loaded: Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None = None, on_engine_unloading: Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None = None, time_fn: Callable[[], float] \| None = None, auto_unload_idle_seconds: float = 0) -> None` | Method `ResidencyManager.__init__` updates `self._engine_factory`, `self._on_engine_loaded`, `self._on_engine_unloading`, `self._time_fn`; calls `__import__`, `asyncio.Lock`. | [#L72-L91](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L72-L91) |
| [`ResidencyManager.register_model`](#contract-vllm_mlx.lifecycle.ResidencyManager.register_model) | method | `ResidencyManager.register_model(spec: ModelSpec) -> str` | Register a model spec, or replace a dormant resident entry. | [#L93-L111](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L93-L111) |
| [`ResidencyManager.get_engine`](#contract-vllm_mlx.lifecycle.ResidencyManager.get_engine) | method | `ResidencyManager.get_engine(model_key: str) -> BaseEngine \| None` | Get the currently loaded engine, if any. | [#L113-L115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L113-L115) |
| [`ResidencyManager.get_status`](#contract-vllm_mlx.lifecycle.ResidencyManager.get_status) | method | `ResidencyManager.get_status(model_key: str) -> dict[str, Any]` | Return a serializable snapshot of resident state. | [#L117-L130](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L117-L130) |
| [`ResidencyManager.ensure_loaded`](#contract-vllm_mlx.lifecycle.ResidencyManager.ensure_loaded) | method | `async ResidencyManager.ensure_loaded(model_key: str) -> BaseEngine` | Load and start a resident engine if needed. | [#L132-L184](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L132-L184) |
| [`ResidencyManager.acquire`](#contract-vllm_mlx.lifecycle.ResidencyManager.acquire) | method | `async ResidencyManager.acquire(model_key: str, *, count_activity: bool = True) -> BaseEngine` | Acquire a resident engine for request processing. | [#L186-L206](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L186-L206) |
| [`ResidencyManager.release`](#contract-vllm_mlx.lifecycle.ResidencyManager.release) | method | `async ResidencyManager.release(model_key: str, *, count_activity: bool = True) -> None` | Release a previously acquired resident engine. | [#L208-L215](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L208-L215) |
| [`ResidencyManager.unload_if_idle`](#contract-vllm_mlx.lifecycle.ResidencyManager.unload_if_idle) | method | `async ResidencyManager.unload_if_idle(model_key: str) -> bool` | Unload a resident engine if it has been idle past the threshold. | [#L217-L253](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L217-L253) |
| [`ResidencyManager.shutdown`](#contract-vllm_mlx.lifecycle.ResidencyManager.shutdown) | method | `async ResidencyManager.shutdown() -> None` | Stop all loaded residents. | [#L255-L312](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L255-L312) |
| [`ResidencyManager._load_engine`](#contract-vllm_mlx.lifecycle.ResidencyManager._load_engine) | method | `async ResidencyManager._load_engine(resident: ResidentModel) -> BaseEngine` | Create and start a resident engine. | [#L314-L354](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L314-L354) |
| [`ResidencyManager._unload_engine`](#contract-vllm_mlx.lifecycle.ResidencyManager._unload_engine) | method | `async ResidencyManager._unload_engine(resident: ResidentModel) -> bool` | Stop and drop a resident engine. | [#L356-L388](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L356-L388) |
| [`ResidencyManager._resident`](#contract-vllm_mlx.lifecycle.ResidencyManager._resident) | method | `ResidencyManager._resident(model_key: str) -> ResidentModel` | Method `ResidencyManager._resident` calls `KeyError`; can raise `KeyError`; returns `self._residents[model_key]`. | [#L390-L394](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L390-L394) |
| [`ResidencyManager._run_hook`](#contract-vllm_mlx.lifecycle.ResidencyManager._run_hook) | method | `async ResidencyManager._run_hook(hook: Callable[[ModelSpec, BaseEngine], Awaitable[None] \| None] \| None, spec: ModelSpec, engine: BaseEngine) -> None` | Method `ResidencyManager._run_hook` calls `hook`, `inspect.isawaitable`; awaits asynchronous work; returns `None`. | [#L396-L407](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L396-L407) |
| [`ResidencyManager._prepare_engine_start`](#contract-vllm_mlx.lifecycle.ResidencyManager._prepare_engine_start) | method | `async ResidencyManager._prepare_engine_start(resident: ResidentModel, engine: BaseEngine) -> None` | Run blocking startup work away from the serving event loop. | [#L409-L445](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L409-L445) |
| [`ResidencyManager._cleanup_cancelled_load`](#contract-vllm_mlx.lifecycle.ResidencyManager._cleanup_cancelled_load) | method | `async ResidencyManager._cleanup_cancelled_load(resident: ResidentModel, engine: BaseEngine \| None) -> None` | Stop a partially loaded engine and unwind resident state. | [#L447-L465](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L447-L465) |
| [`ResidencyManager._release_load_waiter`](#contract-vllm_mlx.lifecycle.ResidencyManager._release_load_waiter) | method | `async ResidencyManager._release_load_waiter(model_key: str, task: asyncio.Task[BaseEngine]) -> None` | Drop one waiter from a shared load, canceling abandoned solo loads. | [#L467-L493](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/lifecycle.py#L467-L493) |