# `vllm_mlx.mllm_cache`

MLLM (Multimodal Language Model) Prefix Cache Manager.

[View the complete module source at #L1-L459](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L1-L459).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.mllm_cache
    options:
      members:
        - logger
        - MLLMCacheStats
        - MLLMPrefixCacheEntry
        - compute_image_hash
        - compute_images_hash
        - MLLMPrefixCacheManager
        - MLLMCacheManager
        - VLMCacheStats
        - VLMPrefixCacheEntry
        - VLMCacheEntry
        - VLMPrefixCacheManager
        - VLMCacheManager
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMCacheStats" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMCacheStats</code> · class</summary>

```python
vllm_mlx.mllm_cache.MLLMCacheStats(hits: int = 0, misses: int = 0, partial_hits: int = 0, tokens_saved: int = 0, image_cache_hits: int = 0, vision_encoder_skips: int = 0, total_queries: int = 0, evictions: int = 0)
```

Statistics for MLLM cache performance.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `hits` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `misses` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `partial_hits` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `tokens_saved` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `image_cache_hits` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `vision_encoder_skips` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `total_queries` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `evictions` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |

**Returns**

- Constructs: `vllm_mlx.mllm_cache.MLLMCacheStats`

**Exceptions and behavior**

Class `MLLMCacheStats` declares 2 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L34-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L34-L65).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMCacheStats.hit_rate" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMCacheStats.hit_rate</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMCacheStats.hit_rate() -> float
```

Calculate cache hit rate.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `float`
- Direct return expressions: `0.0`; `self.hits / self.total_queries`

**Exceptions and behavior**

Method `MLLMCacheStats.hit_rate` has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L47-L51](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L47-L51).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMCacheStats.to_dict" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMCacheStats.to_dict</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMCacheStats.to_dict() -> dict
```

Convert stats to dictionary.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict`
- Direct return expressions: `{'hits': self.hits, 'misses': self.misses, 'partial_hits': self.partial_hits, 'hit_rate': self.hit_rate, 'tokens_saved'…`

**Exceptions and behavior**

Method `MLLMCacheStats.to_dict` returns `{'hits': self.hits, 'misses': self.misses, 'partial_hits': self.partial_hits, 'hit_rate': self.hit_rate, 'tokens_saved'…`.
No direct `raise` statement appears in this definition.

[View source #L53-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L53-L65).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheEntry</code> · class</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheEntry(image_hash: str, prompt_hash: str, vision_embeddings: Any = None, kv_cache: list[Any] = field(default_factory=list), token_ids: list[int] = field(default_factory=list), num_image_tokens: int = 0, num_text_tokens: int = 0, prompt_tokens: int = 0, created_at: float = field(default_factory=time.time), hit_count: int = 0, model_name: str = '')
```

Enhanced cache entry storing vision embeddings, KV cache, and token IDs.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `image_hash` | `str` | `yes` | `none` | Required constructor field. |
| `prompt_hash` | `str` | `yes` | `none` | Required constructor field. |
| `vision_embeddings` | `Any` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `kv_cache` | `list[Any]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `token_ids` | `list[int]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `num_image_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `num_text_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `prompt_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `created_at` | `float` | `no` | `field(default_factory=time.time)` | Optional constructor field; defaults to `field(default_factory=time.time)`. |
| `hit_count` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `model_name` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. |

**Returns**

- Constructs: `vllm_mlx.mllm_cache.MLLMPrefixCacheEntry`

**Exceptions and behavior**

Class `MLLMPrefixCacheEntry` declares 3 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L69-L133](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L69-L133).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.total_tokens" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.total_tokens</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.total_tokens() -> int
```

Return the number of token IDs represented by this cache entry.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int`
- Direct return expressions: `len(self.token_ids)`

**Exceptions and behavior**

Method `MLLMPrefixCacheEntry.total_tokens` calls `len`; returns `len(self.token_ids)`.
No direct `raise` statement appears in this definition.

[View source #L99-L102](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L99-L102).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.memory_size" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.memory_size</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.memory_size() -> int
```

Estimate memory usage in bytes.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int`
- Direct return expressions: `size`

**Exceptions and behavior**

Method `MLLMPrefixCacheEntry.memory_size` calls `hasattr`; returns `size`.
No direct `raise` statement appears in this definition.

[View source #L105-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L105-L119).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.get_prefix_match_length" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.get_prefix_match_length</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.get_prefix_match_length(new_token_ids: list[int]) -> int
```

Find how many tokens match between cached prefix and new input.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `new_token_ids` | `list[int]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `int`
- Direct return expressions: `match_length`

**Exceptions and behavior**

Method `MLLMPrefixCacheEntry.get_prefix_match_length` calls `enumerate`, `zip`; returns `match_length`.
No direct `raise` statement appears in this definition.

[View source #L121-L133](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L121-L133).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.compute_image_hash" markdown="1">
<summary><code>vllm_mlx.mllm_cache.compute_image_hash</code> · function</summary>

```python
vllm_mlx.mllm_cache.compute_image_hash(image_path: str) -> str
```

Compute hash of image content for cache key.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `image_path` | `str` | `yes` | `none` | Path to image file |

**Returns**

- Type: `str`
- Direct return expressions: `hashlib.sha256(content).hexdigest()[:16]`; `hashlib.sha256(image_path.encode()).hexdigest()[:16]`; `hashlib.sha256(str(image_path).encode()).hexdigest()[:16]`

**Exceptions and behavior**

Function `compute_image_hash` calls `Path`, `path.exists`, `path.read_bytes`, `hashlib.sha256(content).hexdigest`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L136-L161](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L136-L161).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.compute_images_hash" markdown="1">
<summary><code>vllm_mlx.mllm_cache.compute_images_hash</code> · function</summary>

```python
vllm_mlx.mllm_cache.compute_images_hash(images: list[str]) -> str
```

Compute combined hash for multiple images.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | List of image paths/URLs |

**Returns**

- Type: `str`
- Direct return expressions: `'no_images'`; `hashlib.sha256(combined.encode()).hexdigest()[:16]`

**Exceptions and behavior**

Function `compute_images_hash` calls `compute_image_hash`, `'_'.join`, `sorted`, `hashlib.sha256(combined.encode()).hexdigest`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L164-L179](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L164-L179).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager</code> · class</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager(max_entries: int = 50, max_memory_mb: int = 2048)
```

LRU Cache manager for MLLM prefix states with vision embedding caching.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `max_entries` | `int` | `no` | `50` | Maximum number of cache entries (default: 50) |
| `max_memory_mb` | `int` | `no` | `2048` | Maximum memory in MB (default: 2048) |

**Returns**

- Constructs: `vllm_mlx.mllm_cache.MLLMPrefixCacheManager`

**Exceptions and behavior**

Class `MLLMPrefixCacheManager` declares 14 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L182-L448](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L182-L448).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__init__" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__init__</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__init__(max_entries: int = 50, max_memory_mb: int = 2048) -> not annotated
```

Initialize MLLM prefix cache manager.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `max_entries` | `int` | `no` | `50` | Maximum number of cache entries (default: 50) |
| `max_memory_mb` | `int` | `no` | `2048` | Maximum memory in MB (default: 2048) |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.__init__` updates `self.max_size`, `self.max_memory`, `self._cache`, `self._current_memory`; calls `OrderedDict`, `MLLMCacheStats`.
No direct `raise` statement appears in this definition.

[View source #L211-L227](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L211-L227).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_cache_key" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_cache_key</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_cache_key(images: list[str], prompt: str) -> str
```

Create cache key from images and prompt.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `f'{image_hash}_{prompt_hash}'`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager._make_cache_key` calls `compute_images_hash`, `hashlib.sha256(prompt.encode()).hexdigest`, `hashlib.sha256`, `prompt.encode`; returns `f'{image_hash}_{prompt_hash}'`.
No direct `raise` statement appears in this definition.

[View source #L229-L233](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L229-L233).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_image_only_key" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_image_only_key</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_image_only_key(images: list[str]) -> str
```

Create cache key for image-only lookup (vision embedding reuse).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `compute_images_hash(images)`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager._make_image_only_key` calls `compute_images_hash`; returns `compute_images_hash(images)`.
No direct `raise` statement appears in this definition.

[View source #L235-L237](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L235-L237).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_memory" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_memory</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_memory(required_size: int) -> None
```

Evict entries until we have enough memory.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `required_size` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager._evict_by_memory` updates `self._current_memory`, `self.stats.evictions`; calls `next`, `iter`, `self._cache.pop`, `logger.debug`.
No direct `raise` statement appears in this definition.

[View source #L239-L246](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L239-L246).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_count" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_count</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_count() -> None
```

Evict entries until we're under max_size.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager._evict_by_count` updates `self._current_memory`, `self.stats.evictions`; calls `len`, `next`, `iter`, `self._cache.pop`.
No direct `raise` statement appears in this definition.

[View source #L248-L255](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L248-L255).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch(images: list[str], prompt: str, token_ids: list[int] | None = None) -> tuple[MLLMPrefixCacheEntry | None, int]
```

Fetch cached prefix state with prefix matching.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | List of image paths |
| `prompt` | `str` | `yes` | `none` | Text prompt |
| `token_ids` | `list[int] \| None` | `no` | `None` | Optional token IDs for prefix matching |

**Returns**

- Type: `tuple[MLLMPrefixCacheEntry | None, int]`
- Direct return expressions: `(entry, match_length)`; `(entry, 0)`; `(None, 0)`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.fetch` updates `self.stats.total_queries`, `self.stats.hits`, `self.stats.image_cache_hits`, `self.stats.vision_encoder_skips`; calls `self._make_cache_key`, `self._cache.pop`, `entry.get_prefix_match_length`, `logger.debug`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L257-L329](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L257-L329).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch_cache" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch_cache</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch_cache(images: list[str], prompt: str) -> tuple[list[Any] | None, bool]
```

Legacy API: Fetch cached KV state for image+prompt combination.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[list[Any] | None, bool]`
- Direct return expressions: `(entry.kv_cache, True)`; `(None, False)`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.fetch_cache` calls `self.fetch`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L331-L345](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L331-L345).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store(images: list[str], prompt: str, vision_embeddings: Any, kv_cache: list[Any], token_ids: list[int], num_image_tokens: int = 0, model_name: str = '') -> None
```

Store prefix state in cache.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | List of image paths |
| `prompt` | `str` | `yes` | `none` | Text prompt |
| `vision_embeddings` | `Any` | `yes` | `none` | Output of vision encoder (can be None for text-only) |
| `kv_cache` | `list[Any]` | `yes` | `none` | Language model KV cache states |
| `token_ids` | `list[int]` | `yes` | `none` | Full token sequence |
| `num_image_tokens` | `int` | `no` | `0` | Number of image tokens (e.g., 256 for Gemma 3) |
| `model_name` | `str` | `no` | `''` | Model name for validation |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.store` updates `self._current_memory`; calls `self._make_cache_key`, `MLLMPrefixCacheEntry`, `compute_images_hash`, `hashlib.sha256(prompt.encode()).hexdigest`.
No direct `raise` statement appears in this definition.

[View source #L347-L396](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L347-L396).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store_cache" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store_cache</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store_cache(images: list[str], prompt: str, cache: list[Any] | None, num_tokens: int = 0) -> None
```

Legacy API: Store KV cache for future reuse.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `images` | `list[str]` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |
| `cache` | `list[Any] \| None` | `yes` | `none` | Required positional or keyword input. |
| `num_tokens` | `int` | `no` | `0` | Optional positional or keyword input; defaults to `0`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.store_cache` calls `isinstance`, `len`, `self.store`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L398-L421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L398-L421).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.get_stats" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.get_stats</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.get_stats() -> dict[str, Any]
```

Get cache statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict[str, Any]`
- Direct return expressions: `stats`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.get_stats` calls `self.stats.to_dict`, `len`; returns `stats`.
No direct `raise` statement appears in this definition.

[View source #L423-L430](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L423-L430).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.reset_stats" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.reset_stats</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.reset_stats() -> None
```

Reset statistics counters.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.reset_stats` updates `self.stats`; calls `MLLMCacheStats`.
No direct `raise` statement appears in this definition.

[View source #L432-L434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L432-L434).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.clear" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.clear</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.clear() -> None
```

Clear all cached entries and reset stats.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.clear` updates `self._current_memory`; calls `self._cache.clear`, `self.reset_stats`.
No direct `raise` statement appears in this definition.

[View source #L436-L440](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L436-L440).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__len__" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__len__</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__len__() -> int
```

Return number of cached entries.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int`
- Direct return expressions: `len(self._cache)`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.__len__` calls `len`; returns `len(self._cache)`.
No direct `raise` statement appears in this definition.

[View source #L442-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L442-L444).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__repr__" markdown="1">
<summary><code>vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__repr__</code> · method</summary>

```python
vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__repr__() -> str
```

Method `MLLMPrefixCacheManager.__repr__` calls `len`; returns `f'<MLLMPrefixCacheManager entries={len(self)} memory={mem_mb:.1f}MB>'`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`
- Direct return expressions: `f'<MLLMPrefixCacheManager entries={len(self)} memory={mem_mb:.1f}MB>'`

**Exceptions and behavior**

Method `MLLMPrefixCacheManager.__repr__` calls `len`; returns `f'<MLLMPrefixCacheManager entries={len(self)} memory={mem_mb:.1f}MB>'`.
No direct `raise` statement appears in this definition.

[View source #L446-L448](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L446-L448).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`MLLMCacheStats`](#contract-vllm_mlx.mllm_cache.MLLMCacheStats) | class | `MLLMCacheStats(hits: int = 0, misses: int = 0, partial_hits: int = 0, tokens_saved: int = 0, image_cache_hits: int = 0, vision_encoder_skips: int = 0, total_queries: int = 0, evictions: int = 0)` | Statistics for MLLM cache performance. | [#L34-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L34-L65) |
| [`MLLMCacheStats.hit_rate`](#contract-vllm_mlx.mllm_cache.MLLMCacheStats.hit_rate) | method | `MLLMCacheStats.hit_rate() -> float` | Calculate cache hit rate. | [#L47-L51](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L47-L51) |
| [`MLLMCacheStats.to_dict`](#contract-vllm_mlx.mllm_cache.MLLMCacheStats.to_dict) | method | `MLLMCacheStats.to_dict() -> dict` | Convert stats to dictionary. | [#L53-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L53-L65) |
| [`MLLMPrefixCacheEntry`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry) | class | `MLLMPrefixCacheEntry(image_hash: str, prompt_hash: str, vision_embeddings: Any = None, kv_cache: list[Any] = field(default_factory=list), token_ids: list[int] = field(default_factory=list), num_image_tokens: int = 0, num_text_tokens: int = 0, prompt_tokens: int = 0, created_at: float = field(default_factory=time.time), hit_count: int = 0, model_name: str = '')` | Enhanced cache entry storing vision embeddings, KV cache, and token IDs. | [#L69-L133](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L69-L133) |
| [`MLLMPrefixCacheEntry.total_tokens`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.total_tokens) | method | `MLLMPrefixCacheEntry.total_tokens() -> int` | Return the number of token IDs represented by this cache entry. | [#L99-L102](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L99-L102) |
| [`MLLMPrefixCacheEntry.memory_size`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.memory_size) | method | `MLLMPrefixCacheEntry.memory_size() -> int` | Estimate memory usage in bytes. | [#L105-L119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L105-L119) |
| [`MLLMPrefixCacheEntry.get_prefix_match_length`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheEntry.get_prefix_match_length) | method | `MLLMPrefixCacheEntry.get_prefix_match_length(new_token_ids: list[int]) -> int` | Find how many tokens match between cached prefix and new input. | [#L121-L133](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L121-L133) |
| [`compute_image_hash`](#contract-vllm_mlx.mllm_cache.compute_image_hash) | function | `compute_image_hash(image_path: str) -> str` | Compute hash of image content for cache key. | [#L136-L161](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L136-L161) |
| [`compute_images_hash`](#contract-vllm_mlx.mllm_cache.compute_images_hash) | function | `compute_images_hash(images: list[str]) -> str` | Compute combined hash for multiple images. | [#L164-L179](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L164-L179) |
| [`MLLMPrefixCacheManager`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager) | class | `MLLMPrefixCacheManager(max_entries: int = 50, max_memory_mb: int = 2048)` | LRU Cache manager for MLLM prefix states with vision embedding caching. | [#L182-L448](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L182-L448) |
| [`MLLMPrefixCacheManager.__init__`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__init__) | method | `MLLMPrefixCacheManager.__init__(max_entries: int = 50, max_memory_mb: int = 2048) -> not annotated` | Initialize MLLM prefix cache manager. | [#L211-L227](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L211-L227) |
| [`MLLMPrefixCacheManager._make_cache_key`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_cache_key) | method | `MLLMPrefixCacheManager._make_cache_key(images: list[str], prompt: str) -> str` | Create cache key from images and prompt. | [#L229-L233](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L229-L233) |
| [`MLLMPrefixCacheManager._make_image_only_key`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._make_image_only_key) | method | `MLLMPrefixCacheManager._make_image_only_key(images: list[str]) -> str` | Create cache key for image-only lookup (vision embedding reuse). | [#L235-L237](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L235-L237) |
| [`MLLMPrefixCacheManager._evict_by_memory`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_memory) | method | `MLLMPrefixCacheManager._evict_by_memory(required_size: int) -> None` | Evict entries until we have enough memory. | [#L239-L246](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L239-L246) |
| [`MLLMPrefixCacheManager._evict_by_count`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager._evict_by_count) | method | `MLLMPrefixCacheManager._evict_by_count() -> None` | Evict entries until we're under max_size. | [#L248-L255](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L248-L255) |
| [`MLLMPrefixCacheManager.fetch`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch) | method | `MLLMPrefixCacheManager.fetch(images: list[str], prompt: str, token_ids: list[int] \| None = None) -> tuple[MLLMPrefixCacheEntry \| None, int]` | Fetch cached prefix state with prefix matching. | [#L257-L329](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L257-L329) |
| [`MLLMPrefixCacheManager.fetch_cache`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.fetch_cache) | method | `MLLMPrefixCacheManager.fetch_cache(images: list[str], prompt: str) -> tuple[list[Any] \| None, bool]` | Legacy API: Fetch cached KV state for image+prompt combination. | [#L331-L345](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L331-L345) |
| [`MLLMPrefixCacheManager.store`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store) | method | `MLLMPrefixCacheManager.store(images: list[str], prompt: str, vision_embeddings: Any, kv_cache: list[Any], token_ids: list[int], num_image_tokens: int = 0, model_name: str = '') -> None` | Store prefix state in cache. | [#L347-L396](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L347-L396) |
| [`MLLMPrefixCacheManager.store_cache`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.store_cache) | method | `MLLMPrefixCacheManager.store_cache(images: list[str], prompt: str, cache: list[Any] \| None, num_tokens: int = 0) -> None` | Legacy API: Store KV cache for future reuse. | [#L398-L421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L398-L421) |
| [`MLLMPrefixCacheManager.get_stats`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.get_stats) | method | `MLLMPrefixCacheManager.get_stats() -> dict[str, Any]` | Get cache statistics. | [#L423-L430](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L423-L430) |
| [`MLLMPrefixCacheManager.reset_stats`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.reset_stats) | method | `MLLMPrefixCacheManager.reset_stats() -> None` | Reset statistics counters. | [#L432-L434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L432-L434) |
| [`MLLMPrefixCacheManager.clear`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.clear) | method | `MLLMPrefixCacheManager.clear() -> None` | Clear all cached entries and reset stats. | [#L436-L440](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L436-L440) |
| [`MLLMPrefixCacheManager.__len__`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__len__) | method | `MLLMPrefixCacheManager.__len__() -> int` | Return number of cached entries. | [#L442-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L442-L444) |
| [`MLLMPrefixCacheManager.__repr__`](#contract-vllm_mlx.mllm_cache.MLLMPrefixCacheManager.__repr__) | method | `MLLMPrefixCacheManager.__repr__() -> str` | Method `MLLMPrefixCacheManager.__repr__` calls `len`; returns `f'<MLLMPrefixCacheManager entries={len(self)} memory={mem_mb:.1f}MB>'`. | [#L446-L448](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_cache.py#L446-L448) |