# `vllm_mlx.mllm_scheduler`

MLLM Scheduler for multimodal continuous batching.

[View the complete module source at #L1-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1-L1242).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.mllm_scheduler
    options:
      members:
        - logger
        - MLLMSchedulerConfig
        - MLLMRequest
        - MLLMSchedulerOutput
        - MLLMScheduler
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMSchedulerConfig" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMSchedulerConfig</code> · class</summary>

```python
vllm_mlx.mllm_scheduler.MLLMSchedulerConfig(max_num_seqs: int = 16, prefill_batch_size: int = 16, completion_batch_size: int = 16, prefill_step_size: int = 1024, enable_vision_cache: bool = True, vision_cache_size: int = 100, default_max_tokens: int = 256, default_video_fps: float = 2.0, cache_memory_mb: Optional[int] = None, max_video_frames: int = 128, enable_mtp: bool = False, mtp_num_draft_tokens: int = 1, enable_prefix_cache: bool = True, use_memory_aware_cache: bool = True, prefix_cache_memory_mb: Optional[int] = None, kv_cache_quantization: bool = False, kv_cache_quantization_bits: int = 8, kv_cache_quantization_group_size: int = 64, chunked_prefill_tokens: int = 0, max_kv_size: int = 0, ssd_cache_dir: Optional[str] = None, ssd_cache_max_gb: float = 10.0)
```

Configuration for MLLM scheduler.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `max_num_seqs` | `int` | `no` | `16` | Optional constructor field; defaults to `16`. |
| `prefill_batch_size` | `int` | `no` | `16` | Optional constructor field; defaults to `16`. |
| `completion_batch_size` | `int` | `no` | `16` | Optional constructor field; defaults to `16`. |
| `prefill_step_size` | `int` | `no` | `1024` | Optional constructor field; defaults to `1024`. |
| `enable_vision_cache` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `vision_cache_size` | `int` | `no` | `100` | Optional constructor field; defaults to `100`. |
| `default_max_tokens` | `int` | `no` | `256` | Optional constructor field; defaults to `256`. |
| `default_video_fps` | `float` | `no` | `2.0` | Optional constructor field; defaults to `2.0`. |
| `cache_memory_mb` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `max_video_frames` | `int` | `no` | `128` | Optional constructor field; defaults to `128`. |
| `enable_mtp` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `mtp_num_draft_tokens` | `int` | `no` | `1` | Optional constructor field; defaults to `1`. |
| `enable_prefix_cache` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `use_memory_aware_cache` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `prefix_cache_memory_mb` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `kv_cache_quantization` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `kv_cache_quantization_bits` | `int` | `no` | `8` | Optional constructor field; defaults to `8`. |
| `kv_cache_quantization_group_size` | `int` | `no` | `64` | Optional constructor field; defaults to `64`. |
| `chunked_prefill_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `max_kv_size` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `ssd_cache_dir` | `Optional[str]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `ssd_cache_max_gb` | `float` | `no` | `10.0` | Optional constructor field; defaults to `10.0`. |

**Returns**

- Constructs: `vllm_mlx.mllm_scheduler.MLLMSchedulerConfig`

**Exceptions and behavior**

Class `MLLMSchedulerConfig` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L46-L92](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L46-L92).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMRequest" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMRequest</code> · class</summary>

```python
vllm_mlx.mllm_scheduler.MLLMRequest(request_id: str, prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, sampling_params: SamplingParams = field(default_factory=SamplingParams), arrival_time: float = field(default_factory=time.time), batch_uid: Optional[int] = None, status: RequestStatus = RequestStatus.WAITING, output_text: str = '', output_tokens: List[int] = field(default_factory=list), finish_reason: Optional[str] = None, num_prompt_tokens: int = 0, num_output_tokens: int = 0, mtp_drafts: int = 0, mtp_accepted: int = 0, first_token_time: Optional[float] = None)
```

Extended request for MLLM processing.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required constructor field. |
| `prompt` | `str` | `yes` | `none` | Required constructor field. |
| `images` | `Optional[List[str]]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `videos` | `Optional[List[str]]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `audio` | `Optional[List[str]]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `sampling_params` | `SamplingParams` | `no` | `field(default_factory=SamplingParams)` | Optional constructor field; defaults to `field(default_factory=SamplingParams)`. |
| `arrival_time` | `float` | `no` | `field(default_factory=time.time)` | Optional constructor field; defaults to `field(default_factory=time.time)`. |
| `batch_uid` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `status` | `RequestStatus` | `no` | `RequestStatus.WAITING` | Optional constructor field; defaults to `RequestStatus.WAITING`. |
| `output_text` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. |
| `output_tokens` | `List[int]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `finish_reason` | `Optional[str]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `num_prompt_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `num_output_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `mtp_drafts` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `mtp_accepted` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `first_token_time` | `Optional[float]` | `no` | `None` | Optional constructor field; defaults to `None`. |

**Returns**

- Constructs: `vllm_mlx.mllm_scheduler.MLLMRequest`

**Exceptions and behavior**

Class `MLLMRequest` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L96-L127](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L96-L127).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMSchedulerOutput" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMSchedulerOutput</code> · class</summary>

```python
vllm_mlx.mllm_scheduler.MLLMSchedulerOutput(scheduled_request_ids: List[str] = field(default_factory=list), num_scheduled_tokens: int = 0, finished_request_ids: Set[str] = field(default_factory=set), outputs: List[RequestOutput] = field(default_factory=list), has_work: bool = False)
```

Output from a scheduling step.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `scheduled_request_ids` | `List[str]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `num_scheduled_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `finished_request_ids` | `Set[str]` | `no` | `field(default_factory=set)` | Optional constructor field; defaults to `field(default_factory=set)`. |
| `outputs` | `List[RequestOutput]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `has_work` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |

**Returns**

- Constructs: `vllm_mlx.mllm_scheduler.MLLMSchedulerOutput`

**Exceptions and behavior**

Class `MLLMSchedulerOutput` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L131-L147](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L131-L147).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler</code> · class</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler(model: Any, processor: Any, config: Optional[MLLMSchedulerConfig] = None)
```

Scheduler for Vision Language Model requests with continuous batching.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The VLM model |
| `processor` | `Any` | `yes` | `none` | The VLM processor |
| `config` | `Optional[MLLMSchedulerConfig]` | `no` | `None` | Scheduler configuration |

**Returns**

- Constructs: `vllm_mlx.mllm_scheduler.MLLMScheduler`

**Exceptions and behavior**

Class `MLLMScheduler` declares 24 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L150-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L150-L1242).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.__init__" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.__init__</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.__init__(model: Any, processor: Any, config: Optional[MLLMSchedulerConfig] = None) -> not annotated
```

Initialize MLLM scheduler.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The VLM model |
| `processor` | `Any` | `yes` | `none` | The VLM processor |
| `config` | `Optional[MLLMSchedulerConfig]` | `no` | `None` | Scheduler configuration |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `MLLMScheduler.__init__` updates `self.model`, `self.processor`, `self.config`, `self.model_config`; calls `MLLMSchedulerConfig`, `getattr`, `MultimodalProcessor`, `self._get_stop_tokens`.
No direct `raise` statement appears in this definition.

[View source #L183-L248](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L183-L248).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._get_stop_tokens" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._get_stop_tokens</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._get_stop_tokens() -> Set[int]
```

Get stop token IDs from tokenizer and generation_config.json.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Set[int]`
- Direct return expressions: `stop_tokens`

**Exceptions and behavior**

Method `MLLMScheduler._get_stop_tokens` calls `set`, `hasattr`, `isinstance`, `stop_tokens.update`; returns `stop_tokens`.
No direct `raise` statement appears in this definition.

[View source #L250-L290](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L250-L290).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._ensure_batch_generator" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._ensure_batch_generator</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._ensure_batch_generator() -> None
```

Ensure batch generator exists.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMScheduler._ensure_batch_generator` updates `self.batch_generator`, `self._ssd_tier`; calls `make_sampler`, `MemoryCacheConfig`, `MLLMBatchGenerator`, `getattr`.
No direct `raise` statement appears in this definition.

[View source #L292-L374](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L292-L374).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.add_request" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.add_request</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.add_request(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, request_id: Optional[str] = None, **kwargs) -> str
```

Add a multimodal request to the scheduler (sync version).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Text prompt (should be formatted with chat template) |
| `images` | `Optional[List[str]]` | `no` | `None` | List of image inputs (paths, URLs, base64) |
| `videos` | `Optional[List[str]]` | `no` | `None` | List of video inputs |
| `audio` | `Optional[List[str]]` | `no` | `None` | List of audio inputs |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `request_id` | `Optional[str]` | `no` | `None` | Optional custom request ID |
| `**kwargs` | `not annotated` | `no` | `none` | Additional generation parameters. ``logits_processors`` — list of callables ``(tokens, logits) -> logits`` applied during sampling (e.g. constrained JSON decoding). |

**Returns**

- Type: `str`
- Direct return expressions: `request_id`

**Exceptions and behavior**

Method `MLLMScheduler.add_request` calls `str`, `uuid.uuid4`, `SamplingParams`, `kwargs.pop`; returns `request_id`.
No direct `raise` statement appears in this definition.

[View source #L378-L453](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L378-L453).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.abort_request" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.abort_request</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.abort_request(request_id: str) -> bool
```

Abort a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | The request ID to abort |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `True`

**Exceptions and behavior**

Method `MLLMScheduler.abort_request` updates `self.total_completion_tokens`, `self.total_prompt_tokens`; calls `self.requests.get`, `self.batch_generator.abort_prefill`, `self.waiting.remove`, `self.batch_generator.schedule_removal`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L455-L532](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L455-L532).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.has_requests" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.has_requests</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.has_requests() -> bool
```

Check if there are any pending or running requests.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `bool(self.waiting or self.running)`

**Exceptions and behavior**

Method `MLLMScheduler.has_requests` calls `bool`; returns `bool(self.waiting or self.running)`.
No direct `raise` statement appears in this definition.

[View source #L534-L536](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L534-L536).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_waiting" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_waiting</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_waiting() -> int
```

Get number of waiting requests.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int`
- Direct return expressions: `len(self.waiting)`

**Exceptions and behavior**

Method `MLLMScheduler.get_num_waiting` calls `len`; returns `len(self.waiting)`.
No direct `raise` statement appears in this definition.

[View source #L538-L540](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L538-L540).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_running" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_running</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_running() -> int
```

Get number of running requests.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int`
- Direct return expressions: `len(self.running)`

**Exceptions and behavior**

Method `MLLMScheduler.get_num_running` calls `len`; returns `len(self.running)`.
No direct `raise` statement appears in this definition.

[View source #L542-L544](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L542-L544).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._schedule_waiting" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._schedule_waiting</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._schedule_waiting() -> List[MLLMRequest]
```

Move requests from waiting queue to running.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `List[MLLMRequest]`
- Direct return expressions: `scheduled`

**Exceptions and behavior**

Method `MLLMScheduler._schedule_waiting` updates `self.total_prompt_tokens`; calls `self._ensure_batch_generator`, `len`, `self.waiting.popleft`, `MLLMBatchRequest`; returns `scheduled`.
No direct `raise` statement appears in this definition.

[View source #L546-L597](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L546-L597).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_batch_responses" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._process_batch_responses</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._process_batch_responses(responses: List[MLLMBatchResponse]) -> Tuple[List[RequestOutput], Set[str]]
```

Process responses from batch generator.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `responses` | `List[MLLMBatchResponse]` | `yes` | `none` | List of MLLMBatchResponse objects |

**Returns**

- Type: `Tuple[List[RequestOutput], Set[str]]`
- Direct return expressions: `(outputs, finished_ids)`

**Exceptions and behavior**

Method `MLLMScheduler._process_batch_responses` updates `self.num_requests_processed`, `self.total_completion_tokens`; calls `set`, `hasattr`, `self.uid_to_request_id.get`, `self.running.get`; returns `(outputs, finished_ids)`.
No direct `raise` statement appears in this definition.

[View source #L599-L716](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L599-L716).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._cleanup_finished" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._cleanup_finished</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._cleanup_finished(finished_ids: Set[str]) -> None
```

Clean up finished requests.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `finished_ids` | `Set[str]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMScheduler._cleanup_finished` calls `self.requests.pop`, `self._detokenizer_pool.pop`, `self.finished_req_ids.add`, `mx.clear_cache`.
No direct `raise` statement appears in this definition.

[View source #L718-L744](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L718-L744).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.step" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.step</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.step() -> MLLMSchedulerOutput
```

Execute one scheduling step.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `MLLMSchedulerOutput`
- Direct return expressions: `output`

**Exceptions and behavior**

Method `MLLMScheduler.step` updates `self._step_count`, `self.finished_req_ids`; calls `MLLMSchedulerOutput`, `self.batch_generator.process_pending_removals`, `self._schedule_waiting`, `sum`; returns `output`.
No direct `raise` statement appears in this definition.

[View source #L746-L813](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L746-L813).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_request" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.get_request</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.get_request(request_id: str) -> Optional[MLLMRequest]
```

Get a request by ID.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `Optional[MLLMRequest]`
- Direct return expressions: `self.requests.get(request_id)`

**Exceptions and behavior**

Method `MLLMScheduler.get_request` calls `self.requests.get`; returns `self.requests.get(request_id)`.
No direct `raise` statement appears in this definition.

[View source #L815-L817](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L815-L817).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.remove_finished_request" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.remove_finished_request</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.remove_finished_request(request_id: str) -> Optional[MLLMRequest]
```

Remove a finished request from tracking.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `Optional[MLLMRequest]`
- Direct return expressions: `self.requests.pop(request_id, None)`

**Exceptions and behavior**

Method `MLLMScheduler.remove_finished_request` calls `self.requests.pop`; returns `self.requests.pop(request_id, None)`.
No direct `raise` statement appears in this definition.

[View source #L819-L821](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L819-L821).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.start" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.start</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler.start() -> None
```

Start the async scheduler processing loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Method `MLLMScheduler.start` updates `self._running`, `self._processing_task`; calls `asyncio.create_task`, `self._process_loop`, `logger.info`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L825-L834](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L825-L834).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.stop" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.stop</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler.stop() -> None
```

Stop the scheduler.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMScheduler.stop` updates `self._running`, `self.batch_generator`; calls `self._processing_task.cancel`, `self.batch_generator.close`, `logger.info`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L836-L850](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L836-L850).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop() -> None
```

Main async processing loop.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMScheduler._process_loop` calls `asyncio.get_running_loop`, `list`, `getattr`, `time.perf_counter`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L852-L947](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L852-L947).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop._ensure_streams_bound" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop._ensure_streams_bound</code> · nested function</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop._ensure_streams_bound() -> None
```

Nested Function `MLLMScheduler._process_loop._ensure_streams_bound` calls `bind_generation_streams`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Nested Function `MLLMScheduler._process_loop._ensure_streams_bound` calls `bind_generation_streams`.
No direct `raise` statement appears in this definition.

[View source #L867-L871](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L867-L871).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.add_request_async" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.add_request_async</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler.add_request_async(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, **kwargs) -> str
```

Add a multimodal request (async version with output queue).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Text prompt |
| `images` | `Optional[List[str]]` | `no` | `None` | List of image inputs |
| `videos` | `Optional[List[str]]` | `no` | `None` | List of video inputs |
| `audio` | `Optional[List[str]]` | `no` | `None` | List of audio inputs |
| `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate |
| `temperature` | `float` | `no` | `0.7` | Sampling temperature |
| `top_p` | `float` | `no` | `0.9` | Top-p sampling |
| `**kwargs` | `not annotated` | `no` | `none` | Additional parameters |

**Returns**

- Type: `str`
- Direct return expressions: `request_id`

**Exceptions and behavior**

Method `MLLMScheduler.add_request_async` calls `self.add_request`, `asyncio.Queue`; returns `request_id`.
No direct `raise` statement appears in this definition.

[View source #L949-L990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L949-L990).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.stream_outputs" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.stream_outputs</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler.stream_outputs(request_id: str) -> AsyncIterator[RequestOutput]
```

Stream outputs for a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | The request ID to stream |

**Returns**

- Type: `AsyncIterator[RequestOutput]`
- Direct return expressions: `None`
- Yields values incrementally.

**Exceptions and behavior**

Method `MLLMScheduler.stream_outputs` calls `self.output_queues.get`, `output_queue.get`, `logger.info`, `self.abort_request`; awaits asynchronous work; yields values incrementally; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L992-L1027](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L992-L1027).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.generate" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.generate</code> · method</summary>

```python
async vllm_mlx.mllm_scheduler.MLLMScheduler.generate(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, **kwargs) -> RequestOutput
```

Generate complete output for a request (non-streaming).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompt` | `str` | `yes` | `none` | Text prompt |
| `images` | `Optional[List[str]]` | `no` | `None` | Image inputs |
| `videos` | `Optional[List[str]]` | `no` | `None` | Video inputs |
| `audio` | `Optional[List[str]]` | `no` | `None` | Audio inputs |
| `**kwargs` | `not annotated` | `no` | `none` | Generation parameters |

**Returns**

- Type: `RequestOutput`
- Direct return expressions: `final_output`

**Exceptions and behavior**

Method `MLLMScheduler.generate` calls `self.add_request_async`, `self.stream_outputs`, `RequestOutput`; awaits asynchronous work; returns `final_output`.
No direct `raise` statement appears in this definition.

[View source #L1029-L1078](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1029-L1078).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_running_requests_info" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.get_running_requests_info</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.get_running_requests_info() -> List[Dict[str, Any]]
```

Per-request details for status endpoint.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `List[Dict[str, Any]]`
- Direct return expressions: `result`

**Exceptions and behavior**

Method `MLLMScheduler.get_running_requests_info` calls `time.time`, `result.append`, `round`, `self.running.values`; returns `result`.
No direct `raise` statement appears in this definition.

[View source #L1082-L1151](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1082-L1151).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_stats" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.get_stats</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.get_stats() -> Dict[str, Any]
```

Get scheduler statistics.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, Any]`
- Direct return expressions: `stats`

**Exceptions and behavior**

Method `MLLMScheduler.get_stats` calls `len`, `self.get_running_requests_info`, `self.batch_generator.stats`, `batch_stats.to_dict`; returns `stats`.
No direct `raise` statement appears in this definition.

[View source #L1153-L1204](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1153-L1204).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.clear_runtime_caches" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.clear_runtime_caches</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.clear_runtime_caches() -> Dict[str, bool]
```

Clear runtime caches without resetting scheduler/request state.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `Dict[str, bool]`
- Direct return expressions: `cleared`

**Exceptions and behavior**

Method `MLLMScheduler.clear_runtime_caches` calls `self.vision_cache.clear`, `self.batch_generator.prefix_cache.clear`; returns `cleared`.
No direct `raise` statement appears in this definition.

[View source #L1206-L1221](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1206-L1221).

</details>

<details class="api-contract" id="contract-vllm_mlx.mllm_scheduler.MLLMScheduler.reset" markdown="1">
<summary><code>vllm_mlx.mllm_scheduler.MLLMScheduler.reset</code> · method</summary>

```python
vllm_mlx.mllm_scheduler.MLLMScheduler.reset() -> None
```

Reset the scheduler state.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `MLLMScheduler.reset` updates `self.batch_generator`; calls `list`, `self.requests.keys`, `self.abort_request`, `self.waiting.clear`.
No direct `raise` statement appears in this definition.

[View source #L1223-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1223-L1242).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`MLLMSchedulerConfig`](#contract-vllm_mlx.mllm_scheduler.MLLMSchedulerConfig) | class | `MLLMSchedulerConfig(max_num_seqs: int = 16, prefill_batch_size: int = 16, completion_batch_size: int = 16, prefill_step_size: int = 1024, enable_vision_cache: bool = True, vision_cache_size: int = 100, default_max_tokens: int = 256, default_video_fps: float = 2.0, cache_memory_mb: Optional[int] = None, max_video_frames: int = 128, enable_mtp: bool = False, mtp_num_draft_tokens: int = 1, enable_prefix_cache: bool = True, use_memory_aware_cache: bool = True, prefix_cache_memory_mb: Optional[int] = None, kv_cache_quantization: bool = False, kv_cache_quantization_bits: int = 8, kv_cache_quantization_group_size: int = 64, chunked_prefill_tokens: int = 0, max_kv_size: int = 0, ssd_cache_dir: Optional[str] = None, ssd_cache_max_gb: float = 10.0)` | Configuration for MLLM scheduler. | [#L46-L92](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L46-L92) |
| [`MLLMRequest`](#contract-vllm_mlx.mllm_scheduler.MLLMRequest) | class | `MLLMRequest(request_id: str, prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, sampling_params: SamplingParams = field(default_factory=SamplingParams), arrival_time: float = field(default_factory=time.time), batch_uid: Optional[int] = None, status: RequestStatus = RequestStatus.WAITING, output_text: str = '', output_tokens: List[int] = field(default_factory=list), finish_reason: Optional[str] = None, num_prompt_tokens: int = 0, num_output_tokens: int = 0, mtp_drafts: int = 0, mtp_accepted: int = 0, first_token_time: Optional[float] = None)` | Extended request for MLLM processing. | [#L96-L127](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L96-L127) |
| [`MLLMSchedulerOutput`](#contract-vllm_mlx.mllm_scheduler.MLLMSchedulerOutput) | class | `MLLMSchedulerOutput(scheduled_request_ids: List[str] = field(default_factory=list), num_scheduled_tokens: int = 0, finished_request_ids: Set[str] = field(default_factory=set), outputs: List[RequestOutput] = field(default_factory=list), has_work: bool = False)` | Output from a scheduling step. | [#L131-L147](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L131-L147) |
| [`MLLMScheduler`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler) | class | `MLLMScheduler(model: Any, processor: Any, config: Optional[MLLMSchedulerConfig] = None)` | Scheduler for Vision Language Model requests with continuous batching. | [#L150-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L150-L1242) |
| [`MLLMScheduler.__init__`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.__init__) | method | `MLLMScheduler.__init__(model: Any, processor: Any, config: Optional[MLLMSchedulerConfig] = None) -> not annotated` | Initialize MLLM scheduler. | [#L183-L248](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L183-L248) |
| [`MLLMScheduler._get_stop_tokens`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._get_stop_tokens) | method | `MLLMScheduler._get_stop_tokens() -> Set[int]` | Get stop token IDs from tokenizer and generation_config.json. | [#L250-L290](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L250-L290) |
| [`MLLMScheduler._ensure_batch_generator`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._ensure_batch_generator) | method | `MLLMScheduler._ensure_batch_generator() -> None` | Ensure batch generator exists. | [#L292-L374](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L292-L374) |
| [`MLLMScheduler.add_request`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.add_request) | method | `MLLMScheduler.add_request(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, request_id: Optional[str] = None, **kwargs) -> str` | Add a multimodal request to the scheduler (sync version). | [#L378-L453](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L378-L453) |
| [`MLLMScheduler.abort_request`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.abort_request) | method | `MLLMScheduler.abort_request(request_id: str) -> bool` | Abort a request. | [#L455-L532](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L455-L532) |
| [`MLLMScheduler.has_requests`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.has_requests) | method | `MLLMScheduler.has_requests() -> bool` | Check if there are any pending or running requests. | [#L534-L536](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L534-L536) |
| [`MLLMScheduler.get_num_waiting`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_waiting) | method | `MLLMScheduler.get_num_waiting() -> int` | Get number of waiting requests. | [#L538-L540](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L538-L540) |
| [`MLLMScheduler.get_num_running`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_num_running) | method | `MLLMScheduler.get_num_running() -> int` | Get number of running requests. | [#L542-L544](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L542-L544) |
| [`MLLMScheduler._schedule_waiting`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._schedule_waiting) | method | `MLLMScheduler._schedule_waiting() -> List[MLLMRequest]` | Move requests from waiting queue to running. | [#L546-L597](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L546-L597) |
| [`MLLMScheduler._process_batch_responses`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_batch_responses) | method | `MLLMScheduler._process_batch_responses(responses: List[MLLMBatchResponse]) -> Tuple[List[RequestOutput], Set[str]]` | Process responses from batch generator. | [#L599-L716](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L599-L716) |
| [`MLLMScheduler._cleanup_finished`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._cleanup_finished) | method | `MLLMScheduler._cleanup_finished(finished_ids: Set[str]) -> None` | Clean up finished requests. | [#L718-L744](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L718-L744) |
| [`MLLMScheduler.step`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.step) | method | `MLLMScheduler.step() -> MLLMSchedulerOutput` | Execute one scheduling step. | [#L746-L813](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L746-L813) |
| [`MLLMScheduler.get_request`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_request) | method | `MLLMScheduler.get_request(request_id: str) -> Optional[MLLMRequest]` | Get a request by ID. | [#L815-L817](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L815-L817) |
| [`MLLMScheduler.remove_finished_request`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.remove_finished_request) | method | `MLLMScheduler.remove_finished_request(request_id: str) -> Optional[MLLMRequest]` | Remove a finished request from tracking. | [#L819-L821](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L819-L821) |
| [`MLLMScheduler.start`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.start) | method | `async MLLMScheduler.start() -> None` | Start the async scheduler processing loop. | [#L825-L834](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L825-L834) |
| [`MLLMScheduler.stop`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.stop) | method | `async MLLMScheduler.stop() -> None` | Stop the scheduler. | [#L836-L850](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L836-L850) |
| [`MLLMScheduler._process_loop`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop) | method | `async MLLMScheduler._process_loop() -> None` | Main async processing loop. | [#L852-L947](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L852-L947) |
| [`MLLMScheduler._process_loop._ensure_streams_bound`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler._process_loop._ensure_streams_bound) | nested function | `MLLMScheduler._process_loop._ensure_streams_bound() -> None` | Nested Function `MLLMScheduler._process_loop._ensure_streams_bound` calls `bind_generation_streams`. | [#L867-L871](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L867-L871) |
| [`MLLMScheduler.add_request_async`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.add_request_async) | method | `async MLLMScheduler.add_request_async(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, **kwargs) -> str` | Add a multimodal request (async version with output queue). | [#L949-L990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L949-L990) |
| [`MLLMScheduler.stream_outputs`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.stream_outputs) | method | `async MLLMScheduler.stream_outputs(request_id: str) -> AsyncIterator[RequestOutput]` | Stream outputs for a request. | [#L992-L1027](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L992-L1027) |
| [`MLLMScheduler.generate`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.generate) | method | `async MLLMScheduler.generate(prompt: str, images: Optional[List[str]] = None, videos: Optional[List[str]] = None, audio: Optional[List[str]] = None, **kwargs) -> RequestOutput` | Generate complete output for a request (non-streaming). | [#L1029-L1078](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1029-L1078) |
| [`MLLMScheduler.get_running_requests_info`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_running_requests_info) | method | `MLLMScheduler.get_running_requests_info() -> List[Dict[str, Any]]` | Per-request details for status endpoint. | [#L1082-L1151](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1082-L1151) |
| [`MLLMScheduler.get_stats`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.get_stats) | method | `MLLMScheduler.get_stats() -> Dict[str, Any]` | Get scheduler statistics. | [#L1153-L1204](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1153-L1204) |
| [`MLLMScheduler.clear_runtime_caches`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.clear_runtime_caches) | method | `MLLMScheduler.clear_runtime_caches() -> Dict[str, bool]` | Clear runtime caches without resetting scheduler/request state. | [#L1206-L1221](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1206-L1221) |
| [`MLLMScheduler.reset`](#contract-vllm_mlx.mllm_scheduler.MLLMScheduler.reset) | method | `MLLMScheduler.reset() -> None` | Reset the scheduler state. | [#L1223-L1242](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/mllm_scheduler.py#L1223-L1242) |