# `vllm_mlx.scheduler`
Scheduler for vllm-mlx continuous batching.
[View the complete module source at #L1-L3518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1-L3518).
## API details
Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.
::: vllm_mlx.scheduler
options:
members:
- logger
- CACHE_CORRUPTION_PATTERNS
- _normalize_logits_processors
- _sanitize_batch_generator_logits_processors
- SchedulingPolicy
- SchedulerConfig
- SchedulerOutput
- _install_prompt_cache_save
- _install_chunked_prefill
- _MTPStatsState
- _configure_chunked_prefill
- _install_mtp
- _mtp_status_snapshot
- Scheduler
filters: []
show_if_no_docstring: true
## Complete contract reference
Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.
vllm_mlx.scheduler._normalize_logits_processors · function
```python
vllm_mlx.scheduler._normalize_logits_processors(logits_processors) -> not annotated
```
Normalize empty per-sequence processor slots to lists.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `logits_processors` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `None`; `[processors or [] for processors in logits_processors]`
**Exceptions and behavior**
Function `_normalize_logits_processors` has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L46-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L46-L50).
vllm_mlx.scheduler._sanitize_batch_generator_logits_processors · function
```python
vllm_mlx.scheduler._sanitize_batch_generator_logits_processors(batch_generator) -> None
```
Sanitize stale BatchGenerator processor state before decode.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `batch_generator` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Function `_sanitize_batch_generator_logits_processors` calls `getattr`, `hasattr`, `_normalize_logits_processors`, `isinstance`.
No direct `raise` statement appears in this definition.
[View source #L53-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L53-L65).
vllm_mlx.scheduler.SchedulingPolicy · class
```python
vllm_mlx.scheduler.SchedulingPolicy()
```
Scheduling policy for request ordering.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Constructs: `vllm_mlx.scheduler.SchedulingPolicy`
**Exceptions and behavior**
Class `SchedulingPolicy` derives from `Enum` and declares 0 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L68-L72](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L68-L72).
vllm_mlx.scheduler.SchedulerConfig · class
```python
vllm_mlx.scheduler.SchedulerConfig(max_num_seqs: int = 256, max_num_batched_tokens: int = 8192, policy: SchedulingPolicy = SchedulingPolicy.FCFS, prefill_batch_size: int = 8, completion_batch_size: int = 32, prefill_step_size: int = 2048, mllm_prefill_step_size: Optional[int] = None, enable_prefix_cache: bool = True, prefix_cache_size: int = 100, use_memory_aware_cache: bool = True, cache_memory_mb: Optional[int] = None, cache_memory_percent: float = 0.2, kv_cache_quantization: bool = False, kv_cache_quantization_bits: int = 8, kv_cache_quantization_group_size: int = 64, kv_cache_min_quantize_tokens: int = 256, use_paged_cache: bool = False, paged_cache_block_size: int = 64, max_cache_blocks: int = 1000, chunked_prefill_tokens: int = 0, mid_prefill_save_interval: int = 8192, ssd_cache_dir: Optional[str] = None, ssd_cache_max_gb: float = 10.0, max_kv_size: int = 0, enable_mtp: bool = False, mtp_num_draft_tokens: int = 1, mtp_optimistic: bool = False)
```
Configuration for the scheduler.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `max_num_seqs` | `int` | `no` | `256` | Optional constructor field; defaults to `256`. |
| `max_num_batched_tokens` | `int` | `no` | `8192` | Optional constructor field; defaults to `8192`. |
| `policy` | `SchedulingPolicy` | `no` | `SchedulingPolicy.FCFS` | Optional constructor field; defaults to `SchedulingPolicy.FCFS`. |
| `prefill_batch_size` | `int` | `no` | `8` | Optional constructor field; defaults to `8`. |
| `completion_batch_size` | `int` | `no` | `32` | Optional constructor field; defaults to `32`. |
| `prefill_step_size` | `int` | `no` | `2048` | Optional constructor field; defaults to `2048`. |
| `mllm_prefill_step_size` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `enable_prefix_cache` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `prefix_cache_size` | `int` | `no` | `100` | Optional constructor field; defaults to `100`. |
| `use_memory_aware_cache` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. |
| `cache_memory_mb` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `cache_memory_percent` | `float` | `no` | `0.2` | Optional constructor field; defaults to `0.2`. |
| `kv_cache_quantization` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `kv_cache_quantization_bits` | `int` | `no` | `8` | Optional constructor field; defaults to `8`. |
| `kv_cache_quantization_group_size` | `int` | `no` | `64` | Optional constructor field; defaults to `64`. |
| `kv_cache_min_quantize_tokens` | `int` | `no` | `256` | Optional constructor field; defaults to `256`. |
| `use_paged_cache` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `paged_cache_block_size` | `int` | `no` | `64` | Optional constructor field; defaults to `64`. |
| `max_cache_blocks` | `int` | `no` | `1000` | Optional constructor field; defaults to `1000`. |
| `chunked_prefill_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `mid_prefill_save_interval` | `int` | `no` | `8192` | Optional constructor field; defaults to `8192`. |
| `ssd_cache_dir` | `Optional[str]` | `no` | `None` | Optional constructor field; defaults to `None`. |
| `ssd_cache_max_gb` | `float` | `no` | `10.0` | Optional constructor field; defaults to `10.0`. |
| `max_kv_size` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `enable_mtp` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
| `mtp_num_draft_tokens` | `int` | `no` | `1` | Optional constructor field; defaults to `1`. |
| `mtp_optimistic` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
**Returns**
- Constructs: `vllm_mlx.scheduler.SchedulerConfig`
**Exceptions and behavior**
Class `SchedulerConfig` declares 1 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L76-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L76-L140).
vllm_mlx.scheduler.SchedulerConfig.__post_init__ · method
```python
vllm_mlx.scheduler.SchedulerConfig.__post_init__() -> None
```
Method `SchedulerConfig.__post_init__` calls `ValueError`; can raise `ValueError`.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `SchedulerConfig.__post_init__` calls `ValueError`; can raise `ValueError`.
Directly raised exceptions: `ValueError`.
[View source #L138-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L138-L140).
vllm_mlx.scheduler.SchedulerOutput · class
```python
vllm_mlx.scheduler.SchedulerOutput(scheduled_request_ids: List[str] = field(default_factory=list), num_scheduled_tokens: int = 0, finished_request_ids: Set[str] = field(default_factory=set), outputs: List[RequestOutput] = field(default_factory=list), has_work: bool = False)
```
Output from a scheduling step.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `scheduled_request_ids` | `List[str]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `num_scheduled_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. |
| `finished_request_ids` | `Set[str]` | `no` | `field(default_factory=set)` | Optional constructor field; defaults to `field(default_factory=set)`. |
| `outputs` | `List[RequestOutput]` | `no` | `field(default_factory=list)` | Optional constructor field; defaults to `field(default_factory=list)`. |
| `has_work` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. |
**Returns**
- Constructs: `vllm_mlx.scheduler.SchedulerOutput`
**Exceptions and behavior**
Class `SchedulerOutput` declares 0 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L144-L160](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L144-L160).
vllm_mlx.scheduler._install_prompt_cache_save · function
```python
vllm_mlx.scheduler._install_prompt_cache_save(batch_gen: 'BatchGenerator', prompt_cache_save) -> None
```
Monkey-patch ``_process_prompts`` to capture prompt-only cache state.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `batch_gen` | `'BatchGenerator'` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache_save` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Function `_install_prompt_cache_save` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.
[View source #L163-L187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L163-L187).
vllm_mlx.scheduler._install_prompt_cache_save._patched_process_prompts · nested function
```python
vllm_mlx.scheduler._install_prompt_cache_save._patched_process_prompts(prompts, _self = batch_gen) -> not annotated
```
Nested Function `_install_prompt_cache_save._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompts` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `_self` | `not annotated` | `no` | `batch_gen` | Optional positional or keyword input; defaults to `batch_gen`. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `batch`
**Exceptions and behavior**
Nested Function `_install_prompt_cache_save._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`.
No direct `raise` statement appears in this definition.
[View source #L177-L185](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L177-L185).
vllm_mlx.scheduler._install_chunked_prefill · function
```python
vllm_mlx.scheduler._install_chunked_prefill(batch_gen: 'BatchGenerator', budget: int, mid_prefill_save = None, prompt_cache_save = None, pending_abort_ids: Optional[Set[str]] = None, uid_to_request_id: Optional[Dict[int, str]] = None, requests: Optional[Dict[str, Any]] = None) -> None
```
Monkey-patch a BatchGenerator instance so that large prefills are broken into chunks of at most *budget* tokens each.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `batch_gen` | `'BatchGenerator'` | `yes` | `none` | The BatchGenerator to patch. |
| `budget` | `int` | `yes` | `none` | Max tokens per prefill chunk. |
| `mid_prefill_save` | `not annotated` | `no` | `None` | Optional callback(uid, processed, prompt_cache) called after each chunk to save intermediate KV cache state. |
| `prompt_cache_save` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `pending_abort_ids` | `Optional[Set[str]]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `uid_to_request_id` | `Optional[Dict[int, str]]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `requests` | `Optional[Dict[str, Any]]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Function `_install_chunked_prefill` calls `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L190-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L190-L697).
vllm_mlx.scheduler._install_chunked_prefill._batch_cls · nested class
```python
vllm_mlx.scheduler._install_chunked_prefill._batch_cls(uids: List[int], y: Any, logprobs: List[Any], max_tokens: List[int], num_tokens: List[int], cache: List[Any], samplers: List[Any], logits_processors: List[Any], tokens: List[Any])
```
Nested Class `_install_chunked_prefill._batch_cls` declares 4 direct member(s).
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `uids` | `List[int]` | `yes` | `none` | Required constructor field. |
| `y` | `Any` | `yes` | `none` | Required constructor field. |
| `logprobs` | `List[Any]` | `yes` | `none` | Required constructor field. |
| `max_tokens` | `List[int]` | `yes` | `none` | Required constructor field. |
| `num_tokens` | `List[int]` | `yes` | `none` | Required constructor field. |
| `cache` | `List[Any]` | `yes` | `none` | Required constructor field. |
| `samplers` | `List[Any]` | `yes` | `none` | Required constructor field. |
| `logits_processors` | `List[Any]` | `yes` | `none` | Required constructor field. |
| `tokens` | `List[Any]` | `yes` | `none` | Required constructor field. |
**Returns**
- Constructs: `vllm_mlx.scheduler._install_chunked_prefill._batch_cls`
**Exceptions and behavior**
Nested Class `_install_chunked_prefill._batch_cls` declares 4 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L233-L273](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L233-L273).
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.__len__ · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.__len__() -> not annotated
```
Nested Function `_install_chunked_prefill._batch_cls.__len__` calls `len`; returns `len(self.uids)`.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `not annotated`
- Direct return expressions: `len(self.uids)`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._batch_cls.__len__` calls `len`; returns `len(self.uids)`.
No direct `raise` statement appears in this definition.
[View source #L244-L245](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L244-L245).
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.filter · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.filter(keep_idx: List[int]) -> not annotated
```
Nested Function `_install_chunked_prefill._batch_cls.filter` updates `self.uids`, `self.logprobs`, `self.max_tokens`, `self.num_tokens`; calls `mx.array`, `c.filter`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `keep_idx` | `List[int]` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._batch_cls.filter` updates `self.uids`, `self.logprobs`, `self.max_tokens`, `self.num_tokens`; calls `mx.array`, `c.filter`.
No direct `raise` statement appears in this definition.
[View source #L247-L258](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L247-L258).
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.extend · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._batch_cls.extend(other) -> not annotated
```
Nested Function `_install_chunked_prefill._batch_cls.extend` updates `self.y`; calls `self.uids.extend`, `mx.concatenate`, `self.logprobs.extend`, `self.num_tokens.extend`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `other` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._batch_cls.extend` updates `self.y`; calls `self.uids.extend`, `mx.concatenate`, `self.logprobs.extend`, `self.num_tokens.extend`.
No direct `raise` statement appears in this definition.
[View source #L260-L270](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L260-L270).
vllm_mlx.scheduler._install_chunked_prefill._patched_process_prompts · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._patched_process_prompts(prompts, _self = batch_gen) -> not annotated
```
Nested Function `_install_chunked_prefill._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prompts` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `_self` | `not annotated` | `no` | `batch_gen` | Optional positional or keyword input; defaults to `batch_gen`. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `batch`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`.
No direct `raise` statement appears in this definition.
[View source #L291-L299](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L291-L299).
vllm_mlx.scheduler._install_chunked_prefill._generation_step · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._generation_step() -> not annotated
```
Run one generation step on the active batch.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `not annotated`
- Direct return expressions: `[]`; `responses`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._generation_step` updates `self._stats.generation_time`, `self.active_batch`, `self._stats.generation_tokens`; calls `len`, `_time.perf_counter`, `enumerate`, `mx.concatenate`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L303-L360](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L303-L360).
vllm_mlx.scheduler._install_chunked_prefill._chunked_next · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._chunked_next() -> not annotated
```
Replacement for _next() that chunks large prefills.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `not annotated`
- Direct return expressions: `self._generation_step()`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._chunked_next` updates `self._partial`, `self.active_batch`, `self._stats.prompt_time`, `self._stats.generation_time`; calls `uid_to_request_id.get`, `logger.info`, `mx.clear_cache`, `self._generation_step`; returns `self._generation_step()`.
No direct `raise` statement appears in this definition.
[View source #L362-L678](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L362-L678).
vllm_mlx.scheduler._install_chunked_prefill._patched_remove · nested function
```python
vllm_mlx.scheduler._install_chunked_prefill._patched_remove(uids_to_remove, _self = batch_gen) -> not annotated
```
Clear partial state if aborted request is being prefilled.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `uids_to_remove` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `_self` | `not annotated` | `no` | `batch_gen` | Optional positional or keyword input; defaults to `batch_gen`. |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Nested Function `_install_chunked_prefill._patched_remove` calls `set`, `logger.info`, `mx.clear_cache`, `_orig_remove`.
No direct `raise` statement appears in this definition.
[View source #L680-L691](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L680-L691).
vllm_mlx.scheduler._MTPStatsState · class
```python
vllm_mlx.scheduler._MTPStatsState(counters: Dict[str, int] = field(default_factory=lambda: {'attempted': 0, 'accepted': 0, 'rejected': 0, 'errors': 0}), bypass_counts: Dict[str, int] = field(default_factory=lambda: {'prefill': 0, 'no_active_batch': 0, 'cache_mismatch': 0}), lock: Any = field(default_factory=Lock))
```
Cumulative native-MTP counters shared across generator instances.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `counters` | `Dict[str, int]` | `no` | `field(default_factory=lambda: {'attempted': 0, 'accepted': 0, 'rejected': 0, 'errors': 0})` | Optional constructor field; defaults to `field(default_factory=lambda: {'attempted': 0, 'accepted': 0, 'rejected': 0, 'errors': 0})`. |
| `bypass_counts` | `Dict[str, int]` | `no` | `field(default_factory=lambda: {'prefill': 0, 'no_active_batch': 0, 'cache_mismatch': 0})` | Optional constructor field; defaults to `field(default_factory=lambda: {'prefill': 0, 'no_active_batch': 0, 'cache_mismatch': 0})`. |
| `lock` | `Any` | `no` | `field(default_factory=Lock)` | Optional constructor field; defaults to `field(default_factory=Lock)`. |
**Returns**
- Constructs: `vllm_mlx.scheduler._MTPStatsState`
**Exceptions and behavior**
Class `_MTPStatsState` declares 0 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L701-L719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L701-L719).
vllm_mlx.scheduler._configure_chunked_prefill · function
```python
vllm_mlx.scheduler._configure_chunked_prefill(scheduler: 'Scheduler', batch_gen: 'BatchGenerator', budget: int, prompt_cache_save) -> None
```
Enable the matching legacy or native mlx-lm chunked-prefill API.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `scheduler` | `'Scheduler'` | `yes` | `none` | Required positional or keyword input. |
| `batch_gen` | `'BatchGenerator'` | `yes` | `none` | Required positional or keyword input. |
| `budget` | `int` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache_save` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Function `_configure_chunked_prefill` calls `hasattr`, `scheduler._make_mid_prefill_save_callback`, `logger.info`, `_install_chunked_prefill`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L722-L777](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L722-L777).
vllm_mlx.scheduler._install_mtp · function
```python
vllm_mlx.scheduler._install_mtp(batch_gen: 'BatchGenerator', model: Any, num_draft_tokens: int = 1, optimistic: bool = False, stats_state: Optional['_MTPStatsState'] = None) -> None
```
Monkey-patch a BatchGenerator to use MTP (Multi-Token Prediction) with always-advance strategy for hybrid MambaCache + KVCache.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `batch_gen` | `'BatchGenerator'` | `yes` | `none` | Required positional or keyword input. |
| `model` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `num_draft_tokens` | `int` | `no` | `1` | Optional positional or keyword input; defaults to `1`. |
| `optimistic` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. |
| `stats_state` | `Optional['_MTPStatsState']` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Function `_install_mtp` calls `make_sampler`, `_MTPStatsState`, `logger.warning`, `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L780-L1262](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L780-L1262).
vllm_mlx.scheduler._install_mtp._get_mtp_stats · nested function
```python
vllm_mlx.scheduler._install_mtp._get_mtp_stats() -> Dict[str, Any]
```
Nested Function `_install_mtp._get_mtp_stats` calls `dict`; returns `{'enabled': True, 'requested_draft_tokens': num_draft_tokens, 'effective_draft_tokens': 1, 'mode': 'always_advance_opti…`.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Dict[str, Any]`
- Direct return expressions: `{'enabled': True, 'requested_draft_tokens': num_draft_tokens, 'effective_draft_tokens': 1, 'mode': 'always_advance_opti…`
**Exceptions and behavior**
Nested Function `_install_mtp._get_mtp_stats` calls `dict`; returns `{'enabled': True, 'requested_draft_tokens': num_draft_tokens, 'effective_draft_tokens': 1, 'mode': 'always_advance_opti…`.
No direct `raise` statement appears in this definition.
[View source #L823-L845](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L823-L845).
vllm_mlx.scheduler._install_mtp._mtp_bypass_reasons · nested function
```python
vllm_mlx.scheduler._install_mtp._mtp_bypass_reasons(input_tokens, prompt_cache) -> not annotated
```
Nested Function `_install_mtp._mtp_bypass_reasons` calls `reasons.append`; returns `reasons`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `input_tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `reasons`
**Exceptions and behavior**
Nested Function `_install_mtp._mtp_bypass_reasons` calls `reasons.append`; returns `reasons`.
No direct `raise` statement appears in this definition.
[View source #L849-L857](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L849-L857).
vllm_mlx.scheduler._install_mtp._record_mtp_bypass · nested function
```python
vllm_mlx.scheduler._install_mtp._record_mtp_bypass(reasons) -> None
```
Nested Function `_install_mtp._record_mtp_bypass` contains no state mutation, call, raise, return, await, or yield.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `reasons` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Nested Function `_install_mtp._record_mtp_bypass` contains no state mutation, call, raise, return, await, or yield.
No direct `raise` statement appears in this definition.
[View source #L859-L862](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L859-L862).
vllm_mlx.scheduler._install_mtp._mtp_step · nested function
```python
vllm_mlx.scheduler._install_mtp._mtp_step(input_tokens, prompt_cache, samplers, logits_processors, tokens) -> not annotated
```
Extended _step with MTP always-advance strategy.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `input_tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `samplers` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `logits_processors` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `_orig_step(input_tokens, prompt_cache, samplers, logits_processors, tokens)`; `(primary_tokens, list(logprobs))`
**Exceptions and behavior**
Nested Function `_install_mtp._mtp_step` calls `_mtp_bypass_reasons`, `_record_mtp_bypass`, `_orig_step`, `model`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L864-L1138](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L864-L1138).
vllm_mlx.scheduler._install_mtp._mtp_next · nested function
```python
vllm_mlx.scheduler._install_mtp._mtp_next() -> not annotated
```
Wrapper around _next that emits deferred MTP draft tokens.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `not annotated`
- Direct return expressions: `responses`; `augmented`
**Exceptions and behavior**
Nested Function `_install_mtp._mtp_next` updates `self.active_batch`; calls `_deferred_drafts.clear`, `_deferred_drafts.pop`, `self._inner_next`, `set`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L1147-L1247](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1147-L1247).
vllm_mlx.scheduler._mtp_status_snapshot · function
```python
vllm_mlx.scheduler._mtp_status_snapshot(batch_generator) -> Dict[str, Any]
```
Function `_mtp_status_snapshot` calls `getattr`, `callable`, `get_mtp_stats`; has 2 explicit return paths.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `batch_generator` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Dict[str, Any]`
- Direct return expressions: `{'mtp': get_mtp_stats()}`; `{}`
**Exceptions and behavior**
Function `_mtp_status_snapshot` calls `getattr`, `callable`, `get_mtp_stats`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L1265-L1269](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1265-L1269).
vllm_mlx.scheduler.Scheduler · class
```python
vllm_mlx.scheduler.Scheduler(model: Any, tokenizer: Any, config: Optional[SchedulerConfig] = None)
```
Scheduler for continuous batching using mlx-lm BatchGenerator.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The MLX model |
| `tokenizer` | `Any` | `yes` | `none` | The tokenizer |
| `config` | `Optional[SchedulerConfig]` | `no` | `None` | Scheduler configuration |
**Returns**
- Constructs: `vllm_mlx.scheduler.Scheduler`
**Exceptions and behavior**
Class `Scheduler` declares 52 direct member(s).
No direct `raise` statement appears in this definition.
[View source #L1272-L3518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1272-L3518).
vllm_mlx.scheduler.Scheduler.__init__ · method
```python
vllm_mlx.scheduler.Scheduler.__init__(model: Any, tokenizer: Any, config: Optional[SchedulerConfig] = None) -> not annotated
```
Initialize the scheduler.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `Any` | `yes` | `none` | The MLX model |
| `tokenizer` | `Any` | `yes` | `none` | The tokenizer |
| `config` | `Optional[SchedulerConfig]` | `no` | `None` | Scheduler configuration |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Method `Scheduler.__init__` updates `self.model`, `self.tokenizer`, `self.config`, `self._actual_tokenizer`; calls `SchedulerConfig`, `self._get_actual_tokenizer`, `deque`, `set`.
No direct `raise` statement appears in this definition.
[View source #L1286-L1402](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1286-L1402).
vllm_mlx.scheduler.Scheduler._get_actual_tokenizer · method
```python
vllm_mlx.scheduler.Scheduler._get_actual_tokenizer(tokenizer: Any) -> Any
```
Get the actual tokenizer from a processor or tokenizer.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tokenizer` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Any`
- Direct return expressions: `tokenizer`; `tokenizer.tokenizer`
**Exceptions and behavior**
Method `Scheduler._get_actual_tokenizer` calls `hasattr`, `callable`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L1404-L1418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1404-L1418).
vllm_mlx.scheduler.Scheduler._decode_tokens · method
```python
vllm_mlx.scheduler.Scheduler._decode_tokens(token_ids: List[int]) -> str
```
Decode token IDs to text, handling both tokenizers and processors.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `token_ids` | `List[int]` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `str`
- Direct return expressions: `self._actual_tokenizer.decode(token_ids)`
**Exceptions and behavior**
Method `Scheduler._decode_tokens` calls `self._actual_tokenizer.decode`; returns `self._actual_tokenizer.decode(token_ids)`.
No direct `raise` statement appears in this definition.
[View source #L1420-L1424](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1420-L1424).
vllm_mlx.scheduler.Scheduler._get_detokenizer · method
```python
vllm_mlx.scheduler.Scheduler._get_detokenizer(request_id: str) -> Any
```
Get or create a streaming detokenizer for a request.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Any`
- Direct return expressions: `self._detokenizer_pool[request_id]`
**Exceptions and behavior**
Method `Scheduler._get_detokenizer` calls `NaiveStreamingDetokenizer`; returns `self._detokenizer_pool[request_id]`.
No direct `raise` statement appears in this definition.
[View source #L1426-L1431](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1426-L1431).
vllm_mlx.scheduler.Scheduler._cleanup_detokenizer · method
```python
vllm_mlx.scheduler.Scheduler._cleanup_detokenizer(request_id: str) -> None
```
Remove the streaming detokenizer for a finished request.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._cleanup_detokenizer` calls `self._detokenizer_pool.pop`.
No direct `raise` statement appears in this definition.
[View source #L1433-L1435](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1433-L1435).
vllm_mlx.scheduler.Scheduler._get_stop_tokens · method
```python
vllm_mlx.scheduler.Scheduler._get_stop_tokens() -> Set[int]
```
Get stop token IDs from tokenizer or processor.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Set[int]`
- Direct return expressions: `stop_tokens`
**Exceptions and behavior**
Method `Scheduler._get_stop_tokens` calls `set`, `hasattr`, `isinstance`, `stop_tokens.update`; returns `stop_tokens`.
No direct `raise` statement appears in this definition.
[View source #L1437-L1455](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1437-L1455).
vllm_mlx.scheduler.Scheduler._create_batch_generator · method
```python
vllm_mlx.scheduler.Scheduler._create_batch_generator(sampling_params: SamplingParams) -> BatchGenerator
```
Create a BatchGenerator with the given sampling parameters.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sampling_params` | `SamplingParams` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `BatchGenerator`
- Direct return expressions: `bg`
**Exceptions and behavior**
Method `Scheduler._create_batch_generator` calls `make_sampler`, `self._get_stop_tokens`, `stop_tokens.update`, `BatchGenerator`; returns `bg`.
No direct `raise` statement appears in this definition.
[View source #L1457-L1539](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1457-L1539).
vllm_mlx.scheduler.Scheduler._create_batch_generator._prefill_progress · nested function
```python
vllm_mlx.scheduler.Scheduler._create_batch_generator._prefill_progress(progress_list) -> not annotated
```
Log prefill progress for each uid chunk.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `progress_list` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
**Exceptions and behavior**
Nested Function `Scheduler._create_batch_generator._prefill_progress` calls `self.uid_to_request_id.get`, `logger.info`, `isinstance`.
No direct `raise` statement appears in this definition.
[View source #L1472-L1479](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1472-L1479).
vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback · method
```python
vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback() -> not annotated
```
Create a callback that stores prompt-only KV/Mamba cache.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `not annotated`
- Direct return expressions: `_prompt_cache_save`
**Exceptions and behavior**
Method `Scheduler._make_prompt_cache_save_callback` returns `_prompt_cache_save`.
No direct `raise` statement appears in this definition.
[View source #L1541-L1585](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1541-L1585).
vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback._prompt_cache_save · nested function
```python
vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback._prompt_cache_save(uid, extracted_cache) -> not annotated
```
Nested Function `Scheduler._make_prompt_cache_save_callback._prompt_cache_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `list`, `_trim_cache_offset`; returns `None`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `uid` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `extracted_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `None`
**Exceptions and behavior**
Nested Function `Scheduler._make_prompt_cache_save_callback._prompt_cache_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `list`, `_trim_cache_offset`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L1554-L1583](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1554-L1583).
vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback · method
```python
vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback(save_interval: int) -> not annotated
```
Create a callback for saving intermediate KV cache during chunked prefill.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `save_interval` | `int` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `_mid_prefill_save`
**Exceptions and behavior**
Method `Scheduler._make_mid_prefill_save_callback` returns `_mid_prefill_save`.
No direct `raise` statement appears in this definition.
[View source #L1587-L1655](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1587-L1655).
vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback._mid_prefill_save · nested function
```python
vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback._mid_prefill_save(uid, processed_tokens, prompt_cache) -> not annotated
```
Nested Function `Scheduler._make_mid_prefill_save_callback._mid_prefill_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `getattr`, `self._extract_cache_states`; returns `None`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `uid` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `processed_tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_cache` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `None`
**Exceptions and behavior**
Nested Function `Scheduler._make_mid_prefill_save_callback._mid_prefill_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `getattr`, `self._extract_cache_states`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L1598-L1653](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1598-L1653).
vllm_mlx.scheduler.Scheduler._close_batch_generator · method
```python
vllm_mlx.scheduler.Scheduler._close_batch_generator() -> None
```
Properly close BatchGenerator to restore wired_limit.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._close_batch_generator` updates `self.batch_generator`; calls `hasattr`, `self.batch_generator.close`, `logger.debug`.
No direct `raise` statement appears in this definition.
[View source #L1657-L1665](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1657-L1665).
vllm_mlx.scheduler.Scheduler._ensure_batch_generator · method
```python
vllm_mlx.scheduler.Scheduler._ensure_batch_generator(sampling_params: SamplingParams) -> None
```
Ensure BatchGenerator exists with compatible settings.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sampling_params` | `SamplingParams` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Method `Scheduler._ensure_batch_generator` updates `self.batch_generator`, `self._current_sampler_params`; calls `logger.warning`, `len`, `hasattr`, `logger.info`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L1667-L1709](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1667-L1709).
vllm_mlx.scheduler.Scheduler._validate_cache · method
```python
vllm_mlx.scheduler.Scheduler._validate_cache(cache: Any) -> bool
```
Validate that a cache object is usable.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache` | `Any` | `yes` | `none` | The cache object to validate |
**Returns**
- Type: `bool`
- Direct return expressions: `False`; `True`
**Exceptions and behavior**
Method `Scheduler._validate_cache` calls `isinstance`, `len`, `hasattr`, `logger.debug`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L1711-L1769](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1711-L1769).
vllm_mlx.scheduler.Scheduler._reconstruct_cache_from_states · method
```python
vllm_mlx.scheduler.Scheduler._reconstruct_cache_from_states(extracted_states: List[Dict[str, Any]]) -> Optional[List[Any]]
```
Reconstruct cache objects from extracted cache states.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `extracted_states` | `List[Dict[str, Any]]` | `yes` | `none` | List of dicts from _extract_cache_states() |
**Returns**
- Type: `Optional[List[Any]]`
- Direct return expressions: `None`; `caches`
**Exceptions and behavior**
Method `Scheduler._reconstruct_cache_from_states` calls `layer_state.get`, `hasattr`, `_KVCache`, `cache_cls.from_state`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L1808-L1872](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1808-L1872).
vllm_mlx.scheduler.Scheduler.add_request · method
```python
vllm_mlx.scheduler.Scheduler.add_request(request: Request) -> None
```
Add a new request to the scheduler.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | The request to add |
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler.add_request` calls `ValueError`, `isinstance`, `hasattr`, `self.tokenizer.encode`; can raise `ValueError`, `AttributeError`.
Directly raised exceptions: `ValueError`, `AttributeError`.
[View source #L1874-L1997](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1874-L1997).
vllm_mlx.scheduler.Scheduler.abort_request · method
```python
vllm_mlx.scheduler.Scheduler.abort_request(request_id: str) -> bool
```
Queue request for abort.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | The request ID to abort |
**Returns**
- Type: `bool`
- Direct return expressions: `True`
**Exceptions and behavior**
Method `Scheduler.abort_request` calls `self._pending_abort_ids.add`, `logger.info`; returns `True`.
No direct `raise` statement appears in this definition.
[View source #L1999-L2014](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1999-L2014).
vllm_mlx.scheduler.Scheduler._process_pending_aborts · method
```python
vllm_mlx.scheduler.Scheduler._process_pending_aborts() -> None
```
Drain and process pending abort requests.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._process_pending_aborts` calls `self._pending_abort_ids.pop`, `self._do_abort_request`.
No direct `raise` statement appears in this definition.
[View source #L2016-L2020](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2016-L2020).
vllm_mlx.scheduler.Scheduler._do_abort_request · method
```python
vllm_mlx.scheduler.Scheduler._do_abort_request(request_id: str) -> bool
```
Actually abort a request.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | The request ID to abort |
**Returns**
- Type: `bool`
- Direct return expressions: `True`
**Exceptions and behavior**
Method `Scheduler._do_abort_request` updates `self.total_completion_tokens`, `self.total_prompt_tokens`; calls `self.requests.get`, `self.waiting.remove`, `self.batch_generator.remove`, `request.set_finished`; returns `True`.
No direct `raise` statement appears in this definition.
[View source #L2022-L2087](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2022-L2087).
vllm_mlx.scheduler.Scheduler.has_requests · method
```python
vllm_mlx.scheduler.Scheduler.has_requests() -> bool
```
Check if there are any pending or running requests.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `bool`
- Direct return expressions: `bool(self.waiting or self.running)`
**Exceptions and behavior**
Method `Scheduler.has_requests` calls `bool`; returns `bool(self.waiting or self.running)`.
No direct `raise` statement appears in this definition.
[View source #L2089-L2091](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2089-L2091).
vllm_mlx.scheduler.Scheduler.get_num_waiting · method
```python
vllm_mlx.scheduler.Scheduler.get_num_waiting() -> int
```
Get number of waiting requests.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `int`
- Direct return expressions: `len(self.waiting)`
**Exceptions and behavior**
Method `Scheduler.get_num_waiting` calls `len`; returns `len(self.waiting)`.
No direct `raise` statement appears in this definition.
[View source #L2093-L2095](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2093-L2095).
vllm_mlx.scheduler.Scheduler.get_num_running · method
```python
vllm_mlx.scheduler.Scheduler.get_num_running() -> int
```
Get number of running requests.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `int`
- Direct return expressions: `len(self.running)`
**Exceptions and behavior**
Method `Scheduler.get_num_running` calls `len`; returns `len(self.running)`.
No direct `raise` statement appears in this definition.
[View source #L2097-L2099](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2097-L2099).
vllm_mlx.scheduler.Scheduler._schedule_waiting · method
```python
vllm_mlx.scheduler.Scheduler._schedule_waiting() -> List[Request]
```
Move requests from waiting queue to running.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `List[Request]`
- Direct return expressions: `scheduled`
**Exceptions and behavior**
Method `Scheduler._schedule_waiting` updates `self.total_prompt_tokens`; calls `self._try_promote_ssd_pending`, `len`, `self.waiting.popleft`, `self._ensure_batch_generator`; returns `scheduled`.
No direct `raise` statement appears in this definition.
[View source #L2101-L2276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2101-L2276).
vllm_mlx.scheduler.Scheduler._copy_cache_state · method
```python
vllm_mlx.scheduler.Scheduler._copy_cache_state(value: Any) -> Any
```
Deep-copy a cache ``state`` payload.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `value` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Any`
- Direct return expressions: `value + 0`; `type(value)(copied) if isinstance(value, tuple) else copied`; `value`
**Exceptions and behavior**
Method `Scheduler._copy_cache_state` calls `isinstance`, `Scheduler._copy_cache_state`, `type(value)`, `type`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2279-L2295](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2279-L2295).
vllm_mlx.scheduler.Scheduler._prompt_output_entry_is_useless · method
```python
vllm_mlx.scheduler.Scheduler._prompt_output_entry_is_useless(cache: Any) -> bool
```
Would a prompt+output entry built from this cache ever be reusable?
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `not can_trim_prompt_cache(cache)`; `False`
**Exceptions and behavior**
Method `Scheduler._prompt_output_entry_is_useless` calls `can_trim_prompt_cache`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2303-L2317](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2303-L2317).
vllm_mlx.scheduler.Scheduler._make_snapshot_destination · method
```python
vllm_mlx.scheduler.Scheduler._make_snapshot_destination(live_cache: Any) -> Any
```
Build a destination cache with the same topology as the live one.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `live_cache` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Any`
- Direct return expressions: `[_mirror(layer) for layer in live_cache]`; `None`
**Exceptions and behavior**
Method `Scheduler._make_snapshot_destination` calls `_mirror`, `logger.warning`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2338-L2380](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2338-L2380).
vllm_mlx.scheduler.Scheduler._make_snapshot_destination._mirror · nested function
```python
vllm_mlx.scheduler.Scheduler._make_snapshot_destination._mirror(layer: Any) -> Any
```
Nested Function `Scheduler._make_snapshot_destination._mirror` calls `getattr`, `_mirror`, `copy.copy`, `type(children)`; has 2 explicit return paths.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `layer` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Any`
- Direct return expressions: `container`; `copy.copy(layer)`
**Exceptions and behavior**
Nested Function `Scheduler._make_snapshot_destination._mirror` calls `getattr`, `_mirror`, `copy.copy`, `type(children)`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2360-L2370](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2360-L2370).
vllm_mlx.scheduler.Scheduler._cache_coverage · method
```python
vllm_mlx.scheduler.Scheduler._cache_coverage(cache: Any) -> int | None
```
How many tokens the live cache actually holds.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `int | None`
- Direct return expressions: `found`; `None`
**Exceptions and behavior**
Method `Scheduler._cache_coverage` calls `_offset_of`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2383-L2409](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2383-L2409).
vllm_mlx.scheduler.Scheduler._cache_coverage._offset_of · nested function
```python
vllm_mlx.scheduler.Scheduler._cache_coverage._offset_of(layer: Any) -> int | None
```
Nested Function `Scheduler._cache_coverage._offset_of` calls `getattr`, `isinstance`, `_offset_of`; has 3 explicit return paths.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `layer` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `int | None`
- Direct return expressions: `offset`; `found`; `None`
**Exceptions and behavior**
Nested Function `Scheduler._cache_coverage._offset_of` calls `getattr`, `isinstance`, `_offset_of`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2393-L2403](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2393-L2403).
vllm_mlx.scheduler.Scheduler._cache_key_for_snapshot · method
```python
vllm_mlx.scheduler.Scheduler._cache_key_for_snapshot(request: Any, response: Any, raw_cache: Any) -> list[int] | None
```
Key the entry by the tokens the cache covers, not by the prompt.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `response` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `raw_cache` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `list[int] | None`
- Direct return expressions: `None`; `prompt_ids`; `prompt_ids + generated[:overshoot]`
**Exceptions and behavior**
Method `Scheduler._cache_key_for_snapshot` calls `self._cache_coverage`, `list`, `logger.debug`, `', '.join`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L2411-L2468](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2411-L2468).
vllm_mlx.scheduler.Scheduler._store_prompt_only_cache · method
```python
vllm_mlx.scheduler.Scheduler._store_prompt_only_cache(request: Any, response: Any) -> None
```
Store the post-prefill cache under the prompt tokens alone.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Any` | `yes` | `none` | Required positional or keyword input. |
| `response` | `Any` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Method `Scheduler._store_prompt_only_cache` calls `getattr`, `len`, `callable`, `raw_cache`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L2470-L2581](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2470-L2581).
vllm_mlx.scheduler.Scheduler._process_batch_responses · method
```python
vllm_mlx.scheduler.Scheduler._process_batch_responses(responses: List[Any]) -> Tuple[List[RequestOutput], Set[str]]
```
Process responses from BatchGenerator.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `responses` | `List[Any]` | `yes` | `none` | List of BatchGenerator.Response objects |
**Returns**
- Type: `Tuple[List[RequestOutput], Set[str]]`
- Direct return expressions: `(outputs, finished_ids)`
**Exceptions and behavior**
Method `Scheduler._process_batch_responses` updates `self.total_completion_tokens`, `self.num_requests_processed`; calls `set`, `self.uid_to_request_id.get`, `self.running.get`, `self._store_prompt_only_cache`; returns `(outputs, finished_ids)`.
No direct `raise` statement appears in this definition.
[View source #L2583-L2710](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2583-L2710).
vllm_mlx.scheduler.Scheduler._cleanup_finished · method
```python
vllm_mlx.scheduler.Scheduler._cleanup_finished(finished_ids: Set[str]) -> None
```
Clean up finished requests and store caches for reuse.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `finished_ids` | `Set[str]` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._cleanup_finished` calls `self.running.get`, `hasattr`, `list`, `self.block_aware_cache.store_cache`.
No direct `raise` statement appears in this definition.
[View source #L2712-L2865](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2712-L2865).
vllm_mlx.scheduler.Scheduler._is_cache_corruption_error · method
```python
vllm_mlx.scheduler.Scheduler._is_cache_corruption_error(error: Exception) -> bool
```
Check if an error indicates cache corruption.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `error` | `Exception` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `any((pattern in error_str for pattern in CACHE_CORRUPTION_PATTERNS))`
**Exceptions and behavior**
Method `Scheduler._is_cache_corruption_error` calls `str`, `any`; returns `any((pattern in error_str for pattern in CACHE_CORRUPTION_PATTERNS))`.
No direct `raise` statement appears in this definition.
[View source #L2867-L2870](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2867-L2870).
vllm_mlx.scheduler.Scheduler._is_stream_thread_error · method
```python
vllm_mlx.scheduler.Scheduler._is_stream_thread_error(error: Exception) -> bool
```
Check if an error indicates MLX stream/thread ownership mismatch.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `error` | `Exception` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `'no Stream(' in error_str or 'no Stream(gpu' in error_str`
**Exceptions and behavior**
Method `Scheduler._is_stream_thread_error` calls `str`; returns `'no Stream(' in error_str or 'no Stream(gpu' in error_str`.
No direct `raise` statement appears in this definition.
[View source #L2872-L2875](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2872-L2875).
vllm_mlx.scheduler.Scheduler._recover_from_cache_error · method
```python
vllm_mlx.scheduler.Scheduler._recover_from_cache_error() -> None
```
Recover from cache corruption error.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._recover_from_cache_error` updates `self._current_sampler_params`; calls `self._close_batch_generator`, `self.block_aware_cache.clear`, `self.memory_aware_cache.clear`, `self.prefix_cache.clear`.
No direct `raise` statement appears in this definition.
[View source #L2877-L2895](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2877-L2895).
vllm_mlx.scheduler.Scheduler._recover_from_generation_error · method
```python
vllm_mlx.scheduler.Scheduler._recover_from_generation_error() -> Set[str]
```
Recover from fatal generation error (OOM, Metal crash).
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Set[str]`
- Direct return expressions: `aborted_ids`
**Exceptions and behavior**
Method `Scheduler._recover_from_generation_error` updates `self._current_sampler_params`; calls `self._close_batch_generator`, `set`, `list`, `self.running.get`; returns `aborted_ids`.
No direct `raise` statement appears in this definition.
[View source #L2897-L2933](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2897-L2933).
vllm_mlx.scheduler.Scheduler._reschedule_running_requests · method
```python
vllm_mlx.scheduler.Scheduler._reschedule_running_requests() -> None
```
Move running requests back to waiting queue for retry.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._reschedule_running_requests` calls `len`, `list`, `self.running.items`, `self.waiting.appendleft`.
No direct `raise` statement appears in this definition.
[View source #L2935-L2951](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2935-L2951).
vllm_mlx.scheduler.Scheduler.step · method
```python
vllm_mlx.scheduler.Scheduler.step(max_retries: int = 1) -> SchedulerOutput
```
Execute one scheduling step with automatic error recovery.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `max_retries` | `int` | `no` | `1` | Number of times to retry on cache errors (default 1) |
**Returns**
- Type: `SchedulerOutput`
- Direct return expressions: `output`
**Exceptions and behavior**
Method `Scheduler.step` updates `self.finished_req_ids`, `self._step_count`; calls `SchedulerOutput`, `self._process_pending_aborts`, `range`, `self._schedule_waiting`; returns `output`.
No direct `raise` statement appears in this definition.
[View source #L2953-L3089](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2953-L3089).
vllm_mlx.scheduler.Scheduler.get_request · method
```python
vllm_mlx.scheduler.Scheduler.get_request(request_id: str) -> Optional[Request]
```
Get a request by ID.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Optional[Request]`
- Direct return expressions: `self.requests.get(request_id)`
**Exceptions and behavior**
Method `Scheduler.get_request` calls `self.requests.get`; returns `self.requests.get(request_id)`.
No direct `raise` statement appears in this definition.
[View source #L3091-L3093](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3091-L3093).
vllm_mlx.scheduler.Scheduler.remove_finished_request · method
```python
vllm_mlx.scheduler.Scheduler.remove_finished_request(request_id: str) -> Optional[Request]
```
Remove a finished request from tracking.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `Optional[Request]`
- Direct return expressions: `self.requests.pop(request_id, None)`
**Exceptions and behavior**
Method `Scheduler.remove_finished_request` calls `self.requests.pop`; returns `self.requests.pop(request_id, None)`.
No direct `raise` statement appears in this definition.
[View source #L3095-L3097](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3095-L3097).
vllm_mlx.scheduler.Scheduler.get_running_requests_info · method
```python
vllm_mlx.scheduler.Scheduler.get_running_requests_info() -> List[Dict[str, Any]]
```
Per-request details for status endpoint.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `List[Dict[str, Any]]`
- Direct return expressions: `result`
**Exceptions and behavior**
Method `Scheduler.get_running_requests_info` calls `_time.time`, `result.append`, `round`, `self.running.values`; returns `result`.
No direct `raise` statement appears in this definition.
[View source #L3099-L3165](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3099-L3165).
vllm_mlx.scheduler.Scheduler.get_stats · method
```python
vllm_mlx.scheduler.Scheduler.get_stats() -> Dict[str, Any]
```
Get scheduler statistics.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Dict[str, Any]`
- Direct return expressions: `stats`
**Exceptions and behavior**
Method `Scheduler.get_stats` calls `len`, `stats.update`, `_mtp_status_snapshot`, `mx.metal.is_available`; returns `stats`.
No direct `raise` statement appears in this definition.
[View source #L3167-L3193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3167-L3193).
vllm_mlx.scheduler.Scheduler.get_cache_stats · method
```python
vllm_mlx.scheduler.Scheduler.get_cache_stats() -> Optional[Dict[str, Any]]
```
Get cache statistics.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Optional[Dict[str, Any]]`
- Direct return expressions: `self.block_aware_cache.get_stats()`; `self.memory_aware_cache.get_stats()`; `self.prefix_cache.get_stats()`; `None`
**Exceptions and behavior**
Method `Scheduler.get_cache_stats` calls `self.block_aware_cache.get_stats`, `self.memory_aware_cache.get_stats`, `self.prefix_cache.get_stats`; has 4 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3195-L3203](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3195-L3203).
vllm_mlx.scheduler.Scheduler.clear_runtime_caches · method
```python
vllm_mlx.scheduler.Scheduler.clear_runtime_caches() -> Dict[str, bool]
```
Clear prefix-cache state without resetting scheduler/request state.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `Dict[str, bool]`
- Direct return expressions: `cleared`
**Exceptions and behavior**
Method `Scheduler.clear_runtime_caches` calls `self.block_aware_cache.clear`, `self.memory_aware_cache.clear`, `self.prefix_cache.clear`; returns `cleared`.
No direct `raise` statement appears in this definition.
[View source #L3205-L3221](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3205-L3221).
vllm_mlx.scheduler.Scheduler.reset · method
```python
vllm_mlx.scheduler.Scheduler.reset() -> None
```
Reset the scheduler state.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler.reset` updates `self._current_sampler_params`; calls `self._pending_abort_ids.clear`, `list`, `self.requests.keys`, `self._do_abort_request`.
No direct `raise` statement appears in this definition.
[View source #L3223-L3246](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3223-L3246).
vllm_mlx.scheduler.Scheduler.deep_reset · method
```python
vllm_mlx.scheduler.Scheduler.deep_reset() -> None
```
Deep reset that clears ALL cache state including model-level caches.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler.deep_reset` updates `self.model.cache`; calls `self.reset`, `hasattr`, `gc.collect`, `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L3248-L3276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3248-L3276).
vllm_mlx.scheduler.Scheduler.save_cache_to_disk · method
```python
vllm_mlx.scheduler.Scheduler.save_cache_to_disk(cache_dir: str) -> bool
```
Save prefix cache to disk for persistence across restarts.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `self.memory_aware_cache.save_to_disk(cache_dir)`; `False`
**Exceptions and behavior**
Method `Scheduler.save_cache_to_disk` calls `self.memory_aware_cache.save_to_disk`, `logger.info`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3282-L3287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3282-L3287).
vllm_mlx.scheduler.Scheduler.load_cache_from_disk · method
```python
vllm_mlx.scheduler.Scheduler.load_cache_from_disk(cache_dir: str) -> int
```
Load prefix cache from disk.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `cache_dir` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `int`
- Direct return expressions: `self.memory_aware_cache.load_from_disk(cache_dir)`; `0`
**Exceptions and behavior**
Method `Scheduler.load_cache_from_disk` calls `self.memory_aware_cache.load_from_disk`, `logger.info`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3289-L3294](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3289-L3294).
vllm_mlx.scheduler.Scheduler.clear_prefix_cache · method
```python
vllm_mlx.scheduler.Scheduler.clear_prefix_cache() -> None
```
Clear the in-memory prefix cache (keeps disk cache untouched).
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
- Direct return expressions: `None`
**Exceptions and behavior**
Method `Scheduler.clear_prefix_cache` calls `hasattr`, `self.memory_aware_cache.clear`, `logger.info`, `self.prefix_cache.clear`; returns `None`.
No direct `raise` statement appears in this definition.
[View source #L3296-L3306](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3296-L3306).
vllm_mlx.scheduler.Scheduler.close_ssd_tier · method
```python
vllm_mlx.scheduler.Scheduler.close_ssd_tier() -> None
```
Shut down the SSD cache tier if present.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler.close_ssd_tier` updates `self._ssd_tier`; calls `self._ssd_tier.close`, `logger.info`.
No direct `raise` statement appears in this definition.
[View source #L3308-L3313](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3308-L3313).
vllm_mlx.scheduler.Scheduler._try_promote_ssd_pending · method
```python
vllm_mlx.scheduler.Scheduler._try_promote_ssd_pending() -> None
```
Attempt synchronous SSD promotion for waiting requests tagged ssd_pending.
**Parameters**
This callable has no explicit inputs.
**Returns**
- Type: `None`
**Exceptions and behavior**
Method `Scheduler._try_promote_ssd_pending` updates `self._ssd_tier._stats.promotion_failures`, `self._ssd_tier._stats.ssd_hits`; calls `getattr`, `self.memory_aware_cache.try_reserve_memory`, `logger.info`, `tuple`.
No direct `raise` statement appears in this definition.
[View source #L3315-L3395](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3315-L3395).
vllm_mlx.scheduler.Scheduler.promote_from_ssd · method
```python
async vllm_mlx.scheduler.Scheduler.promote_from_ssd(request) -> bool
```
Promote a cold-tier cache entry for a request (async version).
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `False`; `True`
**Exceptions and behavior**
Method `Scheduler.promote_from_ssd` calls `getattr`, `candidate.get`, `len`, `tuple`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3397-L3460](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3397-L3460).
vllm_mlx.scheduler.Scheduler.promote_from_ssd.reserve_budget · nested function
```python
vllm_mlx.scheduler.Scheduler.promote_from_ssd.reserve_budget(nbytes: int) -> bool
```
Tentatively reserve RAM budget for promotion.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `nbytes` | `int` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `bool`
- Direct return expressions: `False`; `self.memory_aware_cache.try_reserve_memory(nbytes)`
**Exceptions and behavior**
Nested Function `Scheduler.promote_from_ssd.reserve_budget` calls `self.memory_aware_cache.try_reserve_memory`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3412-L3416](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3412-L3416).
vllm_mlx.scheduler.Scheduler.promote_from_ssd.release_budget · nested function
```python
vllm_mlx.scheduler.Scheduler.promote_from_ssd.release_budget(nbytes: int) -> None
```
Release tentatively reserved budget on failure.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `nbytes` | `int` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `None`
**Exceptions and behavior**
Nested Function `Scheduler.promote_from_ssd.release_budget` calls `self.memory_aware_cache.release_reserved_memory`.
No direct `raise` statement appears in this definition.
[View source #L3418-L3421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3418-L3421).
vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers · method
```python
vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers(layer_dicts: list[dict]) -> list | None
```
Reconstruct cache objects from deserialized layer dicts.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `layer_dicts` | `list[dict]` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `list | None`
- Direct return expressions: `None`; `result`
**Exceptions and behavior**
Method `Scheduler._reconstruct_ssd_layers` calls `KVCache`, `mx.array`, `ld.get`, `_mx_dtype_from_name`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.
[View source #L3462-L3518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3462-L3518).
vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers._mx_dtype_from_name · nested function
```python
vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers._mx_dtype_from_name(name: str) -> not annotated
```
Nested Function `Scheduler._reconstruct_ssd_layers._mx_dtype_from_name` calls `getattr`; returns `getattr(mx, name, None)`.
**Parameters**
| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `name` | `str` | `yes` | `none` | Required positional or keyword input. |
**Returns**
- Type: `not annotated`
- Direct return expressions: `getattr(mx, name, None)`
**Exceptions and behavior**
Nested Function `Scheduler._reconstruct_ssd_layers._mx_dtype_from_name` calls `getattr`; returns `getattr(mx, name, None)`.
No direct `raise` statement appears in this definition.
[View source #L3473-L3474](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3473-L3474).
## Complete symbol map
This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.
| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`_normalize_logits_processors`](#contract-vllm_mlx.scheduler._normalize_logits_processors) | function | `_normalize_logits_processors(logits_processors) -> not annotated` | Normalize empty per-sequence processor slots to lists. | [#L46-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L46-L50) |
| [`_sanitize_batch_generator_logits_processors`](#contract-vllm_mlx.scheduler._sanitize_batch_generator_logits_processors) | function | `_sanitize_batch_generator_logits_processors(batch_generator) -> None` | Sanitize stale BatchGenerator processor state before decode. | [#L53-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L53-L65) |
| [`SchedulingPolicy`](#contract-vllm_mlx.scheduler.SchedulingPolicy) | class | `SchedulingPolicy()` | Scheduling policy for request ordering. | [#L68-L72](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L68-L72) |
| [`SchedulerConfig`](#contract-vllm_mlx.scheduler.SchedulerConfig) | class | `SchedulerConfig(max_num_seqs: int = 256, max_num_batched_tokens: int = 8192, policy: SchedulingPolicy = SchedulingPolicy.FCFS, prefill_batch_size: int = 8, completion_batch_size: int = 32, prefill_step_size: int = 2048, mllm_prefill_step_size: Optional[int] = None, enable_prefix_cache: bool = True, prefix_cache_size: int = 100, use_memory_aware_cache: bool = True, cache_memory_mb: Optional[int] = None, cache_memory_percent: float = 0.2, kv_cache_quantization: bool = False, kv_cache_quantization_bits: int = 8, kv_cache_quantization_group_size: int = 64, kv_cache_min_quantize_tokens: int = 256, use_paged_cache: bool = False, paged_cache_block_size: int = 64, max_cache_blocks: int = 1000, chunked_prefill_tokens: int = 0, mid_prefill_save_interval: int = 8192, ssd_cache_dir: Optional[str] = None, ssd_cache_max_gb: float = 10.0, max_kv_size: int = 0, enable_mtp: bool = False, mtp_num_draft_tokens: int = 1, mtp_optimistic: bool = False)` | Configuration for the scheduler. | [#L76-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L76-L140) |
| [`SchedulerConfig.__post_init__`](#contract-vllm_mlx.scheduler.SchedulerConfig.__post_init__) | method | `SchedulerConfig.__post_init__() -> None` | Method `SchedulerConfig.__post_init__` calls `ValueError`; can raise `ValueError`. | [#L138-L140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L138-L140) |
| [`SchedulerOutput`](#contract-vllm_mlx.scheduler.SchedulerOutput) | class | `SchedulerOutput(scheduled_request_ids: List[str] = field(default_factory=list), num_scheduled_tokens: int = 0, finished_request_ids: Set[str] = field(default_factory=set), outputs: List[RequestOutput] = field(default_factory=list), has_work: bool = False)` | Output from a scheduling step. | [#L144-L160](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L144-L160) |
| [`_install_prompt_cache_save`](#contract-vllm_mlx.scheduler._install_prompt_cache_save) | function | `_install_prompt_cache_save(batch_gen: 'BatchGenerator', prompt_cache_save) -> None` | Monkey-patch ``_process_prompts`` to capture prompt-only cache state. | [#L163-L187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L163-L187) |
| [`_install_prompt_cache_save._patched_process_prompts`](#contract-vllm_mlx.scheduler._install_prompt_cache_save._patched_process_prompts) | nested function | `_install_prompt_cache_save._patched_process_prompts(prompts, _self = batch_gen) -> not annotated` | Nested Function `_install_prompt_cache_save._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`. | [#L177-L185](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L177-L185) |
| [`_install_chunked_prefill`](#contract-vllm_mlx.scheduler._install_chunked_prefill) | function | `_install_chunked_prefill(batch_gen: 'BatchGenerator', budget: int, mid_prefill_save = None, prompt_cache_save = None, pending_abort_ids: Optional[Set[str]] = None, uid_to_request_id: Optional[Dict[int, str]] = None, requests: Optional[Dict[str, Any]] = None) -> None` | Monkey-patch a BatchGenerator instance so that large prefills are broken into chunks of at most *budget* tokens each. | [#L190-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L190-L697) |
| [`_install_chunked_prefill._lazy_extract_cache`](#contract-vllm_mlx.scheduler._install_chunked_prefill._lazy_extract_cache) | nested function | `_install_chunked_prefill._lazy_extract_cache(cache, idx) -> not annotated` | Nested Function `_install_chunked_prefill._lazy_extract_cache` calls `c.extract`; returns `(c.extract(idx) for c in cache)`. | [#L225-L226](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L225-L226) |
| [`_install_chunked_prefill._batch_cls`](#contract-vllm_mlx.scheduler._install_chunked_prefill._batch_cls) | nested class | `_install_chunked_prefill._batch_cls(uids: List[int], y: Any, logprobs: List[Any], max_tokens: List[int], num_tokens: List[int], cache: List[Any], samplers: List[Any], logits_processors: List[Any], tokens: List[Any])` | Nested Class `_install_chunked_prefill._batch_cls` declares 4 direct member(s). | [#L233-L273](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L233-L273) |
| [`_install_chunked_prefill._batch_cls.__len__`](#contract-vllm_mlx.scheduler._install_chunked_prefill._batch_cls.__len__) | nested function | `_install_chunked_prefill._batch_cls.__len__() -> not annotated` | Nested Function `_install_chunked_prefill._batch_cls.__len__` calls `len`; returns `len(self.uids)`. | [#L244-L245](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L244-L245) |
| [`_install_chunked_prefill._batch_cls.filter`](#contract-vllm_mlx.scheduler._install_chunked_prefill._batch_cls.filter) | nested function | `_install_chunked_prefill._batch_cls.filter(keep_idx: List[int]) -> not annotated` | Nested Function `_install_chunked_prefill._batch_cls.filter` updates `self.uids`, `self.logprobs`, `self.max_tokens`, `self.num_tokens`; calls `mx.array`, `c.filter`. | [#L247-L258](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L247-L258) |
| [`_install_chunked_prefill._batch_cls.extend`](#contract-vllm_mlx.scheduler._install_chunked_prefill._batch_cls.extend) | nested function | `_install_chunked_prefill._batch_cls.extend(other) -> not annotated` | Nested Function `_install_chunked_prefill._batch_cls.extend` updates `self.y`; calls `self.uids.extend`, `mx.concatenate`, `self.logprobs.extend`, `self.num_tokens.extend`. | [#L260-L270](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L260-L270) |
| [`_install_chunked_prefill._batch_cls.extract_cache`](#contract-vllm_mlx.scheduler._install_chunked_prefill._batch_cls.extract_cache) | nested function | `_install_chunked_prefill._batch_cls.extract_cache(idx) -> not annotated` | Nested Function `_install_chunked_prefill._batch_cls.extract_cache` calls `c.extract`; returns `[c.extract(idx) for c in self.cache]`. | [#L272-L273](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L272-L273) |
| [`_install_chunked_prefill._patched_process_prompts`](#contract-vllm_mlx.scheduler._install_chunked_prefill._patched_process_prompts) | nested function | `_install_chunked_prefill._patched_process_prompts(prompts, _self = batch_gen) -> not annotated` | Nested Function `_install_chunked_prefill._patched_process_prompts` calls `_orig_process_prompts`, `enumerate`, `prompt_cache_save`, `batch.extract_cache`; returns `batch`. | [#L291-L299](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L291-L299) |
| [`_install_chunked_prefill._generation_step`](#contract-vllm_mlx.scheduler._install_chunked_prefill._generation_step) | nested function | `_install_chunked_prefill._generation_step() -> not annotated` | Run one generation step on the active batch. | [#L303-L360](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L303-L360) |
| [`_install_chunked_prefill._chunked_next`](#contract-vllm_mlx.scheduler._install_chunked_prefill._chunked_next) | nested function | `_install_chunked_prefill._chunked_next() -> not annotated` | Replacement for _next() that chunks large prefills. | [#L362-L678](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L362-L678) |
| [`_install_chunked_prefill._patched_remove`](#contract-vllm_mlx.scheduler._install_chunked_prefill._patched_remove) | nested function | `_install_chunked_prefill._patched_remove(uids_to_remove, _self = batch_gen) -> not annotated` | Clear partial state if aborted request is being prefilled. | [#L680-L691](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L680-L691) |
| [`_MTPStatsState`](#contract-vllm_mlx.scheduler._MTPStatsState) | class | `_MTPStatsState(counters: Dict[str, int] = field(default_factory=lambda: {'attempted': 0, 'accepted': 0, 'rejected': 0, 'errors': 0}), bypass_counts: Dict[str, int] = field(default_factory=lambda: {'prefill': 0, 'no_active_batch': 0, 'cache_mismatch': 0}), lock: Any = field(default_factory=Lock))` | Cumulative native-MTP counters shared across generator instances. | [#L701-L719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L701-L719) |
| [`_configure_chunked_prefill`](#contract-vllm_mlx.scheduler._configure_chunked_prefill) | function | `_configure_chunked_prefill(scheduler: 'Scheduler', batch_gen: 'BatchGenerator', budget: int, prompt_cache_save) -> None` | Enable the matching legacy or native mlx-lm chunked-prefill API. | [#L722-L777](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L722-L777) |
| [`_install_mtp`](#contract-vllm_mlx.scheduler._install_mtp) | function | `_install_mtp(batch_gen: 'BatchGenerator', model: Any, num_draft_tokens: int = 1, optimistic: bool = False, stats_state: Optional['_MTPStatsState'] = None) -> None` | Monkey-patch a BatchGenerator to use MTP (Multi-Token Prediction) with always-advance strategy for hybrid MambaCache + KVCache. | [#L780-L1262](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L780-L1262) |
| [`_install_mtp._get_mtp_stats`](#contract-vllm_mlx.scheduler._install_mtp._get_mtp_stats) | nested function | `_install_mtp._get_mtp_stats() -> Dict[str, Any]` | Nested Function `_install_mtp._get_mtp_stats` calls `dict`; returns `{'enabled': True, 'requested_draft_tokens': num_draft_tokens, 'effective_draft_tokens': 1, 'mode': 'always_advance_opti…`. | [#L823-L845](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L823-L845) |
| [`_install_mtp._mtp_bypass_reasons`](#contract-vllm_mlx.scheduler._install_mtp._mtp_bypass_reasons) | nested function | `_install_mtp._mtp_bypass_reasons(input_tokens, prompt_cache) -> not annotated` | Nested Function `_install_mtp._mtp_bypass_reasons` calls `reasons.append`; returns `reasons`. | [#L849-L857](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L849-L857) |
| [`_install_mtp._record_mtp_bypass`](#contract-vllm_mlx.scheduler._install_mtp._record_mtp_bypass) | nested function | `_install_mtp._record_mtp_bypass(reasons) -> None` | Nested Function `_install_mtp._record_mtp_bypass` contains no state mutation, call, raise, return, await, or yield. | [#L859-L862](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L859-L862) |
| [`_install_mtp._mtp_step`](#contract-vllm_mlx.scheduler._install_mtp._mtp_step) | nested function | `_install_mtp._mtp_step(input_tokens, prompt_cache, samplers, logits_processors, tokens) -> not annotated` | Extended _step with MTP always-advance strategy. | [#L864-L1138](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L864-L1138) |
| [`_install_mtp._mtp_next`](#contract-vllm_mlx.scheduler._install_mtp._mtp_next) | nested function | `_install_mtp._mtp_next() -> not annotated` | Wrapper around _next that emits deferred MTP draft tokens. | [#L1147-L1247](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1147-L1247) |
| [`_mtp_status_snapshot`](#contract-vllm_mlx.scheduler._mtp_status_snapshot) | function | `_mtp_status_snapshot(batch_generator) -> Dict[str, Any]` | Function `_mtp_status_snapshot` calls `getattr`, `callable`, `get_mtp_stats`; has 2 explicit return paths. | [#L1265-L1269](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1265-L1269) |
| [`Scheduler`](#contract-vllm_mlx.scheduler.Scheduler) | class | `Scheduler(model: Any, tokenizer: Any, config: Optional[SchedulerConfig] = None)` | Scheduler for continuous batching using mlx-lm BatchGenerator. | [#L1272-L3518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1272-L3518) |
| [`Scheduler.__init__`](#contract-vllm_mlx.scheduler.Scheduler.__init__) | method | `Scheduler.__init__(model: Any, tokenizer: Any, config: Optional[SchedulerConfig] = None) -> not annotated` | Initialize the scheduler. | [#L1286-L1402](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1286-L1402) |
| [`Scheduler._get_actual_tokenizer`](#contract-vllm_mlx.scheduler.Scheduler._get_actual_tokenizer) | method | `Scheduler._get_actual_tokenizer(tokenizer: Any) -> Any` | Get the actual tokenizer from a processor or tokenizer. | [#L1404-L1418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1404-L1418) |
| [`Scheduler._decode_tokens`](#contract-vllm_mlx.scheduler.Scheduler._decode_tokens) | method | `Scheduler._decode_tokens(token_ids: List[int]) -> str` | Decode token IDs to text, handling both tokenizers and processors. | [#L1420-L1424](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1420-L1424) |
| [`Scheduler._get_detokenizer`](#contract-vllm_mlx.scheduler.Scheduler._get_detokenizer) | method | `Scheduler._get_detokenizer(request_id: str) -> Any` | Get or create a streaming detokenizer for a request. | [#L1426-L1431](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1426-L1431) |
| [`Scheduler._cleanup_detokenizer`](#contract-vllm_mlx.scheduler.Scheduler._cleanup_detokenizer) | method | `Scheduler._cleanup_detokenizer(request_id: str) -> None` | Remove the streaming detokenizer for a finished request. | [#L1433-L1435](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1433-L1435) |
| [`Scheduler._get_stop_tokens`](#contract-vllm_mlx.scheduler.Scheduler._get_stop_tokens) | method | `Scheduler._get_stop_tokens() -> Set[int]` | Get stop token IDs from tokenizer or processor. | [#L1437-L1455](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1437-L1455) |
| [`Scheduler._create_batch_generator`](#contract-vllm_mlx.scheduler.Scheduler._create_batch_generator) | method | `Scheduler._create_batch_generator(sampling_params: SamplingParams) -> BatchGenerator` | Create a BatchGenerator with the given sampling parameters. | [#L1457-L1539](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1457-L1539) |
| [`Scheduler._create_batch_generator._prefill_progress`](#contract-vllm_mlx.scheduler.Scheduler._create_batch_generator._prefill_progress) | nested function | `Scheduler._create_batch_generator._prefill_progress(progress_list) -> not annotated` | Log prefill progress for each uid chunk. | [#L1472-L1479](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1472-L1479) |
| [`Scheduler._make_prompt_cache_save_callback`](#contract-vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback) | method | `Scheduler._make_prompt_cache_save_callback() -> not annotated` | Create a callback that stores prompt-only KV/Mamba cache. | [#L1541-L1585](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1541-L1585) |
| [`Scheduler._make_prompt_cache_save_callback._prompt_cache_save`](#contract-vllm_mlx.scheduler.Scheduler._make_prompt_cache_save_callback._prompt_cache_save) | nested function | `Scheduler._make_prompt_cache_save_callback._prompt_cache_save(uid, extracted_cache) -> not annotated` | Nested Function `Scheduler._make_prompt_cache_save_callback._prompt_cache_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `list`, `_trim_cache_offset`; returns `None`. | [#L1554-L1583](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1554-L1583) |
| [`Scheduler._make_mid_prefill_save_callback`](#contract-vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback) | method | `Scheduler._make_mid_prefill_save_callback(save_interval: int) -> not annotated` | Create a callback for saving intermediate KV cache during chunked prefill. | [#L1587-L1655](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1587-L1655) |
| [`Scheduler._make_mid_prefill_save_callback._mid_prefill_save`](#contract-vllm_mlx.scheduler.Scheduler._make_mid_prefill_save_callback._mid_prefill_save) | nested function | `Scheduler._make_mid_prefill_save_callback._mid_prefill_save(uid, processed_tokens, prompt_cache) -> not annotated` | Nested Function `Scheduler._make_mid_prefill_save_callback._mid_prefill_save` calls `self.uid_to_request_id.get`, `self.requests.get`, `getattr`, `self._extract_cache_states`; returns `None`. | [#L1598-L1653](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1598-L1653) |
| [`Scheduler._close_batch_generator`](#contract-vllm_mlx.scheduler.Scheduler._close_batch_generator) | method | `Scheduler._close_batch_generator() -> None` | Properly close BatchGenerator to restore wired_limit. | [#L1657-L1665](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1657-L1665) |
| [`Scheduler._ensure_batch_generator`](#contract-vllm_mlx.scheduler.Scheduler._ensure_batch_generator) | method | `Scheduler._ensure_batch_generator(sampling_params: SamplingParams) -> None` | Ensure BatchGenerator exists with compatible settings. | [#L1667-L1709](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1667-L1709) |
| [`Scheduler._validate_cache`](#contract-vllm_mlx.scheduler.Scheduler._validate_cache) | method | `Scheduler._validate_cache(cache: Any) -> bool` | Validate that a cache object is usable. | [#L1711-L1769](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1711-L1769) |
| [`Scheduler._extract_cache_states`](#contract-vllm_mlx.scheduler.Scheduler._extract_cache_states) | method | `Scheduler._extract_cache_states(raw_cache: List[Any]) -> List[Dict[str, Any]]` | Extract actual tensor state from each layer cache. | [#L1771-L1806](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1771-L1806) |
| [`Scheduler._reconstruct_cache_from_states`](#contract-vllm_mlx.scheduler.Scheduler._reconstruct_cache_from_states) | method | `Scheduler._reconstruct_cache_from_states(extracted_states: List[Dict[str, Any]]) -> Optional[List[Any]]` | Reconstruct cache objects from extracted cache states. | [#L1808-L1872](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1808-L1872) |
| [`Scheduler.add_request`](#contract-vllm_mlx.scheduler.Scheduler.add_request) | method | `Scheduler.add_request(request: Request) -> None` | Add a new request to the scheduler. | [#L1874-L1997](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1874-L1997) |
| [`Scheduler.abort_request`](#contract-vllm_mlx.scheduler.Scheduler.abort_request) | method | `Scheduler.abort_request(request_id: str) -> bool` | Queue request for abort. | [#L1999-L2014](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L1999-L2014) |
| [`Scheduler._process_pending_aborts`](#contract-vllm_mlx.scheduler.Scheduler._process_pending_aborts) | method | `Scheduler._process_pending_aborts() -> None` | Drain and process pending abort requests. | [#L2016-L2020](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2016-L2020) |
| [`Scheduler._do_abort_request`](#contract-vllm_mlx.scheduler.Scheduler._do_abort_request) | method | `Scheduler._do_abort_request(request_id: str) -> bool` | Actually abort a request. | [#L2022-L2087](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2022-L2087) |
| [`Scheduler.has_requests`](#contract-vllm_mlx.scheduler.Scheduler.has_requests) | method | `Scheduler.has_requests() -> bool` | Check if there are any pending or running requests. | [#L2089-L2091](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2089-L2091) |
| [`Scheduler.get_num_waiting`](#contract-vllm_mlx.scheduler.Scheduler.get_num_waiting) | method | `Scheduler.get_num_waiting() -> int` | Get number of waiting requests. | [#L2093-L2095](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2093-L2095) |
| [`Scheduler.get_num_running`](#contract-vllm_mlx.scheduler.Scheduler.get_num_running) | method | `Scheduler.get_num_running() -> int` | Get number of running requests. | [#L2097-L2099](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2097-L2099) |
| [`Scheduler._schedule_waiting`](#contract-vllm_mlx.scheduler.Scheduler._schedule_waiting) | method | `Scheduler._schedule_waiting() -> List[Request]` | Move requests from waiting queue to running. | [#L2101-L2276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2101-L2276) |
| [`Scheduler._copy_cache_state`](#contract-vllm_mlx.scheduler.Scheduler._copy_cache_state) | method | `Scheduler._copy_cache_state(value: Any) -> Any` | Deep-copy a cache ``state`` payload. | [#L2279-L2295](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2279-L2295) |
| [`Scheduler._prompt_output_entry_is_useless`](#contract-vllm_mlx.scheduler.Scheduler._prompt_output_entry_is_useless) | method | `Scheduler._prompt_output_entry_is_useless(cache: Any) -> bool` | Would a prompt+output entry built from this cache ever be reusable? | [#L2303-L2317](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2303-L2317) |
| [`Scheduler._extract_cache_for_uid`](#contract-vllm_mlx.scheduler.Scheduler._extract_cache_for_uid) | method | `Scheduler._extract_cache_for_uid(uid: int) -> Any` | Pull one sequence's cache out of the live BatchGenerator batch. | [#L2319-L2336](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2319-L2336) |
| [`Scheduler._make_snapshot_destination`](#contract-vllm_mlx.scheduler.Scheduler._make_snapshot_destination) | method | `Scheduler._make_snapshot_destination(live_cache: Any) -> Any` | Build a destination cache with the same topology as the live one. | [#L2338-L2380](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2338-L2380) |
| [`Scheduler._make_snapshot_destination._mirror`](#contract-vllm_mlx.scheduler.Scheduler._make_snapshot_destination._mirror) | nested function | `Scheduler._make_snapshot_destination._mirror(layer: Any) -> Any` | Nested Function `Scheduler._make_snapshot_destination._mirror` calls `getattr`, `_mirror`, `copy.copy`, `type(children)`; has 2 explicit return paths. | [#L2360-L2370](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2360-L2370) |
| [`Scheduler._cache_coverage`](#contract-vllm_mlx.scheduler.Scheduler._cache_coverage) | method | `Scheduler._cache_coverage(cache: Any) -> int \| None` | How many tokens the live cache actually holds. | [#L2383-L2409](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2383-L2409) |
| [`Scheduler._cache_coverage._offset_of`](#contract-vllm_mlx.scheduler.Scheduler._cache_coverage._offset_of) | nested function | `Scheduler._cache_coverage._offset_of(layer: Any) -> int \| None` | Nested Function `Scheduler._cache_coverage._offset_of` calls `getattr`, `isinstance`, `_offset_of`; has 3 explicit return paths. | [#L2393-L2403](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2393-L2403) |
| [`Scheduler._cache_key_for_snapshot`](#contract-vllm_mlx.scheduler.Scheduler._cache_key_for_snapshot) | method | `Scheduler._cache_key_for_snapshot(request: Any, response: Any, raw_cache: Any) -> list[int] \| None` | Key the entry by the tokens the cache covers, not by the prompt. | [#L2411-L2468](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2411-L2468) |
| [`Scheduler._store_prompt_only_cache`](#contract-vllm_mlx.scheduler.Scheduler._store_prompt_only_cache) | method | `Scheduler._store_prompt_only_cache(request: Any, response: Any) -> None` | Store the post-prefill cache under the prompt tokens alone. | [#L2470-L2581](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2470-L2581) |
| [`Scheduler._process_batch_responses`](#contract-vllm_mlx.scheduler.Scheduler._process_batch_responses) | method | `Scheduler._process_batch_responses(responses: List[Any]) -> Tuple[List[RequestOutput], Set[str]]` | Process responses from BatchGenerator. | [#L2583-L2710](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2583-L2710) |
| [`Scheduler._cleanup_finished`](#contract-vllm_mlx.scheduler.Scheduler._cleanup_finished) | method | `Scheduler._cleanup_finished(finished_ids: Set[str]) -> None` | Clean up finished requests and store caches for reuse. | [#L2712-L2865](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2712-L2865) |
| [`Scheduler._is_cache_corruption_error`](#contract-vllm_mlx.scheduler.Scheduler._is_cache_corruption_error) | method | `Scheduler._is_cache_corruption_error(error: Exception) -> bool` | Check if an error indicates cache corruption. | [#L2867-L2870](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2867-L2870) |
| [`Scheduler._is_stream_thread_error`](#contract-vllm_mlx.scheduler.Scheduler._is_stream_thread_error) | method | `Scheduler._is_stream_thread_error(error: Exception) -> bool` | Check if an error indicates MLX stream/thread ownership mismatch. | [#L2872-L2875](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2872-L2875) |
| [`Scheduler._recover_from_cache_error`](#contract-vllm_mlx.scheduler.Scheduler._recover_from_cache_error) | method | `Scheduler._recover_from_cache_error() -> None` | Recover from cache corruption error. | [#L2877-L2895](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2877-L2895) |
| [`Scheduler._recover_from_generation_error`](#contract-vllm_mlx.scheduler.Scheduler._recover_from_generation_error) | method | `Scheduler._recover_from_generation_error() -> Set[str]` | Recover from fatal generation error (OOM, Metal crash). | [#L2897-L2933](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2897-L2933) |
| [`Scheduler._reschedule_running_requests`](#contract-vllm_mlx.scheduler.Scheduler._reschedule_running_requests) | method | `Scheduler._reschedule_running_requests() -> None` | Move running requests back to waiting queue for retry. | [#L2935-L2951](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2935-L2951) |
| [`Scheduler.step`](#contract-vllm_mlx.scheduler.Scheduler.step) | method | `Scheduler.step(max_retries: int = 1) -> SchedulerOutput` | Execute one scheduling step with automatic error recovery. | [#L2953-L3089](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L2953-L3089) |
| [`Scheduler.get_request`](#contract-vllm_mlx.scheduler.Scheduler.get_request) | method | `Scheduler.get_request(request_id: str) -> Optional[Request]` | Get a request by ID. | [#L3091-L3093](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3091-L3093) |
| [`Scheduler.remove_finished_request`](#contract-vllm_mlx.scheduler.Scheduler.remove_finished_request) | method | `Scheduler.remove_finished_request(request_id: str) -> Optional[Request]` | Remove a finished request from tracking. | [#L3095-L3097](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3095-L3097) |
| [`Scheduler.get_running_requests_info`](#contract-vllm_mlx.scheduler.Scheduler.get_running_requests_info) | method | `Scheduler.get_running_requests_info() -> List[Dict[str, Any]]` | Per-request details for status endpoint. | [#L3099-L3165](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3099-L3165) |
| [`Scheduler.get_stats`](#contract-vllm_mlx.scheduler.Scheduler.get_stats) | method | `Scheduler.get_stats() -> Dict[str, Any]` | Get scheduler statistics. | [#L3167-L3193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3167-L3193) |
| [`Scheduler.get_cache_stats`](#contract-vllm_mlx.scheduler.Scheduler.get_cache_stats) | method | `Scheduler.get_cache_stats() -> Optional[Dict[str, Any]]` | Get cache statistics. | [#L3195-L3203](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3195-L3203) |
| [`Scheduler.clear_runtime_caches`](#contract-vllm_mlx.scheduler.Scheduler.clear_runtime_caches) | method | `Scheduler.clear_runtime_caches() -> Dict[str, bool]` | Clear prefix-cache state without resetting scheduler/request state. | [#L3205-L3221](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3205-L3221) |
| [`Scheduler.reset`](#contract-vllm_mlx.scheduler.Scheduler.reset) | method | `Scheduler.reset() -> None` | Reset the scheduler state. | [#L3223-L3246](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3223-L3246) |
| [`Scheduler.deep_reset`](#contract-vllm_mlx.scheduler.Scheduler.deep_reset) | method | `Scheduler.deep_reset() -> None` | Deep reset that clears ALL cache state including model-level caches. | [#L3248-L3276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3248-L3276) |
| [`Scheduler.save_cache_to_disk`](#contract-vllm_mlx.scheduler.Scheduler.save_cache_to_disk) | method | `Scheduler.save_cache_to_disk(cache_dir: str) -> bool` | Save prefix cache to disk for persistence across restarts. | [#L3282-L3287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3282-L3287) |
| [`Scheduler.load_cache_from_disk`](#contract-vllm_mlx.scheduler.Scheduler.load_cache_from_disk) | method | `Scheduler.load_cache_from_disk(cache_dir: str) -> int` | Load prefix cache from disk. | [#L3289-L3294](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3289-L3294) |
| [`Scheduler.clear_prefix_cache`](#contract-vllm_mlx.scheduler.Scheduler.clear_prefix_cache) | method | `Scheduler.clear_prefix_cache() -> None` | Clear the in-memory prefix cache (keeps disk cache untouched). | [#L3296-L3306](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3296-L3306) |
| [`Scheduler.close_ssd_tier`](#contract-vllm_mlx.scheduler.Scheduler.close_ssd_tier) | method | `Scheduler.close_ssd_tier() -> None` | Shut down the SSD cache tier if present. | [#L3308-L3313](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3308-L3313) |
| [`Scheduler._try_promote_ssd_pending`](#contract-vllm_mlx.scheduler.Scheduler._try_promote_ssd_pending) | method | `Scheduler._try_promote_ssd_pending() -> None` | Attempt synchronous SSD promotion for waiting requests tagged ssd_pending. | [#L3315-L3395](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3315-L3395) |
| [`Scheduler.promote_from_ssd`](#contract-vllm_mlx.scheduler.Scheduler.promote_from_ssd) | method | `async Scheduler.promote_from_ssd(request) -> bool` | Promote a cold-tier cache entry for a request (async version). | [#L3397-L3460](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3397-L3460) |
| [`Scheduler.promote_from_ssd.reserve_budget`](#contract-vllm_mlx.scheduler.Scheduler.promote_from_ssd.reserve_budget) | nested function | `Scheduler.promote_from_ssd.reserve_budget(nbytes: int) -> bool` | Tentatively reserve RAM budget for promotion. | [#L3412-L3416](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3412-L3416) |
| [`Scheduler.promote_from_ssd.release_budget`](#contract-vllm_mlx.scheduler.Scheduler.promote_from_ssd.release_budget) | nested function | `Scheduler.promote_from_ssd.release_budget(nbytes: int) -> None` | Release tentatively reserved budget on failure. | [#L3418-L3421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3418-L3421) |
| [`Scheduler._reconstruct_ssd_layers`](#contract-vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers) | method | `Scheduler._reconstruct_ssd_layers(layer_dicts: list[dict]) -> list \| None` | Reconstruct cache objects from deserialized layer dicts. | [#L3462-L3518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3462-L3518) |
| [`Scheduler._reconstruct_ssd_layers._mx_dtype_from_name`](#contract-vllm_mlx.scheduler.Scheduler._reconstruct_ssd_layers._mx_dtype_from_name) | nested function | `Scheduler._reconstruct_ssd_layers._mx_dtype_from_name(name: str) -> not annotated` | Nested Function `Scheduler._reconstruct_ssd_layers._mx_dtype_from_name` calls `getattr`; returns `getattr(mx, name, None)`. | [#L3473-L3474](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/scheduler.py#L3473-L3474) |