# `vllm_mlx.server`

Unified OpenAI-compatible API server for vllm-mlx.

[View the complete module source at #L1-L6916](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1-L6916).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.server
    options:
      members:
        - logger
        - _IMPORTED_SIMPLE_ENGINE
        - _engine
        - _model_manager
        - _model_name
        - _model_path
        - _warm_prompts_path
        - _default_model_key
        - _default_max_tokens
        - _max_request_tokens
        - _default_timeout
        - _default_temperature
        - _default_top_p
        - _default_chat_template_kwargs
        - _default_top_k
        - _default_min_p
        - _default_presence_penalty
        - _default_repetition_penalty
        - _metrics_enabled
        - _max_audio_upload_bytes
        - _max_tts_input_chars
        - _force_mllm_model
        - _default_thinking_token_budget
        - _auto_unload_idle_seconds
        - _lazy_load_model
        - _residency_manager
        - _lifecycle_task
        - _lifespan_active
        - _FALLBACK_TEMPERATURE
        - _FALLBACK_TOP_P
        - _FALLBACK_TOP_K
        - _FALLBACK_MIN_P
        - _FALLBACK_PRESENCE_PENALTY
        - _FALLBACK_REPETITION_PENALTY
        - _resolve_temperature
        - _resolve_top_p
        - _resolve_top_k
        - _resolve_min_p
        - _resolve_presence_penalty
        - _resolve_repetition_penalty
        - _resolve_request_max_tokens
        - _resolve_chat_template_kwargs
        - PreparedChatInvocation
        - _prepare_chat_messages
        - _iter_remote_media_urls
        - _validate_remote_media_urls
        - _raise_remote_media_http_error
        - _prepare_json_logits_processor
        - _build_thinking_processor
        - _resolve_no_final_content_token_limit
        - _generation_metadata
        - _ThinkingAwareLogitsProcessor
        - _attach_response_format_logits_processor
        - _coerce_logit_bias
        - _attach_logit_bias_processor
        - _prepare_chat_completion_invocation
        - _prepare_anthropic_invocation
        - _mcp_manager
        - _mcp_executor
        - _embedding_engine
        - _embedding_model_locked
        - _rerank_engine
        - _rerank_model_locked
        - _api_key
        - _auth_warning_logged
        - _reasoning_parser
        - _reasoning_parser_name
        - _thinking_disabled
        - _enable_auto_tool_choice
        - _tool_call_parser
        - _tool_parser_instance
        - _responses_store
        - _RESPONSES_STORE_MAX_SIZE
        - _TOOL_MARKUP_PATTERN
        - _STREAMING_TOOL_MARKERS
        - _STREAMING_BARE_BRACKET_MARKER
        - _STREAMING_BARE_BRACKET_PARTIAL
        - _STREAMING_TOOL_MARKUP_SCAN_CHARS
        - _strip_backslash_before_unicode
        - _sanitize_log_text
        - _log_and_raise_internal_error
        - _raise_engine_busy
        - RequestModelContext
        - _list_available_model_names
        - _response_model_name
        - _acquire_request_model
        - _stream_with_model_context
        - _build_tool_parser
        - _build_reasoning_parser
        - _prepare_streaming_reasoning_parser
        - _prepare_openai_stream_reasoning_state
        - _request_tool_definitions
        - _streaming_json_fence_stripper
        - _idle_unload_enabled
        - _get_idle_unload_event
        - _invalidate_tool_parser_cache
        - _load_prefix_cache_from_disk
        - _save_prefix_cache_to_disk
        - _get_cache_dir
        - _build_engine
        - _engine_factory
        - _run_blocking_engine_cache_io
        - _restore_engine_state
        - _persist_engine_state
        - _activate_engine
        - _sync_engine_from_residency
        - _get_lifecycle_status
        - _public_lifecycle_status
        - _lifecycle_loop
        - _acquire_default_engine
        - _release_default_engine
        - lifespan
        - app
        - security
        - _metrics_result_from_status
        - _metrics_path_for_request
        - _metrics_middleware
        - RateLimiter
        - _rate_limiter
        - check_rate_limit
        - verify_api_key
        - get_engine
        - _coerce_tool_arguments
        - _validate_model_name
        - _get_engine_tokenizer
        - _get_or_init_tool_parser
        - _parse_tool_calls_with_parser
        - _apply_response_format_or_raise
        - _response_format_type
        - _promote_streaming_response_format_delta
        - _new_response_item_id
        - _response_content_to_text
        - _responses_tools_to_chat_tools
        - _responses_input_to_chat_messages
        - _responses_request_to_new_persisted_messages
        - _responses_request_to_persisted_messages
        - _responses_request_to_chat_request
        - _build_responses_output_items
        - _response_output_items_to_chat_messages
        - _build_response_object
        - _prepare_responses_request
        - _prepare_streaming_responses_request
        - _run_responses_request
        - _stream_responses_request
        - _responses_sse_event
        - _HARMONY_ANALYSIS_BLOCK_RE
        - _strip_harmony_analysis_blocks
        - _extract_reasoning_and_tool_calls
        - _detect_native_tool_support
        - _detect_harmony_rendering
        - _tool_choice_disabled
        - _get_streaming_tool_parser
        - _extract_streaming_tool_delta
        - _stream_request_metadata
        - _parse_streaming_tool_content
        - _streaming_tool_markup_possible
        - _streaming_tool_markup_possible_after_delta
        - load_embedding_model
        - load_reranker_model
        - load_model
        - load_model_registry
        - get_usage
        - metrics
        - health
        - status
        - cache_stats
        - clear_cache
        - clear_prefix_cache
        - cancel_request
        - delete_request
        - list_models
        - create_embeddings
        - rerank_documents
        - list_mcp_tools
        - list_mcp_servers
        - execute_mcp_tool
        - _stt_engine
        - _tts_engine
        - create_transcription
        - create_speech
        - list_voices
        - _ensure_sse_terminal
        - _find_uvicorn_cycle
        - _is_client_disconnected
        - _disconnect_guard
        - _wait_with_disconnect
        - _start_request_budget
        - _remaining_request_timeout
        - _active_request_contexts
        - _acquire_default_engine_for_request
        - _release_engine_for_request
        - _make_release_cleanup
        - create_completion
        - create_chat_completion
        - _normalize_messages
        - create_response
        - _get_forced_tool_name
        - _apply_forced_tool_choice
        - _tool_name
        - _inject_json_instruction
        - _convert_anthropic_stop_reason
        - _prepare_anthropic_endpoint_invocation
        - create_anthropic_message
        - count_anthropic_tokens
        - _emit_content_pieces
        - _stream_anthropic_messages
        - stream_completion
        - stream_chat_completion
        - init_mcp
        - _make_keepalive_http_protocol
        - main
        - create_parser
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.server._resolve_temperature" markdown="1">
<summary><code>vllm_mlx.server._resolve_temperature</code> · function</summary>

```python
vllm_mlx.server._resolve_temperature(request_value: float | None) -> float
```

Resolve temperature: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `request_value`; `_default_temperature`; `_FALLBACK_TEMPERATURE`

**Exceptions and behavior**

Function `_resolve_temperature` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L225-L231](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L225-L231).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_top_p" markdown="1">
<summary><code>vllm_mlx.server._resolve_top_p</code> · function</summary>

```python
vllm_mlx.server._resolve_top_p(request_value: float | None) -> float
```

Resolve top_p: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `request_value`; `_default_top_p`; `_FALLBACK_TOP_P`

**Exceptions and behavior**

Function `_resolve_top_p` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L234-L240](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L234-L240).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_top_k" markdown="1">
<summary><code>vllm_mlx.server._resolve_top_k</code> · function</summary>

```python
vllm_mlx.server._resolve_top_k(request_value: int | None) -> int
```

Resolve top_k: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `int \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `int`
- Direct return expressions: `request_value`; `_default_top_k`; `_FALLBACK_TOP_K`

**Exceptions and behavior**

Function `_resolve_top_k` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L243-L249](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L243-L249).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_min_p" markdown="1">
<summary><code>vllm_mlx.server._resolve_min_p</code> · function</summary>

```python
vllm_mlx.server._resolve_min_p(request_value: float | None) -> float
```

Resolve min_p: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `request_value`; `_default_min_p`; `_FALLBACK_MIN_P`

**Exceptions and behavior**

Function `_resolve_min_p` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L252-L258](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L252-L258).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_presence_penalty" markdown="1">
<summary><code>vllm_mlx.server._resolve_presence_penalty</code> · function</summary>

```python
vllm_mlx.server._resolve_presence_penalty(request_value: float | None) -> float
```

Resolve presence_penalty: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `request_value`; `_default_presence_penalty`; `_FALLBACK_PRESENCE_PENALTY`

**Exceptions and behavior**

Function `_resolve_presence_penalty` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L261-L267](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L261-L267).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_repetition_penalty" markdown="1">
<summary><code>vllm_mlx.server._resolve_repetition_penalty</code> · function</summary>

```python
vllm_mlx.server._resolve_repetition_penalty(request_value: float | None) -> float
```

Resolve repetition_penalty: request > CLI default > fallback.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `request_value`; `_default_repetition_penalty`; `_FALLBACK_REPETITION_PENALTY`

**Exceptions and behavior**

Function `_resolve_repetition_penalty` has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L270-L276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L270-L276).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_request_max_tokens" markdown="1">
<summary><code>vllm_mlx.server._resolve_request_max_tokens</code> · function</summary>

```python
vllm_mlx.server._resolve_request_max_tokens(requested_value: int | None) -> int
```

Resolve and validate a request's max_tokens budget.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `requested_value` | `int \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `int`
- Direct return expressions: `_default_max_tokens`; `requested_value`

**Exceptions and behavior**

Function `_resolve_request_max_tokens` calls `HTTPException`; can raise `HTTPException`; has 2 explicit return paths.
Directly raised exceptions: `HTTPException`.

[View source #L279-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L279-L288).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_chat_template_kwargs" markdown="1">
<summary><code>vllm_mlx.server._resolve_chat_template_kwargs</code> · function</summary>

```python
vllm_mlx.server._resolve_chat_template_kwargs(request_value: dict[str, object] | None) -> dict[str, object]
```

Resolve chat template kwargs: request > server default > empty dict.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_value` | `dict[str, object] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict[str, object]`
- Direct return expressions: `resolved`

**Exceptions and behavior**

Function `_resolve_chat_template_kwargs` calls `resolved.update`; returns `resolved`.
No direct `raise` statement appears in this definition.

[View source #L291-L300](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L291-L300).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.PreparedChatInvocation" markdown="1">
<summary><code>vllm_mlx.server.PreparedChatInvocation</code> · class</summary>

```python
vllm_mlx.server.PreparedChatInvocation(messages: list[dict], chat_kwargs: dict[str, object], response_format: object | None, json_logits_processor: object | None, thinking_processor: object | None = None)
```

Fully prepared inputs for a single engine.chat/stream_chat call.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict]` | `yes` | `none` | Required constructor field. |
| `chat_kwargs` | `dict[str, object]` | `yes` | `none` | Required constructor field. |
| `response_format` | `object \| None` | `yes` | `none` | Required constructor field. |
| `json_logits_processor` | `object \| None` | `yes` | `none` | Required constructor field. |
| `thinking_processor` | `object \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |

**Returns**

- Constructs: `vllm_mlx.server.PreparedChatInvocation`

**Exceptions and behavior**

Class `PreparedChatInvocation` declares 0 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L304-L311](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L304-L311).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_chat_messages" markdown="1">
<summary><code>vllm_mlx.server._prepare_chat_messages</code> · function</summary>

```python
vllm_mlx.server._prepare_chat_messages(engine: BaseEngine, request_messages: list[Message | dict]) -> tuple[list[dict], list, list, list, bool]
```

Normalize messages and collect media once for both stream/non-stream paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `request_messages` | `list[Message \| dict]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[list[dict], list, list, list, bool]`
- Direct return expressions: `(messages, images, videos, audios, has_media)`

**Exceptions and behavior**

Function `_prepare_chat_messages` calls `_validate_remote_media_urls`, `bool`, `getattr`, `hasattr`; returns `(messages, images, videos, audios, has_media)`.
No direct `raise` statement appears in this definition.

[View source #L314-L398](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L314-L398).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._iter_remote_media_urls" markdown="1">
<summary><code>vllm_mlx.server._iter_remote_media_urls</code> · function</summary>

```python
vllm_mlx.server._iter_remote_media_urls(messages: list[Message | dict]) -> not annotated
```

Yield remote media URLs from OpenAI-style multimodal message content.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[Message \| dict]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Yields values incrementally.

**Exceptions and behavior**

Function `_iter_remote_media_urls` calls `isinstance`, `msg.get`, `hasattr`, `item.model_dump`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L401-L429](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L401-L429).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._validate_remote_media_urls" markdown="1">
<summary><code>vllm_mlx.server._validate_remote_media_urls</code> · function</summary>

```python
vllm_mlx.server._validate_remote_media_urls(messages: list[Message | dict]) -> None
```

Validate remote media URLs during request preparation.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[Message \| dict]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_validate_remote_media_urls` calls `_iter_remote_media_urls`, `_validate_url_safety`.
No direct `raise` statement appears in this definition.

[View source #L432-L435](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L432-L435).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._raise_remote_media_http_error" markdown="1">
<summary><code>vllm_mlx.server._raise_remote_media_http_error</code> · function</summary>

```python
vllm_mlx.server._raise_remote_media_http_error(exc: UnsafeRemoteURLError) -> None
```

Log internal URL-safety detail while returning a generic client error.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `exc` | `UnsafeRemoteURLError` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_raise_remote_media_http_error` calls `logger.warning`, `_sanitize_log_text`, `HTTPException`; can raise `HTTPException`.
Directly raised exceptions: `HTTPException`.

[View source #L438-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L438-L444).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_json_logits_processor" markdown="1">
<summary><code>vllm_mlx.server._prepare_json_logits_processor</code> · function</summary>

```python
vllm_mlx.server._prepare_json_logits_processor(engine: BaseEngine, messages: list[dict], response_format: object | None, *, tools: list | None, tool_choice: object | None, log_context: str | None = None, thinking_model: bool = False) -> tuple[list[dict], object | None]
```

Inject response_format instruction and build constrained decoding processor.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `messages` | `list[dict]` | `yes` | `none` | Required positional or keyword input. |
| `response_format` | `object \| None` | `yes` | `none` | Required positional or keyword input. |
| `tools` | `list \| None` | `yes` | `none` | Required keyword-only input. |
| `tool_choice` | `object \| None` | `yes` | `none` | Required keyword-only input. |
| `log_context` | `str \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `thinking_model` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. |

**Returns**

- Type: `tuple[list[dict], object | None]`
- Direct return expressions: `(messages, json_logits_processor)`

**Exceptions and behavior**

Function `_prepare_json_logits_processor` calls `build_json_system_prompt`, `_inject_json_instruction`, `_get_engine_tokenizer`, `build_json_logits_processor`; returns `(messages, json_logits_processor)`.
No direct `raise` statement appears in this definition.

[View source #L447-L497](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L447-L497).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_thinking_processor" markdown="1">
<summary><code>vllm_mlx.server._build_thinking_processor</code> · function</summary>

```python
vllm_mlx.server._build_thinking_processor(engine: BaseEngine, thinking_token_budget: int, *, inner: object | None = None, prompt_has_think_tag: bool = True) -> object | None
```

Build a ThinkingAwareLogitsProcessor if the tokenizer has think tokens.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `thinking_token_budget` | `int` | `yes` | `none` | Required positional or keyword input. |
| `inner` | `object \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |
| `prompt_has_think_tag` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `object | None`
- Direct return expressions: `None`; `proc`

**Exceptions and behavior**

Function `_build_thinking_processor` calls `_get_engine_tokenizer`, `tokenizer.encode`, `logger.debug`, `getattr`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L500-L554](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L500-L554).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._resolve_no_final_content_token_limit" markdown="1">
<summary><code>vllm_mlx.server._resolve_no_final_content_token_limit</code> · function</summary>

```python
vllm_mlx.server._resolve_no_final_content_token_limit() -> int | None
```

Function `_resolve_no_final_content_token_limit` calls `os.environ.get`, `raw.strip`, `int`, `logger.warning`; has 2 explicit return paths.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `int | None`
- Direct return expressions: `None`; `value`

**Exceptions and behavior**

Function `_resolve_no_final_content_token_limit` calls `os.environ.get`, `raw.strip`, `int`, `logger.warning`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L557-L568](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L557-L568).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._generation_metadata" markdown="1">
<summary><code>vllm_mlx.server._generation_metadata</code> · function</summary>

```python
vllm_mlx.server._generation_metadata(thinking_processor: object | None) -> GenerationMetadata | None
```

Function `_generation_metadata` calls `GenerationMetadata`, `getattr`, `bool`; has 2 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `thinking_processor` | `object \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `GenerationMetadata | None`
- Direct return expressions: `None`; `GenerationMetadata(no_final_content_watchdog_tokens=getattr(thinking_processor, '_no_final_content_token_limit', None),…`

**Exceptions and behavior**

Function `_generation_metadata` calls `GenerationMetadata`, `getattr`, `bool`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L571-L583](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L571-L583).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor</code> · class</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor(inner, prompt_has_think_tag: bool = False)
```

Wrap a ``JSONSchemaLogitsProcessor`` so JSON constraining only activates after the model emits ``</think>``, letting it reason freely first.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `inner` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_has_think_tag` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. |

**Returns**

- Constructs: `vllm_mlx.server._ThinkingAwareLogitsProcessor`

**Exceptions and behavior**

Class `_ThinkingAwareLogitsProcessor` declares 5 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L586-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L586-L697).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.__init__" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor.__init__</code> · method</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor.__init__(inner, prompt_has_think_tag: bool = False) -> not annotated
```

Method `_ThinkingAwareLogitsProcessor.__init__` updates `self._inner`, `self._active`, `self._in_thinking`, `self._waiting_for_json`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `inner` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `prompt_has_think_tag` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `_ThinkingAwareLogitsProcessor.__init__` updates `self._inner`, `self._active`, `self._in_thinking`, `self._waiting_for_json`.
No direct `raise` statement appears in this definition.

[View source #L597-L608](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L597-L608).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor._scan_for_json_start" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor._scan_for_json_start</code> · method</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor._scan_for_json_start(tokens_list, tokens, logits) -> not annotated
```

Scan generated tokens for the first ``{`` or ``[``.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tokens_list` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `logits` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `self._inner(tokens, logits)`; `logits`

**Exceptions and behavior**

Method `_ThinkingAwareLogitsProcessor._scan_for_json_start` updates `self._active`, `self._inner._prompt_len`; calls `len`, `range`, `self._tokenizer.decode`, `any`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L610-L635](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L610-L635).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.__call__" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor.__call__</code> · method</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor.__call__(tokens, logits) -> not annotated
```

Method `_ThinkingAwareLogitsProcessor.__call__` updates `self._base_prompt_len`, `self._in_thinking`, `self._waiting_for_json`, `self._json_scan_offset`; calls `self._inner`, `hasattr`, `tokens.tolist`, `list`; has 3 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tokens` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `logits` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `self._inner(tokens, logits)`; `self._scan_for_json_start(tokens_list, tokens, logits)`; `logits`

**Exceptions and behavior**

Method `_ThinkingAwareLogitsProcessor.__call__` updates `self._base_prompt_len`, `self._in_thinking`, `self._waiting_for_json`, `self._json_scan_offset`; calls `self._inner`, `hasattr`, `tokens.tolist`, `list`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L637-L688](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L637-L688).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.schema" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor.schema</code> · method</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor.schema() -> not annotated
```

Method `_ThinkingAwareLogitsProcessor.schema` returns `self._inner.schema`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `self._inner.schema`

**Exceptions and behavior**

Method `_ThinkingAwareLogitsProcessor.schema` returns `self._inner.schema`.
No direct `raise` statement appears in this definition.

[View source #L692-L693](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L692-L693).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ThinkingAwareLogitsProcessor._disabled" markdown="1">
<summary><code>vllm_mlx.server._ThinkingAwareLogitsProcessor._disabled</code> · method</summary>

```python
vllm_mlx.server._ThinkingAwareLogitsProcessor._disabled() -> not annotated
```

Method `_ThinkingAwareLogitsProcessor._disabled` returns `self._inner._disabled`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `self._inner._disabled`

**Exceptions and behavior**

Method `_ThinkingAwareLogitsProcessor._disabled` returns `self._inner._disabled`.
No direct `raise` statement appears in this definition.

[View source #L696-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L696-L697).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._attach_response_format_logits_processor" markdown="1">
<summary><code>vllm_mlx.server._attach_response_format_logits_processor</code> · function</summary>

```python
vllm_mlx.server._attach_response_format_logits_processor(chat_kwargs: dict, json_logits_processor: object) -> object
```

Attach response_format constraints and keep thinking disabled.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `chat_kwargs` | `dict` | `yes` | `none` | Required positional or keyword input. |
| `json_logits_processor` | `object` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `object`
- Direct return expressions: `json_logits_processor`

**Exceptions and behavior**

Function `_attach_response_format_logits_processor` calls `dict`, `chat_kwargs.get`, `list`; returns `json_logits_processor`.
No direct `raise` statement appears in this definition.

[View source #L700-L717](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L700-L717).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._coerce_logit_bias" markdown="1">
<summary><code>vllm_mlx.server._coerce_logit_bias</code> · function</summary>

```python
vllm_mlx.server._coerce_logit_bias(logit_bias: dict[str, float]) -> dict[int, float]
```

Function `_coerce_logit_bias` calls `logit_bias.items`, `int`, `float`, `HTTPException`; can raise `HTTPException`; returns `coerced`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `logit_bias` | `dict[str, float]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict[int, float]`
- Direct return expressions: `coerced`

**Exceptions and behavior**

Function `_coerce_logit_bias` calls `logit_bias.items`, `int`, `float`, `HTTPException`; can raise `HTTPException`; returns `coerced`.
Directly raised exceptions: `HTTPException`.

[View source #L720-L730](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L720-L730).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._attach_logit_bias_processor" markdown="1">
<summary><code>vllm_mlx.server._attach_logit_bias_processor</code> · function</summary>

```python
vllm_mlx.server._attach_logit_bias_processor(chat_kwargs: dict, logit_bias: dict[str, float] | None) -> not annotated
```

Function `_attach_logit_bias_processor` calls `make_logits_processors`, `_coerce_logit_bias`, `chat_kwargs.get`, `list`; returns `None`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `chat_kwargs` | `dict` | `yes` | `none` | Required positional or keyword input. |
| `logit_bias` | `dict[str, float] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_attach_logit_bias_processor` calls `make_logits_processors`, `_coerce_logit_bias`, `chat_kwargs.get`, `list`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L733-L744](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L733-L744).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_chat_completion_invocation" markdown="1">
<summary><code>vllm_mlx.server._prepare_chat_completion_invocation</code> · function</summary>

```python
vllm_mlx.server._prepare_chat_completion_invocation(engine: BaseEngine, request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation
```

Precompute messages, kwargs, and decoding constraints for chat completions.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `effective_max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `PreparedChatInvocation`
- Direct return expressions: `PreparedChatInvocation(messages=messages, chat_kwargs=chat_kwargs, response_format=response_format, json_logits_process…`

**Exceptions and behavior**

Function `_prepare_chat_completion_invocation` calls `_prepare_chat_messages`, `_prepare_json_logits_processor`, `bool`, `_resolve_temperature`; returns `PreparedChatInvocation(messages=messages, chat_kwargs=chat_kwargs, response_format=response_format, json_logits_process…`.
No direct `raise` statement appears in this definition.

[View source #L747-L855](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L747-L855).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_anthropic_invocation" markdown="1">
<summary><code>vllm_mlx.server._prepare_anthropic_invocation</code> · function</summary>

```python
vllm_mlx.server._prepare_anthropic_invocation(engine: BaseEngine, openai_request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation
```

Precompute messages, kwargs, and decoding constraints for Anthropic API.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `openai_request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `effective_max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `PreparedChatInvocation`
- Direct return expressions: `PreparedChatInvocation(messages=messages, chat_kwargs=chat_kwargs, response_format=response_format, json_logits_process…`

**Exceptions and behavior**

Function `_prepare_anthropic_invocation` calls `_prepare_chat_messages`, `_prepare_json_logits_processor`, `bool`, `_resolve_temperature`; returns `PreparedChatInvocation(messages=messages, chat_kwargs=chat_kwargs, response_format=response_format, json_logits_process…`.
No direct `raise` statement appears in this definition.

[View source #L858-L910](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L858-L910).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._thinking_disabled" markdown="1">
<summary><code>vllm_mlx.server._thinking_disabled</code> · function</summary>

```python
vllm_mlx.server._thinking_disabled(request, chat_kwargs: dict | None = None) -> bool
```

Return True iff thinking is explicitly disabled for this request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `chat_kwargs` | `dict \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `bool`
- Direct return expressions: `True`; `False`

**Exceptions and behavior**

Function `_thinking_disabled` calls `getattr`, `chat_kwargs.get`, `ctk.get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L934-L950](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L934-L950).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._strip_backslash_before_unicode" markdown="1">
<summary><code>vllm_mlx.server._strip_backslash_before_unicode</code> · function</summary>

```python
vllm_mlx.server._strip_backslash_before_unicode(obj: object) -> object
```

Remove spurious backslashes before non-ASCII chars in JSON string values.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `obj` | `object` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `object`
- Direct return expressions: `{k: _strip_backslash_before_unicode(v) for k, v in obj.items()}`; `[_strip_backslash_before_unicode(v) for v in obj]`; `re.sub('\\\\([^\\x00-\\x7F])', '\\1', obj)`; `obj`

**Exceptions and behavior**

Function `_strip_backslash_before_unicode` calls `isinstance`, `_strip_backslash_before_unicode`, `obj.items`, `re.sub`; has 4 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L983-L997](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L983-L997).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._sanitize_log_text" markdown="1">
<summary><code>vllm_mlx.server._sanitize_log_text</code> · function</summary>

```python
vllm_mlx.server._sanitize_log_text(value: object, limit: int | None = None) -> str
```

Escape control characters before logging untrusted text.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `value` | `object` | `yes` | `none` | Required positional or keyword input. |
| `limit` | `int \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `str`
- Direct return expressions: `sanitized[:limit] + '...'`; `sanitized`

**Exceptions and behavior**

Function `_sanitize_log_text` calls `str`, `escaped.append`, `ch.isprintable`, `ord`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1000-L1022](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1000-L1022).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._log_and_raise_internal_error" markdown="1">
<summary><code>vllm_mlx.server._log_and_raise_internal_error</code> · function</summary>

```python
vllm_mlx.server._log_and_raise_internal_error(log_prefix: str, exc: Exception, detail: str) -> None
```

Log a sanitized exception string and raise a generic 500 response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `log_prefix` | `str` | `yes` | `none` | Required positional or keyword input. |
| `exc` | `Exception` | `yes` | `none` | Required positional or keyword input. |
| `detail` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_log_and_raise_internal_error` calls `logger.error`, `_sanitize_log_text`, `HTTPException`; can raise `HTTPException`.
Directly raised exceptions: `HTTPException`.

[View source #L1025-L1028](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1025-L1028).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._raise_engine_busy" markdown="1">
<summary><code>vllm_mlx.server._raise_engine_busy</code> · function</summary>

```python
vllm_mlx.server._raise_engine_busy(exc: EngineBusy) -> None
```

Translate serialized-engine admission failures into retryable HTTP 503.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `exc` | `EngineBusy` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_raise_engine_busy` calls `HTTPException`, `str`; can raise `HTTPException`.
Directly raised exceptions: `HTTPException`.

[View source #L1031-L1039](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1031-L1039).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.RequestModelContext" markdown="1">
<summary><code>vllm_mlx.server.RequestModelContext</code> · class</summary>

```python
vllm_mlx.server.RequestModelContext(model_name: str, engine: BaseEngine, lease: ModelLease | None = None)
```

Request-scoped engine/lease context.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `yes` | `none` | Required constructor field. |
| `engine` | `BaseEngine` | `yes` | `none` | Required constructor field. |
| `lease` | `ModelLease \| None` | `no` | `None` | Optional constructor field; defaults to `None`. |

**Returns**

- Constructs: `vllm_mlx.server.RequestModelContext`

**Exceptions and behavior**

Class `RequestModelContext` declares 1 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L1043-L1056](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1043-L1056).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.RequestModelContext.release" markdown="1">
<summary><code>vllm_mlx.server.RequestModelContext.release</code> · method</summary>

```python
async vllm_mlx.server.RequestModelContext.release() -> None
```

Release the registry lease once, if this context owns one.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Method `RequestModelContext.release` updates `self.lease`; calls `lease.release`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1050-L1056](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1050-L1056).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._list_available_model_names" markdown="1">
<summary><code>vllm_mlx.server._list_available_model_names</code> · function</summary>

```python
vllm_mlx.server._list_available_model_names() -> list[str]
```

Function `_list_available_model_names` has 2 explicit return paths.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `list[str]`
- Direct return expressions: `_model_manager.registered_model_names`; `[_model_name] if _model_name else []`

**Exceptions and behavior**

Function `_list_available_model_names` has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1059-L1062](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1059-L1062).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._response_model_name" markdown="1">
<summary><code>vllm_mlx.server._response_model_name</code> · function</summary>

```python
vllm_mlx.server._response_model_name(request_model: str) -> str
```

Return the response model field for single-model or registry mode.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_model` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `_model_name or request_model`

**Exceptions and behavior**

Function `_response_model_name` returns `_model_name or request_model`.
No direct `raise` statement appears in this definition.

[View source #L1065-L1067](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1065-L1067).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._acquire_request_model" markdown="1">
<summary><code>vllm_mlx.server._acquire_request_model</code> · function</summary>

```python
async vllm_mlx.server._acquire_request_model(request_model: str) -> RequestModelContext
```

Acquire the model/engine that should serve this request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_model` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `RequestModelContext`
- Direct return expressions: `RequestModelContext(model_name=_model_name or request_model, engine=engine)`; `RequestModelContext(model_name=request_model, engine=lease.engine, lease=lease)`

**Exceptions and behavior**

Function `_acquire_request_model` calls `_validate_model_name`, `get_engine`, `_detect_native_tool_support`, `_detect_harmony_rendering`; awaits asynchronous work; can raise `HTTPException`; has 2 explicit return paths.
Directly raised exceptions: `HTTPException`.

[View source #L1070-L1094](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1070-L1094).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_with_model_context" markdown="1">
<summary><code>vllm_mlx.server._stream_with_model_context</code> · function</summary>

```python
async vllm_mlx.server._stream_with_model_context(context: RequestModelContext, stream: AsyncIterator[str]) -> AsyncIterator[str]
```

Ensure model leases survive for the full streaming response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `context` | `RequestModelContext` | `yes` | `none` | Required positional or keyword input. |
| `stream` | `AsyncIterator[str]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `_stream_with_model_context` calls `context.release`; awaits asynchronous work; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L1097-L1106](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1097-L1106).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_tool_parser" markdown="1">
<summary><code>vllm_mlx.server._build_tool_parser</code> · function</summary>

```python
vllm_mlx.server._build_tool_parser(engine: BaseEngine | None) -> not annotated
```

Create a fresh tool parser instance for a single request/stream.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`; `parser_cls(tokenizer)`; `parser_cls()`

**Exceptions and behavior**

Function `_build_tool_parser` calls `type`, `ToolParserManager.get_tool_parser`, `_get_engine_tokenizer`, `parser_cls`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1109-L1123](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1109-L1123).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_reasoning_parser" markdown="1">
<summary><code>vllm_mlx.server._build_reasoning_parser</code> · function</summary>

```python
vllm_mlx.server._build_reasoning_parser(engine: BaseEngine | None = None) -> not annotated
```

Create a fresh reasoning parser instance for a single request/stream.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `parser_cls(tokenizer)`; `parser_cls()`; `None`; `type(_reasoning_parser)(tokenizer)`; `type(_reasoning_parser)()`

**Exceptions and behavior**

Function `_build_reasoning_parser` calls `getattr`, `get_reasoning_parser`, `parser_cls`, `type(_reasoning_parser)`; has 5 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1126-L1140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1126-L1140).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_streaming_reasoning_parser" markdown="1">
<summary><code>vllm_mlx.server._prepare_streaming_reasoning_parser</code> · function</summary>

```python
vllm_mlx.server._prepare_streaming_reasoning_parser(engine: BaseEngine, request: ChatCompletionRequest | ResponsesRequest | None, chat_kwargs: dict[str, object], *, allowed: bool = True) -> not annotated
```

Build and reset request-local reasoning state when thinking is enabled.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest \| ResponsesRequest \| None` | `yes` | `none` | Required positional or keyword input. |
| `chat_kwargs` | `dict[str, object]` | `yes` | `none` | Required positional or keyword input. |
| `allowed` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`; `parser`

**Exceptions and behavior**

Function `_prepare_streaming_reasoning_parser` calls `_thinking_disabled`, `_build_reasoning_parser`, `parser.reset_state`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1143-L1156](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1143-L1156).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_openai_stream_reasoning_state" markdown="1">
<summary><code>vllm_mlx.server._prepare_openai_stream_reasoning_state</code> · function</summary>

```python
vllm_mlx.server._prepare_openai_stream_reasoning_state(engine: BaseEngine, request: ChatCompletionRequest, chat_kwargs: dict[str, object]) -> tuple[object | None, bool]
```

Return request-local reasoning state and the legacy Nemotron marker state.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `chat_kwargs` | `dict[str, object]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[object | None, bool]`
- Direct return expressions: `(parser, is_thinking_model)`

**Exceptions and behavior**

Function `_prepare_openai_stream_reasoning_state` calls `_prepare_streaming_reasoning_parser`, `(engine.model_name or '').lower`, `_thinking_disabled`; returns `(parser, is_thinking_model)`.
No direct `raise` statement appears in this definition.

[View source #L1159-L1171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1159-L1171).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._request_tool_definitions" markdown="1">
<summary><code>vllm_mlx.server._request_tool_definitions</code> · function</summary>

```python
vllm_mlx.server._request_tool_definitions(request: ChatCompletionRequest) -> list | None
```

Return the request tool schema once for streaming argument coercion.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list | None`
- Direct return expressions: `request.model_dump(include={'tools'}).get('tools')`; `None`

**Exceptions and behavior**

Function `_request_tool_definitions` calls `request.model_dump(include={'tools'}).get`, `request.model_dump`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1174-L1178](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1174-L1178).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._streaming_json_fence_stripper" markdown="1">
<summary><code>vllm_mlx.server._streaming_json_fence_stripper</code> · function</summary>

```python
vllm_mlx.server._streaming_json_fence_stripper(request: ChatCompletionRequest) -> StreamingJsonFenceStripper | None
```

Create a fence stripper only for JSON-constrained streaming responses.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `StreamingJsonFenceStripper | None`
- Direct return expressions: `StreamingJsonFenceStripper()`; `None`

**Exceptions and behavior**

Function `_streaming_json_fence_stripper` calls `getattr`, `isinstance`, `response_format.get`, `StreamingJsonFenceStripper`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1181-L1191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1181-L1191).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_idle_unload_event" markdown="1">
<summary><code>vllm_mlx.server._get_idle_unload_event</code> · function</summary>

```python
vllm_mlx.server._get_idle_unload_event() -> asyncio.Event
```

Return the idle-unload gate event, creating it on first use.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `asyncio.Event`
- Direct return expressions: `_idle_unload_enabled`

**Exceptions and behavior**

Function `_get_idle_unload_event` calls `asyncio.Event`, `_idle_unload_enabled.set`; returns `_idle_unload_enabled`.
No direct `raise` statement appears in this definition.

[View source #L1206-L1217](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1206-L1217).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._invalidate_tool_parser_cache" markdown="1">
<summary><code>vllm_mlx.server._invalidate_tool_parser_cache</code> · function</summary>

```python
vllm_mlx.server._invalidate_tool_parser_cache(reason: str | None = None) -> None
```

Drop cached parser state when the serving tokenizer changes.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `reason` | `str \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_invalidate_tool_parser_cache` calls `logger.debug`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L1220-L1229](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1220-L1229).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._load_prefix_cache_from_disk" markdown="1">
<summary><code>vllm_mlx.server._load_prefix_cache_from_disk</code> · function</summary>

```python
vllm_mlx.server._load_prefix_cache_from_disk(engine: BaseEngine | None = None) -> None
```

Load prefix cache from disk during startup.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_load_prefix_cache_from_disk` calls `_get_cache_dir`, `logger.info`, `target_engine.load_cache_from_disk`, `logger.warning`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L1232-L1250](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1232-L1250).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._save_prefix_cache_to_disk" markdown="1">
<summary><code>vllm_mlx.server._save_prefix_cache_to_disk</code> · function</summary>

```python
vllm_mlx.server._save_prefix_cache_to_disk(engine: BaseEngine | None = None) -> None
```

Save prefix cache to disk during shutdown.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_save_prefix_cache_to_disk` calls `_get_cache_dir`, `logger.info`, `target_engine.save_cache_to_disk`, `logger.warning`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L1253-L1271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1253-L1271).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_cache_dir" markdown="1">
<summary><code>vllm_mlx.server._get_cache_dir</code> · function</summary>

```python
vllm_mlx.server._get_cache_dir() -> str
```

Get cache persistence directory based on actual model path.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `str`
- Direct return expressions: `cache_dir`

**Exceptions and behavior**

Function `_get_cache_dir` calls `logger.info`, `type`, `str(model_name).replace('/', '--').replace`, `str(model_name).replace`; returns `cache_dir`.
No direct `raise` statement appears in this definition.

[View source #L1274-L1290](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1274-L1290).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_engine" markdown="1">
<summary><code>vllm_mlx.server._build_engine</code> · function</summary>

```python
vllm_mlx.server._build_engine(spec: ModelSpec) -> BaseEngine
```

Construct an engine instance from a model spec without starting it.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `BatchedEngine(model_name=spec.model_name, scheduler_config=spec.scheduler_config, stream_interval=spec.stream_interval,…`; `SimpleEngine(model_name=spec.model_name, force_mllm=spec.force_mllm, mtp=spec.mtp, prefill_step_size=spec.prefill_step_…`

**Exceptions and behavior**

Function `_build_engine` calls `logger.info`, `BatchedEngine`, `getattr`, `SimpleEngine`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1293-L1323](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1293-L1323).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._engine_factory" markdown="1">
<summary><code>vllm_mlx.server._engine_factory</code> · function</summary>

```python
async vllm_mlx.server._engine_factory(spec: ModelSpec) -> BaseEngine
```

Async engine factory used by the residency manager.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `_build_engine(spec)`

**Exceptions and behavior**

Function `_engine_factory` calls `_build_engine`; returns `_build_engine(spec)`.
No direct `raise` statement appears in this definition.

[View source #L1326-L1328](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1326-L1328).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._run_blocking_engine_cache_io" markdown="1">
<summary><code>vllm_mlx.server._run_blocking_engine_cache_io</code> · function</summary>

```python
async vllm_mlx.server._run_blocking_engine_cache_io(io_fn, engine: BaseEngine) -> None
```

Run blocking cache persistence off the event loop.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `io_fn` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_run_blocking_engine_cache_io` calls `asyncio.create_task`, `asyncio.to_thread`, `asyncio.shield`, `suspend_cancellation`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1331-L1350](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1331-L1350).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._restore_engine_state" markdown="1">
<summary><code>vllm_mlx.server._restore_engine_state</code> · function</summary>

```python
async vllm_mlx.server._restore_engine_state(spec: ModelSpec, engine: BaseEngine) -> None
```

Restore engine-local state, such as prefix cache, after a cold load.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_restore_engine_state` calls `hasattr`, `_run_blocking_engine_cache_io`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1353-L1356](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1353-L1356).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._persist_engine_state" markdown="1">
<summary><code>vllm_mlx.server._persist_engine_state</code> · function</summary>

```python
async vllm_mlx.server._persist_engine_state(spec: ModelSpec, engine: BaseEngine) -> None
```

Persist engine-local state before an idle unload or shutdown unload.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `spec` | `ModelSpec` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_persist_engine_state` calls `hasattr`, `_run_blocking_engine_cache_io`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1359-L1362](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1359-L1362).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._activate_engine" markdown="1">
<summary><code>vllm_mlx.server._activate_engine</code> · function</summary>

```python
vllm_mlx.server._activate_engine(engine: BaseEngine | None) -> BaseEngine | None
```

Set the global engine pointer and refresh parser-sensitive state.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `BaseEngine | None`
- Direct return expressions: `_engine`

**Exceptions and behavior**

Function `_activate_engine` calls `_invalidate_tool_parser_cache`, `_detect_native_tool_support`, `_detect_harmony_rendering`; returns `_engine`.
No direct `raise` statement appears in this definition.

[View source #L1365-L1375](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1365-L1375).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._sync_engine_from_residency" markdown="1">
<summary><code>vllm_mlx.server._sync_engine_from_residency</code> · function</summary>

```python
vllm_mlx.server._sync_engine_from_residency() -> BaseEngine | None
```

Sync the global engine pointer from the residency manager state.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `BaseEngine | None`
- Direct return expressions: `_engine`; `_activate_engine(_residency_manager.get_engine(_default_model_key))`

**Exceptions and behavior**

Function `_sync_engine_from_residency` calls `_activate_engine`, `_residency_manager.get_engine`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1378-L1388](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1378-L1388).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_lifecycle_status" markdown="1">
<summary><code>vllm_mlx.server._get_lifecycle_status</code> · function</summary>

```python
vllm_mlx.server._get_lifecycle_status() -> dict | None
```

Get lifecycle status for the default resident if lifecycle is enabled.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `dict | None`
- Direct return expressions: `None`; `_residency_manager.get_status(_default_model_key)`

**Exceptions and behavior**

Function `_get_lifecycle_status` calls `_residency_manager.get_status`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1391-L1395](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1391-L1395).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._public_lifecycle_status" markdown="1">
<summary><code>vllm_mlx.server._public_lifecycle_status</code> · function</summary>

```python
vllm_mlx.server._public_lifecycle_status(lifecycle: dict | None) -> dict | None
```

Return residency status safe for unauthenticated public endpoints.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `lifecycle` | `dict \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict | None`
- Direct return expressions: `None`; `public`

**Exceptions and behavior**

Function `_public_lifecycle_status` calls `dict`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1398-L1410](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1398-L1410).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._lifecycle_loop" markdown="1">
<summary><code>vllm_mlx.server._lifecycle_loop</code> · function</summary>

```python
async vllm_mlx.server._lifecycle_loop() -> None
```

Background idle-unload loop for the default resident.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `_lifecycle_loop` calls `asyncio.sleep`, `_get_idle_unload_event().wait`, `_get_idle_unload_event`, `_residency_manager.unload_if_idle`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L1413-L1433](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1413-L1433).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._acquire_default_engine" markdown="1">
<summary><code>vllm_mlx.server._acquire_default_engine</code> · function</summary>

```python
async vllm_mlx.server._acquire_default_engine(*, count_activity: bool = True) -> BaseEngine
```

Acquire the default engine, auto-loading via the residency manager if needed.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `get_engine()`; `activated_engine`

**Exceptions and behavior**

Function `_acquire_default_engine` calls `get_engine`, `_residency_manager.acquire`, `_activate_engine`, `HTTPException`; awaits asynchronous work; can raise `HTTPException`; has 2 explicit return paths.
Directly raised exceptions: `HTTPException`.

[View source #L1436-L1451](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1436-L1451).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._release_default_engine" markdown="1">
<summary><code>vllm_mlx.server._release_default_engine</code> · function</summary>

```python
async vllm_mlx.server._release_default_engine(*, count_activity: bool = True) -> None
```

Release the default engine after request processing.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_release_default_engine` calls `_residency_manager.release`, `_sync_engine_from_residency`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L1454-L1463](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1454-L1463).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.lifespan" markdown="1">
<summary><code>vllm_mlx.server.lifespan</code> · function</summary>

```python
async vllm_mlx.server.lifespan(app: FastAPI) -> not annotated
```

FastAPI lifespan for startup/shutdown events.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `app` | `FastAPI` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Yields values incrementally.

**Exceptions and behavior**

Function `lifespan` calls `_get_idle_unload_event().clear`, `_get_idle_unload_event`, `_residency_manager.ensure_loaded`, `_sync_engine_from_residency`; awaits asynchronous work; yields values incrementally; can raise `primary_exc`, `cleanup_exc`.
Directly raised exceptions: `primary_exc`, `cleanup_exc`.

[View source #L1466-L1589](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1466-L1589).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._metrics_result_from_status" markdown="1">
<summary><code>vllm_mlx.server._metrics_result_from_status</code> · function</summary>

```python
vllm_mlx.server._metrics_result_from_status(status_code: int) -> str
```

Map HTTP-ish status codes to low-cardinality inference results.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `status_code` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `'client_closed'`; `'timeout'`; `'error'`; `'success'`

**Exceptions and behavior**

Function `_metrics_result_from_status` has 4 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1602-L1610](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1602-L1610).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._metrics_path_for_request" markdown="1">
<summary><code>vllm_mlx.server._metrics_path_for_request</code> · function</summary>

```python
vllm_mlx.server._metrics_path_for_request(request: Request) -> str
```

Prefer route templates over raw URLs to keep metrics cardinality bounded.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `str(path)`; `'__unmatched__'`

**Exceptions and behavior**

Function `_metrics_path_for_request` calls `request.scope.get`, `getattr`, `str`, `candidate.matches`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1613-L1626](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1613-L1626).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._metrics_middleware" markdown="1">
<summary><code>vllm_mlx.server._metrics_middleware</code> · function</summary>

```python
async vllm_mlx.server._metrics_middleware(request: Request, call_next) -> not annotated
```

Capture generic HTTP request metrics when enabled.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | Required positional or keyword input. |
| `call_next` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `await call_next(request)`; `response`

**Exceptions and behavior**

Function `_metrics_middleware` calls `call_next`, `_metrics_path_for_request`, `time.perf_counter`, `_metrics.observe_http_start`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1630-L1659](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1630-L1659).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.RateLimiter" markdown="1">
<summary><code>vllm_mlx.server.RateLimiter</code> · class</summary>

```python
vllm_mlx.server.RateLimiter(requests_per_minute: int = 60, enabled: bool = False)
```

Simple in-memory rate limiter using sliding window.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `requests_per_minute` | `int` | `no` | `60` | Optional positional or keyword input; defaults to `60`. |
| `enabled` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. |

**Returns**

- Constructs: `vllm_mlx.server.RateLimiter`

**Exceptions and behavior**

Class `RateLimiter` declares 2 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L1662-L1700](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1662-L1700).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.RateLimiter.__init__" markdown="1">
<summary><code>vllm_mlx.server.RateLimiter.__init__</code> · method</summary>

```python
vllm_mlx.server.RateLimiter.__init__(requests_per_minute: int = 60, enabled: bool = False) -> not annotated
```

Method `RateLimiter.__init__` updates `self.requests_per_minute`, `self.enabled`, `self.window_size`, `self._requests`; calls `defaultdict`, `threading.Lock`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `requests_per_minute` | `int` | `no` | `60` | Optional positional or keyword input; defaults to `60`. |
| `enabled` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Method `RateLimiter.__init__` updates `self.requests_per_minute`, `self.enabled`, `self.window_size`, `self._requests`; calls `defaultdict`, `threading.Lock`.
No direct `raise` statement appears in this definition.

[View source #L1665-L1670](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1665-L1670).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.RateLimiter.is_allowed" markdown="1">
<summary><code>vllm_mlx.server.RateLimiter.is_allowed</code> · method</summary>

```python
vllm_mlx.server.RateLimiter.is_allowed(client_id: str) -> tuple[bool, int]
```

Check if request is allowed for client.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `client_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[bool, int]`
- Direct return expressions: `(True, 0)`; `(False, max(1, retry_after))`

**Exceptions and behavior**

Method `RateLimiter.is_allowed` calls `time.time`, `len`, `min`, `int`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1672-L1700](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1672-L1700).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.check_rate_limit" markdown="1">
<summary><code>vllm_mlx.server.check_rate_limit</code> · function</summary>

```python
async vllm_mlx.server.check_rate_limit(request: Request) -> not annotated
```

Rate limiting dependency.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `check_rate_limit` calls `request.headers.get`, `_rate_limiter.is_allowed`, `HTTPException`, `str`; can raise `HTTPException`.
Directly raised exceptions: `HTTPException`.

[View source #L1707-L1720](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1707-L1720).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.verify_api_key" markdown="1">
<summary><code>vllm_mlx.server.verify_api_key</code> · function</summary>

```python
async vllm_mlx.server.verify_api_key(credentials: HTTPAuthorizationCredentials = Depends(security)) -> not annotated
```

Verify API key if authentication is enabled.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `credentials` | `HTTPAuthorizationCredentials` | `no` | `Depends(security)` | Optional positional or keyword input; defaults to `Depends(security)`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `True`

**Exceptions and behavior**

Function `verify_api_key` calls `logger.warning`, `HTTPException`, `secrets.compare_digest`; can raise `HTTPException`; returns `True`.
Directly raised exceptions: `HTTPException`.

[View source #L1723-L1742](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1723-L1742).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.get_engine" markdown="1">
<summary><code>vllm_mlx.server.get_engine</code> · function</summary>

```python
vllm_mlx.server.get_engine() -> BaseEngine
```

Get the loaded engine, raising error if not loaded.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `BaseEngine`
- Direct return expressions: `_engine`

**Exceptions and behavior**

Function `get_engine` calls `HTTPException`; can raise `HTTPException`; returns `_engine`.
Directly raised exceptions: `HTTPException`.

[View source #L1745-L1749](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1745-L1749).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._coerce_tool_arguments" markdown="1">
<summary><code>vllm_mlx.server._coerce_tool_arguments</code> · function</summary>

```python
vllm_mlx.server._coerce_tool_arguments(arguments_json: str, tool_name: str, tools: list[dict] | None) -> str
```

Coerce tool call arguments to match the tool schema.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `arguments_json` | `str` | `yes` | `none` | Required positional or keyword input. |
| `tool_name` | `str` | `yes` | `none` | Required positional or keyword input. |
| `tools` | `list[dict] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `arguments_json`; `json.dumps(arguments, ensure_ascii=False)`

**Exceptions and behavior**

Function `_coerce_tool_arguments` calls `isinstance`, `tool.get('function', {}).get`, `tool.get`, `tool['function'].get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1752-L1796](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1752-L1796).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._validate_model_name" markdown="1">
<summary><code>vllm_mlx.server._validate_model_name</code> · function</summary>

```python
vllm_mlx.server._validate_model_name(request_model: str) -> None
```

Validate that the request model name matches the served model.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_model` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_validate_model_name` calls `_model_manager.has_model`, `', '.join`, `_list_available_model_names`, `HTTPException`; can raise `HTTPException`; returns `None`.
Directly raised exceptions: `HTTPException`.

[View source #L1799-L1818](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1799-L1818).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_engine_tokenizer" markdown="1">
<summary><code>vllm_mlx.server._get_engine_tokenizer</code> · function</summary>

```python
vllm_mlx.server._get_engine_tokenizer(engine: BaseEngine | None) -> object | None
```

Return tokenizer-like parser state from the active engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `object | None`
- Direct return expressions: `None`; `tokenizer`; `getattr(engine, '_tokenizer', None)`

**Exceptions and behavior**

Function `_get_engine_tokenizer` calls `getattr`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1821-L1828](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1821-L1828).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_or_init_tool_parser" markdown="1">
<summary><code>vllm_mlx.server._get_or_init_tool_parser</code> · function</summary>

```python
vllm_mlx.server._get_or_init_tool_parser(engine: BaseEngine | None = None) -> not annotated
```

Return the cached tool parser, initializing it from the given engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `_tool_parser_instance`

**Exceptions and behavior**

Function `_get_or_init_tool_parser` calls `ToolParserManager.get_tool_parser`, `_get_engine_tokenizer`, `parser_cls`, `logger.info`; returns `_tool_parser_instance`.
No direct `raise` statement appears in this definition.

[View source #L1831-L1841](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1831-L1841).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._parse_tool_calls_with_parser" markdown="1">
<summary><code>vllm_mlx.server._parse_tool_calls_with_parser</code> · function</summary>

```python
vllm_mlx.server._parse_tool_calls_with_parser(output_text: str, request: ChatCompletionRequest | None = None, engine: BaseEngine | None = None) -> tuple[str, list | None]
```

Parse tool calls from model output using the configured parser.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `output_text` | `str` | `yes` | `none` | The model output text |
| `request` | `ChatCompletionRequest \| None` | `no` | `None` | The original request (for context) |
| `engine` | `BaseEngine \| None` | `no` | `None` | The request-local engine to use for parser initialization |

**Returns**

- Type: `tuple[str, list | None]`
- Direct return expressions: `(output_text, None)`; `parse_tool_calls(output_text, request_dict)`; `(result.content or '', tool_calls)`; `(fallback_text, fallback_calls)`; `(result.content, None)`; `(fallback_text, None)`

**Exceptions and behavior**

Function `_parse_tool_calls_with_parser` calls `request.model_dump`, `getattr`, `request_dict.get`, `parse_tool_calls`; has 6 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1844-L1930](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1844-L1930).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._apply_response_format_or_raise" markdown="1">
<summary><code>vllm_mlx.server._apply_response_format_or_raise</code> · function</summary>

```python
vllm_mlx.server._apply_response_format_or_raise(text: str, response_format: object, *, ensure_ascii: bool = False) -> str
```

Return validated JSON content or fail before returning a success response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `response_format` | `object` | `yes` | `none` | Required positional or keyword input. |
| `ensure_ascii` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. |

**Returns**

- Type: `str`
- Direct return expressions: `_strip_backslash_before_unicode(text)`

**Exceptions and behavior**

Function `_apply_response_format_or_raise` calls `apply_response_format_or_error`, `HTTPException`, `_strip_backslash_before_unicode`; can raise `HTTPException`; returns `_strip_backslash_before_unicode(text)`.
Directly raised exceptions: `HTTPException`.

[View source #L1933-L1952](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1933-L1952).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._response_format_type" markdown="1">
<summary><code>vllm_mlx.server._response_format_type</code> · function</summary>

```python
vllm_mlx.server._response_format_type(response_format: object | None) -> str | None
```

Function `_response_format_type` calls `isinstance`, `response_format.get`, `getattr`; has 3 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `response_format` | `object \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str | None`
- Direct return expressions: `None`; `response_format.get('type')`; `getattr(response_format, 'type', None)`

**Exceptions and behavior**

Function `_response_format_type` calls `isinstance`, `response_format.get`, `getattr`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1955-L1960](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1955-L1960).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._promote_streaming_response_format_delta" markdown="1">
<summary><code>vllm_mlx.server._promote_streaming_response_format_delta</code> · function</summary>

```python
vllm_mlx.server._promote_streaming_response_format_delta(content: str | None, reasoning: str | None, request: ChatCompletionRequest) -> tuple[str | None, str | None]
```

Keep response_format JSON on the streaming content channel.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `content` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `reasoning` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[str | None, str | None]`
- Direct return expressions: `(content, reasoning)`; `(reasoning, None)`

**Exceptions and behavior**

Function `_promote_streaming_response_format_delta` calls `_response_format_type`, `getattr`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1963-L1981](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1963-L1981).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._new_response_item_id" markdown="1">
<summary><code>vllm_mlx.server._new_response_item_id</code> · function</summary>

```python
vllm_mlx.server._new_response_item_id(prefix: str) -> str
```

Generate stable OpenAI-style item ids.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `prefix` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `f'{prefix}_{uuid.uuid4().hex}'`

**Exceptions and behavior**

Function `_new_response_item_id` calls `uuid.uuid4`; returns `f'{prefix}_{uuid.uuid4().hex}'`.
No direct `raise` statement appears in this definition.

[View source #L1984-L1986](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1984-L1986).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._response_content_to_text" markdown="1">
<summary><code>vllm_mlx.server._response_content_to_text</code> · function</summary>

```python
vllm_mlx.server._response_content_to_text(content) -> str
```

Normalize Responses API content items into plain text.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `content` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `''`; `content`; `'\n'.join((part for part in text_parts if part))`

**Exceptions and behavior**

Function `_response_content_to_text` calls `isinstance`, `part.get`, `getattr`, `text_parts.append`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L1989-L2006](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1989-L2006).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_tools_to_chat_tools" markdown="1">
<summary><code>vllm_mlx.server._responses_tools_to_chat_tools</code> · function</summary>

```python
vllm_mlx.server._responses_tools_to_chat_tools(tools: list[ResponseFunctionTool | dict]) -> tuple[list[dict] | None, list[str]]
```

Convert supported Responses tools and report unsupported tool types.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tools` | `list[ResponseFunctionTool \| dict]` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[list[dict] | None, list[str]]`
- Direct return expressions: `(None, [])`; `(supported or None, unsupported)`

**Exceptions and behavior**

Function `_responses_tools_to_chat_tools` calls `isinstance`, `tool.get`, `unsupported.append`, `type`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2009-L2049](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2009-L2049).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_input_to_chat_messages" markdown="1">
<summary><code>vllm_mlx.server._responses_input_to_chat_messages</code> · function</summary>

```python
vllm_mlx.server._responses_input_to_chat_messages(request: ResponsesRequest) -> list[dict]
```

Convert Responses API input items into chat-completions-style messages.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[dict]`
- Direct return expressions: `messages`

**Exceptions and behavior**

Function `_responses_input_to_chat_messages` calls `_responses_store.get`, `HTTPException`, `messages.extend`, `copy.deepcopy`; can raise `HTTPException`; returns `messages`.
Directly raised exceptions: `HTTPException`.

[View source #L2052-L2170](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2052-L2170).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_request_to_new_persisted_messages" markdown="1">
<summary><code>vllm_mlx.server._responses_request_to_new_persisted_messages</code> · function</summary>

```python
vllm_mlx.server._responses_request_to_new_persisted_messages(request: ResponsesRequest) -> list[dict]
```

Persist only the current request's replayable input items.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[dict]`
- Direct return expressions: `_responses_input_to_chat_messages(request_without_history)`

**Exceptions and behavior**

Function `_responses_request_to_new_persisted_messages` calls `request.model_copy`, `_responses_input_to_chat_messages`; returns `_responses_input_to_chat_messages(request_without_history)`.
No direct `raise` statement appears in this definition.

[View source #L2173-L2181](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2173-L2181).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_request_to_persisted_messages" markdown="1">
<summary><code>vllm_mlx.server._responses_request_to_persisted_messages</code> · function</summary>

```python
vllm_mlx.server._responses_request_to_persisted_messages(request: ResponsesRequest) -> list[dict]
```

Persist replayable history for chained previous_response_id requests.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[dict]`
- Direct return expressions: `messages`

**Exceptions and behavior**

Function `_responses_request_to_persisted_messages` calls `_responses_store.get`, `HTTPException`, `messages.extend`, `copy.deepcopy`; can raise `HTTPException`; returns `messages`.
Directly raised exceptions: `HTTPException`.

[View source #L2184-L2200](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2184-L2200).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_request_to_chat_request" markdown="1">
<summary><code>vllm_mlx.server._responses_request_to_chat_request</code> · function</summary>

```python
vllm_mlx.server._responses_request_to_chat_request(request: ResponsesRequest) -> ChatCompletionRequest
```

Build a ChatCompletionRequest from a ResponsesRequest.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `ChatCompletionRequest`
- Direct return expressions: `ChatCompletionRequest(model=request.model, messages=[Message(**msg) for msg in messages], temperature=request.temperatu…`

**Exceptions and behavior**

Function `_responses_request_to_chat_request` calls `HTTPException`, `logger.debug`, `_responses_tools_to_chat_tools`, `_responses_input_to_chat_messages`; can raise `HTTPException`; returns `ChatCompletionRequest(model=request.model, messages=[Message(**msg) for msg in messages], temperature=request.temperatu…`.
Directly raised exceptions: `HTTPException`.

[View source #L2203-L2253](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2203-L2253).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_responses_output_items" markdown="1">
<summary><code>vllm_mlx.server._build_responses_output_items</code> · function</summary>

```python
vllm_mlx.server._build_responses_output_items(text: str | None, reasoning: str | None, tool_calls: list[ToolCall] | None) -> list[ResponseMessageItem | ResponseReasoningItem | ResponseFunctionCallItem]
```

Convert parsed assistant output into Responses API output items.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `reasoning` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `tool_calls` | `list[ToolCall] \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[ResponseMessageItem | ResponseReasoningItem | ResponseFunctionCallItem]`
- Direct return expressions: `output_items`

**Exceptions and behavior**

Function `_build_responses_output_items` calls `output_items.append`, `ResponseReasoningItem`, `_new_response_item_id`, `ResponseReasoningTextPart`; returns `output_items`.
No direct `raise` statement appears in this definition.

[View source #L2256-L2293](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2256-L2293).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._response_output_items_to_chat_messages" markdown="1">
<summary><code>vllm_mlx.server._response_output_items_to_chat_messages</code> · function</summary>

```python
vllm_mlx.server._response_output_items_to_chat_messages(output_items: list) -> list[dict]
```

Persist assistant output in chat-completions form for previous_response_id.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `output_items` | `list` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list[dict]`
- Direct return expressions: `[]`; `[{'role': 'assistant', 'content': ''.join(assistant_text_parts), 'tool_calls': assistant_tool_calls or None}]`

**Exceptions and behavior**

Function `_response_output_items_to_chat_messages` calls `isinstance`, `assistant_text_parts.append`, `_response_content_to_text`, `assistant_tool_calls.append`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2296-L2325](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2296-L2325).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._build_response_object" markdown="1">
<summary><code>vllm_mlx.server._build_response_object</code> · function</summary>

```python
vllm_mlx.server._build_response_object(request: ResponsesRequest, output_items: list[ResponseMessageItem | ResponseReasoningItem | ResponseFunctionCallItem], prompt_tokens: int, completion_tokens: int, finish_reason: str | None, response_id: str | None = None) -> ResponseObject
```

Build a full Responses API object.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |
| `output_items` | `list[ResponseMessageItem \| ResponseReasoningItem \| ResponseFunctionCallItem]` | `yes` | `none` | Required positional or keyword input. |
| `prompt_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |
| `completion_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |
| `finish_reason` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `response_id` | `str \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `ResponseObject`
- Direct return expressions: `response`

**Exceptions and behavior**

Function `_build_response_object` calls `ResponseObject`, `_new_response_item_id`, `_resolve_top_p`, `_resolve_temperature`; returns `response`.
No direct `raise` statement appears in this definition.

[View source #L2328-L2367](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2328-L2367).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_responses_request" markdown="1">
<summary><code>vllm_mlx.server._prepare_responses_request</code> · function</summary>

```python
vllm_mlx.server._prepare_responses_request(request: ResponsesRequest, *, validate_remote_media: bool = True) -> tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]
```

Prepare a Responses request for execution on the chat engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |
| `validate_remote_media` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]`
- Direct return expressions: `(engine, chat_request, messages, chat_kwargs)`

**Exceptions and behavior**

Function `_prepare_responses_request` calls `_validate_model_name`, `get_engine`, `_responses_request_to_chat_request`, `logger.info`; returns `(engine, chat_request, messages, chat_kwargs)`.
No direct `raise` statement appears in this definition.

[View source #L2370-L2414](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2370-L2414).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_streaming_responses_request" markdown="1">
<summary><code>vllm_mlx.server._prepare_streaming_responses_request</code> · function</summary>

```python
vllm_mlx.server._prepare_streaming_responses_request(request: ResponsesRequest) -> tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]
```

Prepare a streaming Responses request after eager URL validation.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]`
- Direct return expressions: `_prepare_responses_request(request, validate_remote_media=False)`

**Exceptions and behavior**

Function `_prepare_streaming_responses_request` calls `_prepare_responses_request`; returns `_prepare_responses_request(request, validate_remote_media=False)`.
No direct `raise` statement appears in this definition.

[View source #L2417-L2421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2417-L2421).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._run_responses_request" markdown="1">
<summary><code>vllm_mlx.server._run_responses_request</code> · function</summary>

```python
async vllm_mlx.server._run_responses_request(request: ResponsesRequest, raw_request: Request) -> tuple[ResponseObject | None, list[dict]]
```

Execute a Responses API request against the backend chat engine.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[ResponseObject | None, list[dict]]`
- Direct return expressions: `(None, [])`; `(response_object, persisted_messages)`

**Exceptions and behavior**

Function `_run_responses_request` calls `_prepare_responses_request`, `_wait_with_disconnect`, `engine.chat`, `_parse_tool_calls_with_parser`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2424-L2477](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2424-L2477).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_responses_request" markdown="1">
<summary><code>vllm_mlx.server._stream_responses_request</code> · function</summary>

```python
async vllm_mlx.server._stream_responses_request(request: ResponsesRequest) -> AsyncIterator[str]
```

Execute a Responses API request and stream SSE events incrementally.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `_stream_responses_request` calls `_prepare_streaming_responses_request`, `chat_request.model_dump`, `_new_response_item_id`, `_build_response_object`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L2480-L2868](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2480-L2868).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_responses_request._start_text_item" markdown="1">
<summary><code>vllm_mlx.server._stream_responses_request._start_text_item</code> · nested function</summary>

```python
vllm_mlx.server._stream_responses_request._start_text_item() -> list[str]
```

Nested Function `_stream_responses_request._start_text_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `list[str]`
- Direct return expressions: `events`

**Exceptions and behavior**

Nested Function `_stream_responses_request._start_text_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`.
No direct `raise` statement appears in this definition.

[View source #L2525-L2561](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2525-L2561).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_responses_request._start_reasoning_item" markdown="1">
<summary><code>vllm_mlx.server._stream_responses_request._start_reasoning_item</code> · nested function</summary>

```python
vllm_mlx.server._stream_responses_request._start_reasoning_item() -> list[str]
```

Nested Function `_stream_responses_request._start_reasoning_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `list[str]`
- Direct return expressions: `events`

**Exceptions and behavior**

Nested Function `_stream_responses_request._start_reasoning_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`.
No direct `raise` statement appears in this definition.

[View source #L2563-L2598](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2563-L2598).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._responses_sse_event" markdown="1">
<summary><code>vllm_mlx.server._responses_sse_event</code> · function</summary>

```python
vllm_mlx.server._responses_sse_event(event_type: str, payload: BaseModel | dict) -> str
```

Encode a Responses API SSE event.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `event_type` | `str` | `yes` | `none` | Required positional or keyword input. |
| `payload` | `BaseModel \| dict` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `f'event: {event_type}\ndata: {data}\n\n'`

**Exceptions and behavior**

Function `_responses_sse_event` calls `isinstance`, `payload.model_dump_json`, `json.dumps`; returns `f'event: {event_type}\ndata: {data}\n\n'`.
No direct `raise` statement appears in this definition.

[View source #L2871-L2878](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2871-L2878).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._strip_harmony_analysis_blocks" markdown="1">
<summary><code>vllm_mlx.server._strip_harmony_analysis_blocks</code> · function</summary>

```python
vllm_mlx.server._strip_harmony_analysis_blocks(text: str) -> str
```

Remove harmony analysis-channel blocks (and their content) so reasoning text is never handed to the tool parser, while commentary/final text is preserved.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `_HARMONY_ANALYSIS_BLOCK_RE.sub('', text)`

**Exceptions and behavior**

Function `_strip_harmony_analysis_blocks` calls `_HARMONY_ANALYSIS_BLOCK_RE.sub`; returns `_HARMONY_ANALYSIS_BLOCK_RE.sub('', text)`.
No direct `raise` statement appears in this definition.

[View source #L2888-L2892](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2888-L2892).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._extract_reasoning_and_tool_calls" markdown="1">
<summary><code>vllm_mlx.server._extract_reasoning_and_tool_calls</code> · function</summary>

```python
vllm_mlx.server._extract_reasoning_and_tool_calls(output_text: str, request: ChatCompletionRequest | None = None, *, allow_reasoning: bool = True, engine: BaseEngine | None = None) -> tuple[str | None, str | None, list[ToolCall] | None]
```

Extract reasoning first, then parse tool calls from the cleaned content.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `output_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `allow_reasoning` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |

**Returns**

- Type: `tuple[str | None, str | None, list[ToolCall] | None]`
- Direct return expressions: `(reasoning_text, cleaned_text, tool_calls)`

**Exceptions and behavior**

Function `_extract_reasoning_and_tool_calls` calls `_reasoning_parser.extract_reasoning`, `getattr`, `_strip_harmony_analysis_blocks`, `_parse_tool_calls_with_parser`; returns `(reasoning_text, cleaned_text, tool_calls)`.
No direct `raise` statement appears in this definition.

[View source #L2895-L2951](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2895-L2951).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._detect_native_tool_support" markdown="1">
<summary><code>vllm_mlx.server._detect_native_tool_support</code> · function</summary>

```python
vllm_mlx.server._detect_native_tool_support() -> bool
```

Detect if the active tool parser supports native tool format.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `parser_cls.supports_native_format()`

**Exceptions and behavior**

Function `_detect_native_tool_support` calls `ToolParserManager.get_tool_parser`, `parser_cls.supports_native_format`, `logger.error`, `ToolParserManager.list_registered`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2954-L2983](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2954-L2983).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._detect_harmony_rendering" markdown="1">
<summary><code>vllm_mlx.server._detect_harmony_rendering</code> · function</summary>

```python
vllm_mlx.server._detect_harmony_rendering() -> bool
```

Detect whether the harmony rendering path should handle prompt building.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `True`

**Exceptions and behavior**

Function `_detect_harmony_rendering` calls `is_harmony_parser_name`, `logger.warning`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L2986-L3019](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2986-L3019).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._tool_choice_disabled" markdown="1">
<summary><code>vllm_mlx.server._tool_choice_disabled</code> · function</summary>

```python
vllm_mlx.server._tool_choice_disabled(request: ChatCompletionRequest | None) -> bool
```

Return True when tool_choice explicitly disables tool calling.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `tool_choice == 'none'`

**Exceptions and behavior**

Function `_tool_choice_disabled` calls `getattr`, `request.model_dump`, `request_dict.get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3022-L3031](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3022-L3031).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_streaming_tool_parser" markdown="1">
<summary><code>vllm_mlx.server._get_streaming_tool_parser</code> · function</summary>

```python
vllm_mlx.server._get_streaming_tool_parser(request: ChatCompletionRequest | None, engine: BaseEngine | None = None) -> not annotated
```

Get a streaming-capable tool parser for this request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest \| None` | `yes` | `none` | Required positional or keyword input. |
| `engine` | `BaseEngine \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`; `_build_tool_parser(engine)`; `parser`

**Exceptions and behavior**

Function `_get_streaming_tool_parser` calls `_tool_choice_disabled`, `_get_engine_tokenizer`, `_build_tool_parser`, `logger.warning`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3034-L3071](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3034-L3071).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._extract_streaming_tool_delta" markdown="1">
<summary><code>vllm_mlx.server._extract_streaming_tool_delta</code> · function</summary>

```python
vllm_mlx.server._extract_streaming_tool_delta(parser, previous_text: str, delta_text: str, request_context: dict) -> tuple[str, dict | None]
```

Parse one request-local streaming delta and return new accumulated text.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `parser` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `previous_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `delta_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `request_context` | `dict` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[str, dict | None]`
- Direct return expressions: `(current_text, result)`

**Exceptions and behavior**

Function `_extract_streaming_tool_delta` calls `parser.extract_tool_calls_streaming`; returns `(current_text, result)`.
No direct `raise` statement appears in this definition.

[View source #L3074-L3088](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3074-L3088).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_request_metadata" markdown="1">
<summary><code>vllm_mlx.server._stream_request_metadata</code> · function</summary>

```python
vllm_mlx.server._stream_request_metadata(request: ChatCompletionRequest) -> tuple[dict, list | None, bool]
```

Function `_stream_request_metadata` calls `request.model_dump(include={'tools'}).get`, `request.model_dump`, `bool`; returns `({'tools': tools or []}, tools, include_usage)`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[dict, list | None, bool]`
- Direct return expressions: `({'tools': tools or []}, tools, include_usage)`

**Exceptions and behavior**

Function `_stream_request_metadata` calls `request.model_dump(include={'tools'}).get`, `request.model_dump`, `bool`; returns `({'tools': tools or []}, tools, include_usage)`.
No direct `raise` statement appears in this definition.

[View source #L3091-L3100](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3091-L3100).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._parse_streaming_tool_content" markdown="1">
<summary><code>vllm_mlx.server._parse_streaming_tool_content</code> · function</summary>

```python
vllm_mlx.server._parse_streaming_tool_content(parser, accumulated_text: str, delta_text: str, request_context: dict) -> tuple[str, dict | None, bool]
```

Function `_parse_streaming_tool_content` calls `_extract_streaming_tool_delta`; returns `(accumulated_text, result, suppress)`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `parser` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `accumulated_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `delta_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `request_context` | `dict` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[str, dict | None, bool]`
- Direct return expressions: `(accumulated_text, result, suppress)`

**Exceptions and behavior**

Function `_parse_streaming_tool_content` calls `_extract_streaming_tool_delta`; returns `(accumulated_text, result, suppress)`.
No direct `raise` statement appears in this definition.

[View source #L3103-L3116](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3103-L3116).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._streaming_tool_markup_possible" markdown="1">
<summary><code>vllm_mlx.server._streaming_tool_markup_possible</code> · function</summary>

```python
vllm_mlx.server._streaming_tool_markup_possible(text: str) -> bool
```

Heuristic marker check to avoid parser work on ordinary text chunks.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `text` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `any((marker in text for marker in _STREAMING_TOOL_MARKERS)) or _STREAMING_BARE_BRACKET_MARKER.search(text) is not None …`

**Exceptions and behavior**

Function `_streaming_tool_markup_possible` calls `any`, `_STREAMING_BARE_BRACKET_MARKER.search`, `_STREAMING_BARE_BRACKET_PARTIAL.search`; returns `any((marker in text for marker in _STREAMING_TOOL_MARKERS)) or _STREAMING_BARE_BRACKET_MARKER.search(text) is not None …`.
No direct `raise` statement appears in this definition.

[View source #L3119-L3125](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3119-L3125).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._streaming_tool_markup_possible_after_delta" markdown="1">
<summary><code>vllm_mlx.server._streaming_tool_markup_possible_after_delta</code> · function</summary>

```python
vllm_mlx.server._streaming_tool_markup_possible_after_delta(accumulated_text: str, delta_text: str) -> bool
```

Check only the boundary window needed to detect newly appearing tool markup.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `accumulated_text` | `str` | `yes` | `none` | Required positional or keyword input. |
| `delta_text` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `False`; `_streaming_tool_markup_possible(check_text)`

**Exceptions and behavior**

Function `_streaming_tool_markup_possible_after_delta` calls `_streaming_tool_markup_possible`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3128-L3143](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3128-L3143).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.load_embedding_model" markdown="1">
<summary><code>vllm_mlx.server.load_embedding_model</code> · function</summary>

```python
vllm_mlx.server.load_embedding_model(model_name: str | None, *, lock: bool = False, reuse_existing: bool = True) -> None
```

Load or reuse the embedding model engine when configured.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `lock` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. |
| `reuse_existing` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `load_embedding_model` calls `EmbeddingEngine`, `_embedding_engine.load`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L3146-L3171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3146-L3171).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.load_reranker_model" markdown="1">
<summary><code>vllm_mlx.server.load_reranker_model</code> · function</summary>

```python
vllm_mlx.server.load_reranker_model(model_name: str | None, *, lock: bool = False, reuse_existing: bool = True) -> None
```

Load or reuse the reranker model engine when configured.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str \| None` | `yes` | `none` | Required positional or keyword input. |
| `lock` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. |
| `reuse_existing` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `load_reranker_model` calls `RerankEngine`, `_rerank_engine.load`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L3174-L3199](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3174-L3199).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.load_model" markdown="1">
<summary><code>vllm_mlx.server.load_model</code> · function</summary>

```python
vllm_mlx.server.load_model(model_name: str, use_batching: bool = False, scheduler_config = None, stream_interval: int = 1, max_tokens: int = 32768, max_request_tokens: int = 32768, force_mllm: bool = False, gpu_memory_utilization: float = 0.9, served_model_name: str | None = None, trust_remote_code: bool = False, mtp: bool = False, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str = None, mllm_draft_model: str | None = None, mllm_draft_kind: str | None = None, mllm_draft_block_size: int | None = None, warm_prompts_path: str | None = None, auto_unload_idle_seconds: float = 0.0, lazy_load_model: bool = False) -> not annotated
```

Load a model (auto-detects MLLM vs LLM).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `yes` | `none` | HuggingFace model name or local path |
| `use_batching` | `bool` | `no` | `False` | Use continuous batching (BatchedEngine) vs simple mode (SimpleEngine) |
| `scheduler_config` | `not annotated` | `no` | `None` | Scheduler config for batched mode |
| `stream_interval` | `int` | `no` | `1` | Tokens to batch before streaming (batched mode only) |
| `max_tokens` | `int` | `no` | `32768` | Default max tokens for generation |
| `max_request_tokens` | `int` | `no` | `32768` | Maximum max_tokens accepted from API clients |
| `force_mllm` | `bool` | `no` | `False` | Force loading as MLLM even if not auto-detected |
| `gpu_memory_utilization` | `float` | `no` | `0.9` | Optional positional or keyword input; defaults to `0.9`. |
| `served_model_name` | `str \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `trust_remote_code` | `bool` | `no` | `False` | Allow HuggingFace remote code execution during model/tokenizer loading |
| `mtp` | `bool` | `no` | `False` | Enable native MTP speculative decoding (SimpleEngine only) |
| `prefill_step_size` | `int` | `no` | `2048` | Chunk size for prompt prefill processing (default: 2048) |
| `specprefill_enabled` | `bool` | `no` | `False` | Enable SpecPrefill (SimpleEngine only) |
| `specprefill_threshold` | `int` | `no` | `8192` | Minimum suffix tokens to trigger SpecPrefill (default: 8192) |
| `specprefill_keep_pct` | `float` | `no` | `0.3` | Fraction of tokens to keep (default: 0.3) |
| `specprefill_backbone_pct` | `float` | `no` | `0.0` | Fraction of chunks reserved for evenly spaced coverage |
| `specprefill_draft_model` | `str` | `no` | `None` | Path to small draft model for SpecPrefill scoring |
| `mllm_draft_model` | `str \| None` | `no` | `None` | Optional MLLM speculative draft/assistant model path. |
| `mllm_draft_kind` | `str \| None` | `no` | `None` | Optional mlx-vlm draft kind, for example "mtp". |
| `mllm_draft_block_size` | `int \| None` | `no` | `None` | Optional speculative block size passed to mlx-vlm. |
| `warm_prompts_path` | `str \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `auto_unload_idle_seconds` | `float` | `no` | `0.0` | Idle time before auto-unloading the main model. When non-zero, the main model is managed through lifecycle residency instead of being loaded immediately in this function. |
| `lazy_load_model` | `bool` | `no` | `False` | When lifecycle residency is enabled, defer the first resident load until the first request instead of FastAPI lifespan startup. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `load_model` calls `ValueError`, `RuntimeError`, `getattr`, `isinstance`; can raise `ValueError`, `RuntimeError`; returns `None`.
Directly raised exceptions: `ValueError`, `RuntimeError`.

[View source #L3202-L3431](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3202-L3431).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.load_model_registry" markdown="1">
<summary><code>vllm_mlx.server.load_model_registry</code> · function</summary>

```python
vllm_mlx.server.load_model_registry(config_path: str, *, defaults: RegistryServeDefaults) -> None
```

Load a registry-backed model manager from YAML configuration.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `config_path` | `str` | `yes` | `none` | Required positional or keyword input. |
| `defaults` | `RegistryServeDefaults` | `yes` | `none` | Required keyword-only input. |

**Returns**

- Type: `None`

**Exceptions and behavior**

Function `load_model_registry` calls `load_registry_config`, `ModelManager`, `logger.info`, `len`.
No direct `raise` statement appears in this definition.

[View source #L3434-L3457](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3434-L3457).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.get_usage" markdown="1">
<summary><code>vllm_mlx.server.get_usage</code> · function</summary>

```python
vllm_mlx.server.get_usage(output: GenerationOutput) -> Usage
```

Extract usage metrics from GenerationOutput.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `output` | `GenerationOutput` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `Usage`
- Direct return expressions: `Usage(prompt_tokens=total_prompt_tokens, completion_tokens=total_completion_tokens, total_tokens=total_prompt_tokens + …`

**Exceptions and behavior**

Function `get_usage` calls `hasattr`, `Usage`; returns `Usage(prompt_tokens=total_prompt_tokens, completion_tokens=total_completion_tokens, total_tokens=total_prompt_tokens + …`.
No direct `raise` statement appears in this definition.

[View source #L3460-L3472](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3460-L3472).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.metrics" markdown="1">
<summary><code>vllm_mlx.server.metrics</code> · function</summary>

```python
async vllm_mlx.server.metrics() -> not annotated
```

Prometheus scrape endpoint (disabled by default).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(content=payload, headers={'Content-Type': content_type})`

**Exceptions and behavior**

Function `metrics` calls `HTTPException`, `_metrics.render_metrics`, `Response`; can raise `HTTPException`; returns `Response(content=payload, headers={'Content-Type': content_type})`.
Directly raised exceptions: `HTTPException`.

[View source #L3476-L3485](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3476-L3485).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.health" markdown="1">
<summary><code>vllm_mlx.server.health</code> · function</summary>

```python
async vllm_mlx.server.health() -> not annotated
```

Health check endpoint.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `payload`

**Exceptions and behavior**

Function `health` calls `sum`, `_mcp_manager.get_server_status`, `len`, `_mcp_manager.get_all_tools`; returns `payload`.
No direct `raise` statement appears in this definition.

[View source #L3489-L3544](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3489-L3544).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.status" markdown="1">
<summary><code>vllm_mlx.server.status</code> · function</summary>

```python
async vllm_mlx.server.status() -> not annotated
```

Real-time status with per-request details for debugging and monitoring.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'status': 'running', 'model_manager': {'memory_budget_gb': round(_model_manager.memory_budget_bytes / 1024 ** 3, 2), '…`; `{'status': 'not_loaded', 'model': _model_name, 'residency': lifecycle, 'requests': []}`; `{'status': 'running' if stats.get('running') else 'stopped', 'model': _model_name, 'residency': lifecycle, 'uptime_s': …`

**Exceptions and behavior**

Function `status` calls `round`, `_model_manager.list_models`, `_public_lifecycle_status`, `_get_lifecycle_status`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3548-L3597](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3548-L3597).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.cache_stats" markdown="1">
<summary><code>vllm_mlx.server.cache_stats</code> · function</summary>

```python
async vllm_mlx.server.cache_stats() -> not annotated
```

Get cache statistics for debugging and monitoring.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'engine_cache': engine_cache, 'multimodal_kv_cache': get_multimodal_kv_cache_stats(), 'pixel_values_cache': get_pixel_…`; `{'engine_cache': engine_cache, 'error': 'Cache stats not available (mlx_vlm not loaded)'}`

**Exceptions and behavior**

Function `cache_stats` calls `hasattr`, `_engine.get_cache_stats`, `get_multimodal_kv_cache_stats`, `get_pixel_values_cache_stats`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3601-L3627](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3601-L3627).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.clear_cache" markdown="1">
<summary><code>vllm_mlx.server.clear_cache</code> · function</summary>

```python
async vllm_mlx.server.clear_cache() -> not annotated
```

Clear all caches.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'status': 'cleared', 'engine_cache': cleared_engine, 'caches': ['multimodal_kv', 'pixel_values', 'pil_image']}`; `{'status': 'cleared', 'engine_cache': cleared_engine, 'error': 'Cache clear not available (mlx_vlm not loaded)'}`

**Exceptions and behavior**

Function `clear_cache` calls `hasattr`, `_engine.clear_runtime_caches`, `logger.warning`, `str`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3631-L3659](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3631-L3659).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.clear_prefix_cache" markdown="1">
<summary><code>vllm_mlx.server.clear_prefix_cache</code> · function</summary>

```python
async vllm_mlx.server.clear_prefix_cache() -> not annotated
```

Clear the text prefix cache used for KV reuse in continuous batching.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'status': 'no_engine'}`; `{'status': status, 'rewarm_scheduled': rewarm_scheduled}`

**Exceptions and behavior**

Function `clear_prefix_cache` calls `hasattr`, `_engine.clear_prefix_cache`, `logger.warning`, `_sanitize_log_text`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L3663-L3713](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3663-L3713).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.clear_prefix_cache._rewarm" markdown="1">
<summary><code>vllm_mlx.server.clear_prefix_cache._rewarm</code> · nested function</summary>

```python
async vllm_mlx.server.clear_prefix_cache._rewarm() -> not annotated
```

Nested Function `clear_prefix_cache._rewarm` calls `load_warmup_file`, `warm_prefix_cache`, `logger.info`, `logger.warning`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Nested Function `clear_prefix_cache._rewarm` calls `load_warmup_file`, `warm_prefix_cache`, `logger.info`, `logger.warning`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L3688-L3707](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3688-L3707).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.cancel_request" markdown="1">
<summary><code>vllm_mlx.server.cancel_request</code> · function</summary>

```python
async vllm_mlx.server.cancel_request(request_id: str) -> not annotated
```

Cancel an active or queued request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'object': 'request.cancel', 'id': request_id, 'cancelled': True, 'model': _model_name}`

**Exceptions and behavior**

Function `cancel_request` calls `get_engine`, `engine.abort_request`, `logger.exception`, `HTTPException`; awaits asynchronous work; can raise `HTTPException`; returns `{'object': 'request.cancel', 'id': request_id, 'cancelled': True, 'model': _model_name}`.
Directly raised exceptions: `HTTPException`.

[View source #L3720-L3747](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3720-L3747).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.delete_request" markdown="1">
<summary><code>vllm_mlx.server.delete_request</code> · function</summary>

```python
async vllm_mlx.server.delete_request(request_id: str) -> not annotated
```

OpenAI-style alias for cancelling an active or queued request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `await cancel_request(request_id)`

**Exceptions and behavior**

Function `delete_request` calls `cancel_request`; awaits asynchronous work; returns `await cancel_request(request_id)`.
No direct `raise` statement appears in this definition.

[View source #L3754-L3756](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3754-L3756).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.list_models" markdown="1">
<summary><code>vllm_mlx.server.list_models</code> · function</summary>

```python
async vllm_mlx.server.list_models() -> ModelsResponse
```

List available models.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `ModelsResponse`
- Direct return expressions: `ModelsResponse(data=models)`

**Exceptions and behavior**

Function `list_models` calls `models.extend`, `ModelInfo`, `_model_manager.list_models`, `models.append`; returns `ModelsResponse(data=models)`.
No direct `raise` statement appears in this definition.

[View source #L3760-L3775](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3760-L3775).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_embeddings" markdown="1">
<summary><code>vllm_mlx.server.create_embeddings</code> · function</summary>

```python
async vllm_mlx.server.create_embeddings(request: EmbeddingRequest) -> EmbeddingResponse
```

Create embeddings for the given input text(s).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `EmbeddingRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `EmbeddingResponse`
- Direct return expressions: `response`

**Exceptions and behavior**

Function `create_embeddings` calls `_metrics.track_inference`, `resolve_embedding_model_name`, `load_embedding_model`, `isinstance`; can raise `HTTPException`; returns `response`.
Directly raised exceptions: `HTTPException`.

[View source #L3787-L3908](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3787-L3908).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.rerank_documents" markdown="1">
<summary><code>vllm_mlx.server.rerank_documents</code> · function</summary>

```python
async vllm_mlx.server.rerank_documents(request: RerankRequest) -> RerankResponse
```

Rerank documents against a query using a cross-encoder model.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `RerankRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `RerankResponse`
- Direct return expressions: `RerankResponse(model=model_name, results=results, usage=RerankUsage(total_tokens=total_tokens))`

**Exceptions and behavior**

Function `rerank_documents` calls `HTTPException`, `request.query.strip`, `len`, `isinstance`; awaits asynchronous work; can raise `HTTPException`; returns `RerankResponse(model=model_name, results=results, usage=RerankUsage(total_tokens=total_tokens))`.
Directly raised exceptions: `HTTPException`.

[View source #L3920-L4038](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3920-L4038).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.list_mcp_tools" markdown="1">
<summary><code>vllm_mlx.server.list_mcp_tools</code> · function</summary>

```python
async vllm_mlx.server.list_mcp_tools() -> MCPToolsResponse
```

List all available MCP tools.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `MCPToolsResponse`
- Direct return expressions: `MCPToolsResponse(tools=[], count=0)`; `MCPToolsResponse(tools=tools, count=len(tools))`

**Exceptions and behavior**

Function `list_mcp_tools` calls `MCPToolsResponse`, `_mcp_manager.get_all_tools`, `tools.append`, `MCPToolInfo`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4047-L4063](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4047-L4063).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.list_mcp_servers" markdown="1">
<summary><code>vllm_mlx.server.list_mcp_servers</code> · function</summary>

```python
async vllm_mlx.server.list_mcp_servers() -> MCPServersResponse
```

Get status of all MCP servers.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `MCPServersResponse`
- Direct return expressions: `MCPServersResponse(servers=[])`; `MCPServersResponse(servers=servers)`

**Exceptions and behavior**

Function `list_mcp_servers` calls `MCPServersResponse`, `_mcp_manager.get_server_status`, `servers.append`, `MCPServerInfo`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4067-L4084](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4067-L4084).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.execute_mcp_tool" markdown="1">
<summary><code>vllm_mlx.server.execute_mcp_tool</code> · function</summary>

```python
async vllm_mlx.server.execute_mcp_tool(request: MCPExecuteRequest) -> MCPExecuteResponse
```

Execute an MCP tool.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `MCPExecuteRequest` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `MCPExecuteResponse`
- Direct return expressions: `MCPExecuteResponse(tool_name=result.tool_name, content=result.content, is_error=result.is_error, error_message=result.e…`

**Exceptions and behavior**

Function `execute_mcp_tool` calls `HTTPException`, `ToolExecutor`, `uuid.uuid4`, `_mcp_executor.execute_tool_calls`; awaits asynchronous work; can raise `HTTPException`; returns `MCPExecuteResponse(tool_name=result.tool_name, content=result.content, is_error=result.is_error, error_message=result.e…`.
Directly raised exceptions: `HTTPException`.

[View source #L4088-L4117](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4088-L4117).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_transcription" markdown="1">
<summary><code>vllm_mlx.server.create_transcription</code> · function</summary>

```python
async vllm_mlx.server.create_transcription(file: UploadFile, model: str = 'whisper-large-v3', language: str | None = None, response_format: str = 'json') -> not annotated
```

Transcribe audio to text (OpenAI Whisper API compatible).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `file` | `UploadFile` | `yes` | `none` | Required positional or keyword input. |
| `model` | `str` | `no` | `'whisper-large-v3'` | Optional positional or keyword input; defaults to `'whisper-large-v3'`. |
| `language` | `str \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `response_format` | `str` | `no` | `'json'` | Optional positional or keyword input; defaults to `'json'`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `result.text`; `{'text': result.text, 'language': result.language, 'duration': result.duration}`

**Exceptions and behavior**

Function `create_transcription` calls `_metrics.track_inference`, `resolve_stt_model_name`, `STTEngine`, `_stt_engine.load`; awaits asynchronous work; can raise `HTTPException`; has 2 explicit return paths.
Directly raised exceptions: `HTTPException`.

[View source #L4130-L4196](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4130-L4196).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_speech" markdown="1">
<summary><code>vllm_mlx.server.create_speech</code> · function</summary>

```python
async vllm_mlx.server.create_speech(model: str = 'kokoro', input: str = '', voice: str = 'af_heart', speed: float = 1.0, response_format: str = 'wav') -> not annotated
```

Generate speech from text (OpenAI TTS API compatible).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `str` | `no` | `'kokoro'` | Optional positional or keyword input; defaults to `'kokoro'`. |
| `input` | `str` | `no` | `''` | Optional positional or keyword input; defaults to `''`. |
| `voice` | `str` | `no` | `'af_heart'` | Optional positional or keyword input; defaults to `'af_heart'`. |
| `speed` | `float` | `no` | `1.0` | Optional positional or keyword input; defaults to `1.0`. |
| `response_format` | `str` | `no` | `'wav'` | Optional positional or keyword input; defaults to `'wav'`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(content=audio_bytes, media_type=content_type)`

**Exceptions and behavior**

Function `create_speech` calls `_metrics.track_inference`, `resolve_tts_model_name`, `validate_tts_input_length`, `TTSEngine`; can raise `HTTPException`; returns `Response(content=audio_bytes, media_type=content_type)`.
Directly raised exceptions: `HTTPException`.

[View source #L4200-L4254](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4200-L4254).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.list_voices" markdown="1">
<summary><code>vllm_mlx.server.list_voices</code> · function</summary>

```python
async vllm_mlx.server.list_voices(model: str = 'kokoro') -> not annotated
```

List available voices for a TTS model.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model` | `str` | `no` | `'kokoro'` | Optional positional or keyword input; defaults to `'kokoro'`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'voices': KOKORO_VOICES}`; `{'voices': CHATTERBOX_VOICES}`; `{'voices': ['default']}`

**Exceptions and behavior**

Function `list_voices` calls `model.lower`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4258-L4267](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4258-L4267).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._ensure_sse_terminal" markdown="1">
<summary><code>vllm_mlx.server._ensure_sse_terminal</code> · function</summary>

```python
async vllm_mlx.server._ensure_sse_terminal(generator: AsyncIterator[str], terminal_frame: str) -> AsyncIterator[str]
```

Guarantee that *terminal_frame* is emitted exactly once at the end of *generator*, even if the generator raises mid-stream.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `generator` | `AsyncIterator[str]` | `yes` | `none` | Required positional or keyword input. |
| `terminal_frame` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `_ensure_sse_terminal` calls `logger.error`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L4275-L4296](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4275-L4296).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._find_uvicorn_cycle" markdown="1">
<summary><code>vllm_mlx.server._find_uvicorn_cycle</code> · function</summary>

```python
vllm_mlx.server._find_uvicorn_cycle(obj, depth = 0, visited = None) -> not annotated
```

Walk through middleware wrappers to find uvicorn's RequestResponseCycle.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `obj` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `depth` | `not annotated` | `no` | `0` | Optional positional or keyword input; defaults to `0`. |
| `visited` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`; `obj`; `result`

**Exceptions and behavior**

Function `_find_uvicorn_cycle` calls `set`, `id`, `visited.add`, `hasattr`; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4299-L4346](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4299-L4346).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._is_client_disconnected" markdown="1">
<summary><code>vllm_mlx.server._is_client_disconnected</code> · function</summary>

```python
vllm_mlx.server._is_client_disconnected(raw_request: Request) -> bool
```

Reliable client disconnect check.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `bool`
- Direct return expressions: `True`; `False`

**Exceptions and behavior**

Function `_is_client_disconnected` calls `getattr`, `_find_uvicorn_cycle`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4349-L4374](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4349-L4374).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._disconnect_guard" markdown="1">
<summary><code>vllm_mlx.server._disconnect_guard</code> · function</summary>

```python
async vllm_mlx.server._disconnect_guard(generator: AsyncIterator[str], raw_request: Request, poll_interval: float = 0.5, heartbeat_interval: float = 5.0, cleanup = None, timeout: float | None = None) -> AsyncIterator[str]
```

Wrap streaming generator to abort on client disconnect.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `generator` | `AsyncIterator[str]` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |
| `poll_interval` | `float` | `no` | `0.5` | Optional positional or keyword input; defaults to `0.5`. |
| `heartbeat_interval` | `float` | `no` | `5.0` | Optional positional or keyword input; defaults to `5.0`. |
| `cleanup` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `timeout` | `float \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `_disconnect_guard` calls `_time.monotonic`, `logger.info`, `generator.__aiter__`, `asyncio.create_task`; awaits asynchronous work; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L4377-L4546](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4377-L4546).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._disconnect_guard._elapsed" markdown="1">
<summary><code>vllm_mlx.server._disconnect_guard._elapsed</code> · nested function</summary>

```python
vllm_mlx.server._disconnect_guard._elapsed() -> not annotated
```

Nested Function `_disconnect_guard._elapsed` calls `_time.monotonic`; returns `f'{_time.monotonic() - _t0:.1f}s'`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `f'{_time.monotonic() - _t0:.1f}s'`

**Exceptions and behavior**

Nested Function `_disconnect_guard._elapsed` calls `_time.monotonic`; returns `f'{_time.monotonic() - _t0:.1f}s'`.
No direct `raise` statement appears in this definition.

[View source #L4407-L4408](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4407-L4408).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._disconnect_guard._wait_disconnect" markdown="1">
<summary><code>vllm_mlx.server._disconnect_guard._wait_disconnect</code> · nested function</summary>

```python
async vllm_mlx.server._disconnect_guard._wait_disconnect() -> not annotated
```

Nested Function `_disconnect_guard._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_elapsed`; awaits asynchronous work; returns `None`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `_disconnect_guard._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_elapsed`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L4417-L4429](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4417-L4429).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._disconnect_guard._deferred_generator_close" markdown="1">
<summary><code>vllm_mlx.server._disconnect_guard._deferred_generator_close</code> · nested function</summary>

```python
async vllm_mlx.server._disconnect_guard._deferred_generator_close() -> not annotated
```

Nested Function `_disconnect_guard._deferred_generator_close` calls `asyncio.sleep`, `_gen_to_close.aclose`, `logger.debug`, `type`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Nested Function `_disconnect_guard._deferred_generator_close` calls `asyncio.sleep`, `_gen_to_close.aclose`, `logger.debug`, `type`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L4528-L4536](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4528-L4536).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._wait_with_disconnect" markdown="1">
<summary><code>vllm_mlx.server._wait_with_disconnect</code> · function</summary>

```python
async vllm_mlx.server._wait_with_disconnect(coro, raw_request: Request, timeout: float, poll_interval: float = 0.5, timeout_detail_seconds: float | None = None, cleanup_result = None) -> not annotated
```

Run a coroutine with both timeout and client disconnect detection.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `coro` | `not annotated` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |
| `timeout` | `float` | `yes` | `none` | Required positional or keyword input. |
| `poll_interval` | `float` | `no` | `0.5` | Optional positional or keyword input; defaults to `0.5`. |
| `timeout_detail_seconds` | `float \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `cleanup_result` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`; `task.result()`

**Exceptions and behavior**

Function `_wait_with_disconnect` calls `_time.monotonic`, `asyncio.ensure_future`, `asyncio.create_task`, `_wait_disconnect`; awaits asynchronous work; can raise `HTTPException`; has 2 explicit return paths.
Directly raised exceptions: `HTTPException`.

[View source #L4549-L4638](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4549-L4638).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._wait_with_disconnect._wait_disconnect" markdown="1">
<summary><code>vllm_mlx.server._wait_with_disconnect._wait_disconnect</code> · nested function</summary>

```python
async vllm_mlx.server._wait_with_disconnect._wait_disconnect() -> not annotated
```

Nested Function `_wait_with_disconnect._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_time.monotonic`; awaits asynchronous work; returns `None`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `_wait_with_disconnect._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_time.monotonic`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L4569-L4581](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4569-L4581).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._start_request_budget" markdown="1">
<summary><code>vllm_mlx.server._start_request_budget</code> · function</summary>

```python
vllm_mlx.server._start_request_budget(timeout: float | None) -> tuple[float, float]
```

Return the total timeout and absolute deadline for a request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `timeout` | `float \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `tuple[float, float]`
- Direct return expressions: `(total_timeout, time.monotonic() + total_timeout)`

**Exceptions and behavior**

Function `_start_request_budget` calls `time.monotonic`; returns `(total_timeout, time.monotonic() + total_timeout)`.
No direct `raise` statement appears in this definition.

[View source #L4641-L4644](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4641-L4644).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._remaining_request_timeout" markdown="1">
<summary><code>vllm_mlx.server._remaining_request_timeout</code> · function</summary>

```python
vllm_mlx.server._remaining_request_timeout(total_timeout: float, deadline: float) -> float
```

Compute remaining request budget or raise the standard timeout error.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `total_timeout` | `float` | `yes` | `none` | Required positional or keyword input. |
| `deadline` | `float` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `float`
- Direct return expressions: `remaining`

**Exceptions and behavior**

Function `_remaining_request_timeout` calls `time.monotonic`, `HTTPException`; can raise `HTTPException`; returns `remaining`.
Directly raised exceptions: `HTTPException`.

[View source #L4647-L4655](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4647-L4655).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._acquire_default_engine_for_request" markdown="1">
<summary><code>vllm_mlx.server._acquire_default_engine_for_request</code> · function</summary>

```python
async vllm_mlx.server._acquire_default_engine_for_request(raw_request: Request, *, total_timeout: float, deadline: float, count_activity: bool = True, model: str | None = None) -> BaseEngine | None
```

Acquire the engine for a request, using the model registry when active.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |
| `total_timeout` | `float` | `yes` | `none` | Required keyword-only input. |
| `deadline` | `float` | `yes` | `none` | Required keyword-only input. |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |
| `model` | `str \| None` | `no` | `None` | Optional keyword-only input; defaults to `None`. |

**Returns**

- Type: `BaseEngine | None`
- Direct return expressions: `await _registry_acquire()`; `await _wait_with_disconnect(_registry_acquire(), raw_request, timeout=_remaining_request_timeout(total_timeout, deadlin…`; `await acquire_coro`; `await _wait_with_disconnect(acquire_coro, raw_request, timeout=_remaining_request_timeout(total_timeout, deadline), tim…`

**Exceptions and behavior**

Function `_acquire_default_engine_for_request` calls `_registry_acquire`, `_wait_with_disconnect`, `_remaining_request_timeout`, `_acquire_default_engine`; awaits asynchronous work; has 4 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4661-L4719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4661-L4719).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._acquire_default_engine_for_request._registry_acquire" markdown="1">
<summary><code>vllm_mlx.server._acquire_default_engine_for_request._registry_acquire</code> · nested function</summary>

```python
async vllm_mlx.server._acquire_default_engine_for_request._registry_acquire() -> not annotated
```

Nested Function `_acquire_default_engine_for_request._registry_acquire` calls `_acquire_request_model`, `id`; awaits asynchronous work; returns `ctx.engine`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`
- Direct return expressions: `ctx.engine`

**Exceptions and behavior**

Nested Function `_acquire_default_engine_for_request._registry_acquire` calls `_acquire_request_model`, `id`; awaits asynchronous work; returns `ctx.engine`.
No direct `raise` statement appears in this definition.

[View source #L4681-L4685](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4681-L4685).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._acquire_default_engine_for_request._registry_cleanup" markdown="1">
<summary><code>vllm_mlx.server._acquire_default_engine_for_request._registry_cleanup</code> · nested function</summary>

```python
async vllm_mlx.server._acquire_default_engine_for_request._registry_cleanup(_result) -> not annotated
```

Nested Function `_acquire_default_engine_for_request._registry_cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`; awaits asynchronous work.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `_result` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Nested Function `_acquire_default_engine_for_request._registry_cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L4687-L4690](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4687-L4690).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._release_engine_for_request" markdown="1">
<summary><code>vllm_mlx.server._release_engine_for_request</code> · function</summary>

```python
async vllm_mlx.server._release_engine_for_request(raw_request: Request | None, *, count_activity: bool = True) -> None
```

Release the engine acquired for this request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `raw_request` | `Request \| None` | `yes` | `none` | Required positional or keyword input. |
| `count_activity` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. |

**Returns**

- Type: `None`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `_release_engine_for_request` calls `_active_request_contexts.pop`, `id`, `ctx.release`, `_release_default_engine`; awaits asynchronous work; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L4722-L4737](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4722-L4737).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._make_release_cleanup" markdown="1">
<summary><code>vllm_mlx.server._make_release_cleanup</code> · function</summary>

```python
vllm_mlx.server._make_release_cleanup(raw_request: Request | None) -> not annotated
```

Return a cleanup callable suitable for ``_disconnect_guard``.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `raw_request` | `Request \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `_cleanup`; `_release_default_engine`

**Exceptions and behavior**

Function `_make_release_cleanup` has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4740-L4752](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4740-L4752).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._make_release_cleanup._cleanup" markdown="1">
<summary><code>vllm_mlx.server._make_release_cleanup._cleanup</code> · nested function</summary>

```python
async vllm_mlx.server._make_release_cleanup._cleanup() -> not annotated
```

Nested Function `_make_release_cleanup._cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`, `_release_default_engine`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Nested Function `_make_release_cleanup._cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`, `_release_default_engine`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L4744-L4749](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4744-L4749).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_completion" markdown="1">
<summary><code>vllm_mlx.server.create_completion</code> · function</summary>

```python
async vllm_mlx.server.create_completion(request: CompletionRequest, raw_request: Request) -> not annotated
```

Create a text completion.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `CompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(status_code=499)`; `response`; `CompletionResponse(model=_response_model_name(request.model), choices=choices, usage=Usage(prompt_tokens=total_prompt_t…`

**Exceptions and behavior**

Function `create_completion` calls `_validate_model_name`, `_resolve_request_max_tokens`, `_metrics.track_inference`, `isinstance`; awaits asynchronous work; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4763-L4909](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4763-L4909).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_chat_completion" markdown="1">
<summary><code>vllm_mlx.server.create_chat_completion</code> · function</summary>

```python
async vllm_mlx.server.create_chat_completion(request: ChatCompletionRequest, raw_request: Request) -> not annotated
```

Create a chat completion (supports multimodal content for VLM models).

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(status_code=499)`; `response`; `ChatCompletionResponse(model=_response_model_name(request.model), choices=[ChatCompletionChoice(message=AssistantMessag…`

**Exceptions and behavior**

Function `create_chat_completion` calls `_validate_model_name`, `_resolve_request_max_tokens`, `_metrics.track_inference`, `_start_request_budget`; awaits asynchronous work; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L4916-L5114](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4916-L5114).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._normalize_messages" markdown="1">
<summary><code>vllm_mlx.server._normalize_messages</code> · function</summary>

```python
vllm_mlx.server._normalize_messages(messages: list[dict]) -> list[dict]
```

Normalize message roles and merge consecutive same-role messages.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list[dict]` | `yes` | `none` | List of message dicts with 'role' and 'content' keys. |

**Returns**

- Type: `list[dict]`
- Direct return expressions: `messages`; `merged`

**Exceptions and behavior**

Function `_normalize_messages` calls `messages[0].copy`, `_ROLE_MAP.get`, `isinstance`, `prev.get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5117-L5172](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5117-L5172).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_engine_tokenizer" markdown="1">
<summary><code>vllm_mlx.server._get_engine_tokenizer</code> · function</summary>

```python
vllm_mlx.server._get_engine_tokenizer(engine) -> object | None
```

Return the tokenizer backing ``engine``, if exposed.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `object | None`
- Direct return expressions: `tok`; `None`

**Exceptions and behavior**

Function `_get_engine_tokenizer` calls `getattr`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5175-L5187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5175-L5187).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_response" markdown="1">
<summary><code>vllm_mlx.server.create_response</code> · function</summary>

```python
async vllm_mlx.server.create_response(request: ResponsesRequest, raw_request: Request) -> not annotated
```

Create a Responses API response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `ResponsesRequest` | `yes` | `none` | Required positional or keyword input. |
| `raw_request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `StreamingResponse(_disconnect_guard(_stream_responses_request(request), raw_request), media_type='text/event-stream')`; `Response(status_code=499)`; `response_object`

**Exceptions and behavior**

Function `create_response` calls `_responses_request_to_chat_request`, `_validate_remote_media_urls`, `StreamingResponse`, `_disconnect_guard`; awaits asynchronous work; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5194-L5214](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5194-L5214).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._get_forced_tool_name" markdown="1">
<summary><code>vllm_mlx.server._get_forced_tool_name</code> · function</summary>

```python
vllm_mlx.server._get_forced_tool_name(tool_choice) -> str | None
```

Extract forced tool name from tool_choice, if any.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tool_choice` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str | None`
- Direct return expressions: `None`; `func.get('name')`

**Exceptions and behavior**

Function `_get_forced_tool_name` calls `isinstance`, `tool_choice.get`, `func.get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5217-L5230](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5217-L5230).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._apply_forced_tool_choice" markdown="1">
<summary><code>vllm_mlx.server._apply_forced_tool_choice</code> · function</summary>

```python
vllm_mlx.server._apply_forced_tool_choice(tool_choice, tools, messages, chat_kwargs = None) -> not annotated
```

Apply forced tool_choice by filtering tools and injecting instructions.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tool_choice` | `not annotated` | `yes` | `none` | The tool_choice value from the request |
| `tools` | `not annotated` | `yes` | `none` | List of converted tools for the template |
| `messages` | `not annotated` | `yes` | `none` | The message list (will be copied if modified) |
| `chat_kwargs` | `not annotated` | `no` | `None` | Optional dict to modify (e.g. disable thinking) |

**Returns**

- Type: `not annotated`
- Direct return expressions: `(tools, messages)`

**Exceptions and behavior**

Function `_apply_forced_tool_choice` calls `_get_forced_tool_name`, `_tool_name`, `ValueError`, `_inject_json_instruction`; can raise `ValueError`; returns `(tools, messages)`.
Directly raised exceptions: `ValueError`.

[View source #L5233-L5279](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5233-L5279).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._tool_name" markdown="1">
<summary><code>vllm_mlx.server._tool_name</code> · function</summary>

```python
vllm_mlx.server._tool_name(tool: dict) -> str | None
```

Extract function name from a tool definition dict.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `tool` | `dict` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str | None`
- Direct return expressions: `func.get('name')`; `None`

**Exceptions and behavior**

Function `_tool_name` calls `tool.get`, `isinstance`, `func.get`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5282-L5287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5282-L5287).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._inject_json_instruction" markdown="1">
<summary><code>vllm_mlx.server._inject_json_instruction</code> · function</summary>

```python
vllm_mlx.server._inject_json_instruction(messages: list, instruction: str) -> list
```

Inject JSON instruction into messages.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `messages` | `list` | `yes` | `none` | Required positional or keyword input. |
| `instruction` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `list`
- Direct return expressions: `messages`

**Exceptions and behavior**

Function `_inject_json_instruction` calls `list`, `enumerate`, `isinstance`, `msg.get`; returns `messages`.
No direct `raise` statement appears in this definition.

[View source #L5290-L5319](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5290-L5319).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._convert_anthropic_stop_reason" markdown="1">
<summary><code>vllm_mlx.server._convert_anthropic_stop_reason</code> · function</summary>

```python
vllm_mlx.server._convert_anthropic_stop_reason(openai_reason: str | None) -> str
```

Convert OpenAI finish_reason to Anthropic stop_reason.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `openai_reason` | `str \| None` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `str`
- Direct return expressions: `mapping.get(openai_reason or '', 'end_turn')`

**Exceptions and behavior**

Function `_convert_anthropic_stop_reason` calls `mapping.get`; returns `mapping.get(openai_reason or '', 'end_turn')`.
No direct `raise` statement appears in this definition.

[View source #L5327-L5335](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5327-L5335).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._prepare_anthropic_endpoint_invocation" markdown="1">
<summary><code>vllm_mlx.server._prepare_anthropic_endpoint_invocation</code> · function</summary>

```python
vllm_mlx.server._prepare_anthropic_endpoint_invocation(engine: BaseEngine, openai_request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation
```

Prepare Anthropic invocation and convert URL-safety errors to 400s.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `openai_request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `effective_max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `PreparedChatInvocation`
- Direct return expressions: `_prepare_anthropic_invocation(engine, openai_request, effective_max_tokens)`

**Exceptions and behavior**

Function `_prepare_anthropic_endpoint_invocation` calls `_prepare_anthropic_invocation`, `_raise_remote_media_http_error`; returns `_prepare_anthropic_invocation(engine, openai_request, effective_max_tokens)`.
No direct `raise` statement appears in this definition.

[View source #L5338-L5351](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5338-L5351).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_anthropic_message" markdown="1">
<summary><code>vllm_mlx.server.create_anthropic_message</code> · function</summary>

```python
async vllm_mlx.server.create_anthropic_message(request: Request) -> not annotated
```

Anthropic Messages API endpoint.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(status_code=499)`; `response`; `Response(content=anthropic_response.model_dump_json(exclude_none=True), media_type='application/json')`

**Exceptions and behavior**

Function `create_anthropic_message` calls `_metrics.track_inference`, `request.json`, `str`, `request.body`; awaits asynchronous work; has 3 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5357-L5578](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5357-L5578).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.count_anthropic_tokens" markdown="1">
<summary><code>vllm_mlx.server.count_anthropic_tokens</code> · function</summary>

```python
async vllm_mlx.server.count_anthropic_tokens(request: Request) -> not annotated
```

Count tokens for an Anthropic Messages API request.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `request` | `Request` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `Response(status_code=499)`; `{'input_tokens': total_tokens}`

**Exceptions and behavior**

Function `count_anthropic_tokens` calls `request.json`, `body.get`, `isinstance`, `_validate_model_name`; awaits asynchronous work; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L5585-L5666](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5585-L5666).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._emit_content_pieces" markdown="1">
<summary><code>vllm_mlx.server._emit_content_pieces</code> · function</summary>

```python
vllm_mlx.server._emit_content_pieces(pieces: list[tuple[str, str]], current_block_type: str | None, block_index: int) -> tuple[list[str], str | None, int]
```

Emit Anthropic SSE events for content pieces from the think router.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `pieces` | `list[tuple[str, str]]` | `yes` | `none` | List of (block_type, text) from StreamingThinkRouter |
| `current_block_type` | `str \| None` | `yes` | `none` | Current open block type, or None |
| `block_index` | `int` | `yes` | `none` | Current block index |

**Returns**

- Type: `tuple[list[str], str | None, int]`
- Direct return expressions: `(events, current_block_type, block_index)`

**Exceptions and behavior**

Function `_emit_content_pieces` calls `events.append`, `json.dumps`; returns `(events, current_block_type, block_index)`.
No direct `raise` statement appears in this definition.

[View source #L5669-L5719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5669-L5719).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._stream_anthropic_messages" markdown="1">
<summary><code>vllm_mlx.server._stream_anthropic_messages</code> · function</summary>

```python
async vllm_mlx.server._stream_anthropic_messages(engine: BaseEngine, openai_request: ChatCompletionRequest, anthropic_request: AnthropicRequest, prepared: PreparedChatInvocation, metrics_tracker = None) -> AsyncIterator[str]
```

Stream Anthropic Messages API SSE events.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `openai_request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `anthropic_request` | `AnthropicRequest` | `yes` | `none` | Required positional or keyword input. |
| `prepared` | `PreparedChatInvocation` | `yes` | `none` | Required positional or keyword input. |
| `metrics_tracker` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `_stream_anthropic_messages` calls `uuid.uuid4`, `time.perf_counter`, `dict`, `_response_model_name`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L5722-L5995](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5722-L5995).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.stream_completion" markdown="1">
<summary><code>vllm_mlx.server.stream_completion</code> · function</summary>

```python
async vllm_mlx.server.stream_completion(engine: BaseEngine, prompt: str, request: CompletionRequest, max_tokens: int, repetition_penalty: float | None = None, metrics_tracker = None) -> AsyncIterator[str]
```

Stream completion response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |
| `request` | `CompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `max_tokens` | `int` | `yes` | `none` | Required positional or keyword input. |
| `repetition_penalty` | `float \| None` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `metrics_tracker` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `stream_completion` calls `_resolve_temperature`, `_resolve_top_p`, `_resolve_top_k`, `_resolve_min_p`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L6003-L6084](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6003-L6084).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.stream_chat_completion" markdown="1">
<summary><code>vllm_mlx.server.stream_chat_completion</code> · function</summary>

```python
async vllm_mlx.server.stream_chat_completion(engine: BaseEngine, messages: list, request: ChatCompletionRequest, metrics_tracker = None, **kwargs) -> AsyncIterator[str]
```

Stream chat completion response.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `BaseEngine` | `yes` | `none` | Required positional or keyword input. |
| `messages` | `list` | `yes` | `none` | Required positional or keyword input. |
| `request` | `ChatCompletionRequest` | `yes` | `none` | Required positional or keyword input. |
| `metrics_tracker` | `not annotated` | `no` | `None` | Optional positional or keyword input; defaults to `None`. |
| `**kwargs` | `not annotated` | `no` | `none` | Additional variadic keyword inputs accepted by this callable. |

**Returns**

- Type: `AsyncIterator[str]`
- Yields values incrementally.

**Exceptions and behavior**

Function `stream_chat_completion` calls `uuid.uuid4`, `time.perf_counter`, `_stream_request_metadata`, `ChatCompletionChunk`; yields values incrementally.
No direct `raise` statement appears in this definition.

[View source #L6087-L6512](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6087-L6512).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.init_mcp" markdown="1">
<summary><code>vllm_mlx.server.init_mcp</code> · function</summary>

```python
async vllm_mlx.server.init_mcp(config_path: str) -> not annotated
```

Initialize MCP manager from config file.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `config_path` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `init_mcp` calls `load_mcp_config`, `MCPClientManager`, `_mcp_manager.start`, `ToolSandbox`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L6520-L6546](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6520-L6546).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._make_keepalive_http_protocol" markdown="1">
<summary><code>vllm_mlx.server._make_keepalive_http_protocol</code> · function</summary>

```python
vllm_mlx.server._make_keepalive_http_protocol(idle = 10, interval = 5, count = 3) -> not annotated
```

Create a uvicorn HTTP protocol class with aggressive TCP keepalive.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `idle` | `not annotated` | `no` | `10` | Optional positional or keyword input; defaults to `10`. |
| `interval` | `not annotated` | `no` | `5` | Optional positional or keyword input; defaults to `5`. |
| `count` | `not annotated` | `no` | `3` | Optional positional or keyword input; defaults to `3`. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `_KeepaliveProtocol`

**Exceptions and behavior**

Function `_make_keepalive_http_protocol` returns `_KeepaliveProtocol`.
No direct `raise` statement appears in this definition.

[View source #L6554-L6589](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6554-L6589).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol" markdown="1">
<summary><code>vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol</code> · nested class</summary>

```python
vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol()
```

Nested Class `_make_keepalive_http_protocol._KeepaliveProtocol` derives from `_Base` and declares 1 direct member(s).

**Parameters**

This callable has no explicit inputs.

**Returns**

- Constructs: `vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol`

**Exceptions and behavior**

Nested Class `_make_keepalive_http_protocol._KeepaliveProtocol` derives from `_Base` and declares 1 direct member(s).
No direct `raise` statement appears in this definition.

[View source #L6567-L6587](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6567-L6587).

</details>

<details class="api-contract" id="contract-vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol.connection_made" markdown="1">
<summary><code>vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol.connection_made</code> · nested function</summary>

```python
vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol.connection_made(transport) -> not annotated
```

Nested Function `_make_keepalive_http_protocol._KeepaliveProtocol.connection_made` calls `super().connection_made`, `super`, `transport.get_extra_info`, `sock.setsockopt`; returns `None`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `transport` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Nested Function `_make_keepalive_http_protocol._KeepaliveProtocol.connection_made` calls `super().connection_made`, `super`, `transport.get_extra_info`, `sock.setsockopt`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L6568-L6587](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6568-L6587).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.main" markdown="1">
<summary><code>vllm_mlx.server.main</code> · function</summary>

```python
vllm_mlx.server.main() -> not annotated
```

Run the server.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `main` calls `create_parser`, `parser.parse_args`, `_metrics.configure`, `RateLimiter`.
No direct `raise` statement appears in this definition.

[View source #L6597-L6708](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6597-L6708).

</details>

<details class="api-contract" id="contract-vllm_mlx.server.create_parser" markdown="1">
<summary><code>vllm_mlx.server.create_parser</code> · function</summary>

```python
vllm_mlx.server.create_parser() -> argparse.ArgumentParser
```

Create the standalone server CLI parser.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `argparse.ArgumentParser`
- Direct return expressions: `parser`

**Exceptions and behavior**

Function `create_parser` calls `argparse.ArgumentParser`, `parser.add_argument`, `make_positive_int_arg_parser`, `list_parsers`; returns `parser`.
No direct `raise` statement appears in this definition.

[View source #L6711-L6912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6711-L6912).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`_resolve_temperature`](#contract-vllm_mlx.server._resolve_temperature) | function | `_resolve_temperature(request_value: float \| None) -> float` | Resolve temperature: request > CLI default > fallback. | [#L225-L231](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L225-L231) |
| [`_resolve_top_p`](#contract-vllm_mlx.server._resolve_top_p) | function | `_resolve_top_p(request_value: float \| None) -> float` | Resolve top_p: request > CLI default > fallback. | [#L234-L240](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L234-L240) |
| [`_resolve_top_k`](#contract-vllm_mlx.server._resolve_top_k) | function | `_resolve_top_k(request_value: int \| None) -> int` | Resolve top_k: request > CLI default > fallback. | [#L243-L249](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L243-L249) |
| [`_resolve_min_p`](#contract-vllm_mlx.server._resolve_min_p) | function | `_resolve_min_p(request_value: float \| None) -> float` | Resolve min_p: request > CLI default > fallback. | [#L252-L258](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L252-L258) |
| [`_resolve_presence_penalty`](#contract-vllm_mlx.server._resolve_presence_penalty) | function | `_resolve_presence_penalty(request_value: float \| None) -> float` | Resolve presence_penalty: request > CLI default > fallback. | [#L261-L267](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L261-L267) |
| [`_resolve_repetition_penalty`](#contract-vllm_mlx.server._resolve_repetition_penalty) | function | `_resolve_repetition_penalty(request_value: float \| None) -> float` | Resolve repetition_penalty: request > CLI default > fallback. | [#L270-L276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L270-L276) |
| [`_resolve_request_max_tokens`](#contract-vllm_mlx.server._resolve_request_max_tokens) | function | `_resolve_request_max_tokens(requested_value: int \| None) -> int` | Resolve and validate a request's max_tokens budget. | [#L279-L288](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L279-L288) |
| [`_resolve_chat_template_kwargs`](#contract-vllm_mlx.server._resolve_chat_template_kwargs) | function | `_resolve_chat_template_kwargs(request_value: dict[str, object] \| None) -> dict[str, object]` | Resolve chat template kwargs: request > server default > empty dict. | [#L291-L300](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L291-L300) |
| [`PreparedChatInvocation`](#contract-vllm_mlx.server.PreparedChatInvocation) | class | `PreparedChatInvocation(messages: list[dict], chat_kwargs: dict[str, object], response_format: object \| None, json_logits_processor: object \| None, thinking_processor: object \| None = None)` | Fully prepared inputs for a single engine.chat/stream_chat call. | [#L304-L311](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L304-L311) |
| [`_prepare_chat_messages`](#contract-vllm_mlx.server._prepare_chat_messages) | function | `_prepare_chat_messages(engine: BaseEngine, request_messages: list[Message \| dict]) -> tuple[list[dict], list, list, list, bool]` | Normalize messages and collect media once for both stream/non-stream paths. | [#L314-L398](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L314-L398) |
| [`_iter_remote_media_urls`](#contract-vllm_mlx.server._iter_remote_media_urls) | function | `_iter_remote_media_urls(messages: list[Message \| dict]) -> not annotated` | Yield remote media URLs from OpenAI-style multimodal message content. | [#L401-L429](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L401-L429) |
| [`_validate_remote_media_urls`](#contract-vllm_mlx.server._validate_remote_media_urls) | function | `_validate_remote_media_urls(messages: list[Message \| dict]) -> None` | Validate remote media URLs during request preparation. | [#L432-L435](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L432-L435) |
| [`_raise_remote_media_http_error`](#contract-vllm_mlx.server._raise_remote_media_http_error) | function | `_raise_remote_media_http_error(exc: UnsafeRemoteURLError) -> None` | Log internal URL-safety detail while returning a generic client error. | [#L438-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L438-L444) |
| [`_prepare_json_logits_processor`](#contract-vllm_mlx.server._prepare_json_logits_processor) | function | `_prepare_json_logits_processor(engine: BaseEngine, messages: list[dict], response_format: object \| None, *, tools: list \| None, tool_choice: object \| None, log_context: str \| None = None, thinking_model: bool = False) -> tuple[list[dict], object \| None]` | Inject response_format instruction and build constrained decoding processor. | [#L447-L497](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L447-L497) |
| [`_build_thinking_processor`](#contract-vllm_mlx.server._build_thinking_processor) | function | `_build_thinking_processor(engine: BaseEngine, thinking_token_budget: int, *, inner: object \| None = None, prompt_has_think_tag: bool = True) -> object \| None` | Build a ThinkingAwareLogitsProcessor if the tokenizer has think tokens. | [#L500-L554](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L500-L554) |
| [`_resolve_no_final_content_token_limit`](#contract-vllm_mlx.server._resolve_no_final_content_token_limit) | function | `_resolve_no_final_content_token_limit() -> int \| None` | Function `_resolve_no_final_content_token_limit` calls `os.environ.get`, `raw.strip`, `int`, `logger.warning`; has 2 explicit return paths. | [#L557-L568](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L557-L568) |
| [`_generation_metadata`](#contract-vllm_mlx.server._generation_metadata) | function | `_generation_metadata(thinking_processor: object \| None) -> GenerationMetadata \| None` | Function `_generation_metadata` calls `GenerationMetadata`, `getattr`, `bool`; has 2 explicit return paths. | [#L571-L583](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L571-L583) |
| [`_ThinkingAwareLogitsProcessor`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor) | class | `_ThinkingAwareLogitsProcessor(inner, prompt_has_think_tag: bool = False)` | Wrap a ``JSONSchemaLogitsProcessor`` so JSON constraining only activates after the model emits ``</think>``, letting it reason freely first. | [#L586-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L586-L697) |
| [`_ThinkingAwareLogitsProcessor.__init__`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.__init__) | method | `_ThinkingAwareLogitsProcessor.__init__(inner, prompt_has_think_tag: bool = False) -> not annotated` | Method `_ThinkingAwareLogitsProcessor.__init__` updates `self._inner`, `self._active`, `self._in_thinking`, `self._waiting_for_json`. | [#L597-L608](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L597-L608) |
| [`_ThinkingAwareLogitsProcessor._scan_for_json_start`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor._scan_for_json_start) | method | `_ThinkingAwareLogitsProcessor._scan_for_json_start(tokens_list, tokens, logits) -> not annotated` | Scan generated tokens for the first ``{`` or ``[``. | [#L610-L635](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L610-L635) |
| [`_ThinkingAwareLogitsProcessor.__call__`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.__call__) | method | `_ThinkingAwareLogitsProcessor.__call__(tokens, logits) -> not annotated` | Method `_ThinkingAwareLogitsProcessor.__call__` updates `self._base_prompt_len`, `self._in_thinking`, `self._waiting_for_json`, `self._json_scan_offset`; calls `self._inner`, `hasattr`, `tokens.tolist`, `list`; has 3 explicit return paths. | [#L637-L688](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L637-L688) |
| [`_ThinkingAwareLogitsProcessor.schema`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor.schema) | method | `_ThinkingAwareLogitsProcessor.schema() -> not annotated` | Method `_ThinkingAwareLogitsProcessor.schema` returns `self._inner.schema`. | [#L692-L693](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L692-L693) |
| [`_ThinkingAwareLogitsProcessor._disabled`](#contract-vllm_mlx.server._ThinkingAwareLogitsProcessor._disabled) | method | `_ThinkingAwareLogitsProcessor._disabled() -> not annotated` | Method `_ThinkingAwareLogitsProcessor._disabled` returns `self._inner._disabled`. | [#L696-L697](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L696-L697) |
| [`_attach_response_format_logits_processor`](#contract-vllm_mlx.server._attach_response_format_logits_processor) | function | `_attach_response_format_logits_processor(chat_kwargs: dict, json_logits_processor: object) -> object` | Attach response_format constraints and keep thinking disabled. | [#L700-L717](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L700-L717) |
| [`_coerce_logit_bias`](#contract-vllm_mlx.server._coerce_logit_bias) | function | `_coerce_logit_bias(logit_bias: dict[str, float]) -> dict[int, float]` | Function `_coerce_logit_bias` calls `logit_bias.items`, `int`, `float`, `HTTPException`; can raise `HTTPException`; returns `coerced`. | [#L720-L730](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L720-L730) |
| [`_attach_logit_bias_processor`](#contract-vllm_mlx.server._attach_logit_bias_processor) | function | `_attach_logit_bias_processor(chat_kwargs: dict, logit_bias: dict[str, float] \| None) -> not annotated` | Function `_attach_logit_bias_processor` calls `make_logits_processors`, `_coerce_logit_bias`, `chat_kwargs.get`, `list`; returns `None`. | [#L733-L744](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L733-L744) |
| [`_prepare_chat_completion_invocation`](#contract-vllm_mlx.server._prepare_chat_completion_invocation) | function | `_prepare_chat_completion_invocation(engine: BaseEngine, request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation` | Precompute messages, kwargs, and decoding constraints for chat completions. | [#L747-L855](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L747-L855) |
| [`_prepare_anthropic_invocation`](#contract-vllm_mlx.server._prepare_anthropic_invocation) | function | `_prepare_anthropic_invocation(engine: BaseEngine, openai_request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation` | Precompute messages, kwargs, and decoding constraints for Anthropic API. | [#L858-L910](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L858-L910) |
| [`_thinking_disabled`](#contract-vllm_mlx.server._thinking_disabled) | function | `_thinking_disabled(request, chat_kwargs: dict \| None = None) -> bool` | Return True iff thinking is explicitly disabled for this request. | [#L934-L950](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L934-L950) |
| [`_strip_backslash_before_unicode`](#contract-vllm_mlx.server._strip_backslash_before_unicode) | function | `_strip_backslash_before_unicode(obj: object) -> object` | Remove spurious backslashes before non-ASCII chars in JSON string values. | [#L983-L997](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L983-L997) |
| [`_sanitize_log_text`](#contract-vllm_mlx.server._sanitize_log_text) | function | `_sanitize_log_text(value: object, limit: int \| None = None) -> str` | Escape control characters before logging untrusted text. | [#L1000-L1022](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1000-L1022) |
| [`_log_and_raise_internal_error`](#contract-vllm_mlx.server._log_and_raise_internal_error) | function | `_log_and_raise_internal_error(log_prefix: str, exc: Exception, detail: str) -> None` | Log a sanitized exception string and raise a generic 500 response. | [#L1025-L1028](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1025-L1028) |
| [`_raise_engine_busy`](#contract-vllm_mlx.server._raise_engine_busy) | function | `_raise_engine_busy(exc: EngineBusy) -> None` | Translate serialized-engine admission failures into retryable HTTP 503. | [#L1031-L1039](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1031-L1039) |
| [`RequestModelContext`](#contract-vllm_mlx.server.RequestModelContext) | class | `RequestModelContext(model_name: str, engine: BaseEngine, lease: ModelLease \| None = None)` | Request-scoped engine/lease context. | [#L1043-L1056](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1043-L1056) |
| [`RequestModelContext.release`](#contract-vllm_mlx.server.RequestModelContext.release) | method | `async RequestModelContext.release() -> None` | Release the registry lease once, if this context owns one. | [#L1050-L1056](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1050-L1056) |
| [`_list_available_model_names`](#contract-vllm_mlx.server._list_available_model_names) | function | `_list_available_model_names() -> list[str]` | Function `_list_available_model_names` has 2 explicit return paths. | [#L1059-L1062](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1059-L1062) |
| [`_response_model_name`](#contract-vllm_mlx.server._response_model_name) | function | `_response_model_name(request_model: str) -> str` | Return the response model field for single-model or registry mode. | [#L1065-L1067](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1065-L1067) |
| [`_acquire_request_model`](#contract-vllm_mlx.server._acquire_request_model) | function | `async _acquire_request_model(request_model: str) -> RequestModelContext` | Acquire the model/engine that should serve this request. | [#L1070-L1094](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1070-L1094) |
| [`_stream_with_model_context`](#contract-vllm_mlx.server._stream_with_model_context) | function | `async _stream_with_model_context(context: RequestModelContext, stream: AsyncIterator[str]) -> AsyncIterator[str]` | Ensure model leases survive for the full streaming response. | [#L1097-L1106](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1097-L1106) |
| [`_build_tool_parser`](#contract-vllm_mlx.server._build_tool_parser) | function | `_build_tool_parser(engine: BaseEngine \| None) -> not annotated` | Create a fresh tool parser instance for a single request/stream. | [#L1109-L1123](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1109-L1123) |
| [`_build_reasoning_parser`](#contract-vllm_mlx.server._build_reasoning_parser) | function | `_build_reasoning_parser(engine: BaseEngine \| None = None) -> not annotated` | Create a fresh reasoning parser instance for a single request/stream. | [#L1126-L1140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1126-L1140) |
| [`_prepare_streaming_reasoning_parser`](#contract-vllm_mlx.server._prepare_streaming_reasoning_parser) | function | `_prepare_streaming_reasoning_parser(engine: BaseEngine, request: ChatCompletionRequest \| ResponsesRequest \| None, chat_kwargs: dict[str, object], *, allowed: bool = True) -> not annotated` | Build and reset request-local reasoning state when thinking is enabled. | [#L1143-L1156](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1143-L1156) |
| [`_prepare_openai_stream_reasoning_state`](#contract-vllm_mlx.server._prepare_openai_stream_reasoning_state) | function | `_prepare_openai_stream_reasoning_state(engine: BaseEngine, request: ChatCompletionRequest, chat_kwargs: dict[str, object]) -> tuple[object \| None, bool]` | Return request-local reasoning state and the legacy Nemotron marker state. | [#L1159-L1171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1159-L1171) |
| [`_request_tool_definitions`](#contract-vllm_mlx.server._request_tool_definitions) | function | `_request_tool_definitions(request: ChatCompletionRequest) -> list \| None` | Return the request tool schema once for streaming argument coercion. | [#L1174-L1178](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1174-L1178) |
| [`_streaming_json_fence_stripper`](#contract-vllm_mlx.server._streaming_json_fence_stripper) | function | `_streaming_json_fence_stripper(request: ChatCompletionRequest) -> StreamingJsonFenceStripper \| None` | Create a fence stripper only for JSON-constrained streaming responses. | [#L1181-L1191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1181-L1191) |
| [`_get_idle_unload_event`](#contract-vllm_mlx.server._get_idle_unload_event) | function | `_get_idle_unload_event() -> asyncio.Event` | Return the idle-unload gate event, creating it on first use. | [#L1206-L1217](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1206-L1217) |
| [`_invalidate_tool_parser_cache`](#contract-vllm_mlx.server._invalidate_tool_parser_cache) | function | `_invalidate_tool_parser_cache(reason: str \| None = None) -> None` | Drop cached parser state when the serving tokenizer changes. | [#L1220-L1229](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1220-L1229) |
| [`_load_prefix_cache_from_disk`](#contract-vllm_mlx.server._load_prefix_cache_from_disk) | function | `_load_prefix_cache_from_disk(engine: BaseEngine \| None = None) -> None` | Load prefix cache from disk during startup. | [#L1232-L1250](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1232-L1250) |
| [`_save_prefix_cache_to_disk`](#contract-vllm_mlx.server._save_prefix_cache_to_disk) | function | `_save_prefix_cache_to_disk(engine: BaseEngine \| None = None) -> None` | Save prefix cache to disk during shutdown. | [#L1253-L1271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1253-L1271) |
| [`_get_cache_dir`](#contract-vllm_mlx.server._get_cache_dir) | function | `_get_cache_dir() -> str` | Get cache persistence directory based on actual model path. | [#L1274-L1290](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1274-L1290) |
| [`_build_engine`](#contract-vllm_mlx.server._build_engine) | function | `_build_engine(spec: ModelSpec) -> BaseEngine` | Construct an engine instance from a model spec without starting it. | [#L1293-L1323](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1293-L1323) |
| [`_engine_factory`](#contract-vllm_mlx.server._engine_factory) | function | `async _engine_factory(spec: ModelSpec) -> BaseEngine` | Async engine factory used by the residency manager. | [#L1326-L1328](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1326-L1328) |
| [`_run_blocking_engine_cache_io`](#contract-vllm_mlx.server._run_blocking_engine_cache_io) | function | `async _run_blocking_engine_cache_io(io_fn, engine: BaseEngine) -> None` | Run blocking cache persistence off the event loop. | [#L1331-L1350](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1331-L1350) |
| [`_restore_engine_state`](#contract-vllm_mlx.server._restore_engine_state) | function | `async _restore_engine_state(spec: ModelSpec, engine: BaseEngine) -> None` | Restore engine-local state, such as prefix cache, after a cold load. | [#L1353-L1356](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1353-L1356) |
| [`_persist_engine_state`](#contract-vllm_mlx.server._persist_engine_state) | function | `async _persist_engine_state(spec: ModelSpec, engine: BaseEngine) -> None` | Persist engine-local state before an idle unload or shutdown unload. | [#L1359-L1362](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1359-L1362) |
| [`_activate_engine`](#contract-vllm_mlx.server._activate_engine) | function | `_activate_engine(engine: BaseEngine \| None) -> BaseEngine \| None` | Set the global engine pointer and refresh parser-sensitive state. | [#L1365-L1375](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1365-L1375) |
| [`_sync_engine_from_residency`](#contract-vllm_mlx.server._sync_engine_from_residency) | function | `_sync_engine_from_residency() -> BaseEngine \| None` | Sync the global engine pointer from the residency manager state. | [#L1378-L1388](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1378-L1388) |
| [`_get_lifecycle_status`](#contract-vllm_mlx.server._get_lifecycle_status) | function | `_get_lifecycle_status() -> dict \| None` | Get lifecycle status for the default resident if lifecycle is enabled. | [#L1391-L1395](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1391-L1395) |
| [`_public_lifecycle_status`](#contract-vllm_mlx.server._public_lifecycle_status) | function | `_public_lifecycle_status(lifecycle: dict \| None) -> dict \| None` | Return residency status safe for unauthenticated public endpoints. | [#L1398-L1410](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1398-L1410) |
| [`_lifecycle_loop`](#contract-vllm_mlx.server._lifecycle_loop) | function | `async _lifecycle_loop() -> None` | Background idle-unload loop for the default resident. | [#L1413-L1433](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1413-L1433) |
| [`_acquire_default_engine`](#contract-vllm_mlx.server._acquire_default_engine) | function | `async _acquire_default_engine(*, count_activity: bool = True) -> BaseEngine` | Acquire the default engine, auto-loading via the residency manager if needed. | [#L1436-L1451](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1436-L1451) |
| [`_release_default_engine`](#contract-vllm_mlx.server._release_default_engine) | function | `async _release_default_engine(*, count_activity: bool = True) -> None` | Release the default engine after request processing. | [#L1454-L1463](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1454-L1463) |
| [`lifespan`](#contract-vllm_mlx.server.lifespan) | function | `async lifespan(app: FastAPI) -> not annotated` | FastAPI lifespan for startup/shutdown events. | [#L1466-L1589](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1466-L1589) |
| [`_metrics_result_from_status`](#contract-vllm_mlx.server._metrics_result_from_status) | function | `_metrics_result_from_status(status_code: int) -> str` | Map HTTP-ish status codes to low-cardinality inference results. | [#L1602-L1610](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1602-L1610) |
| [`_metrics_path_for_request`](#contract-vllm_mlx.server._metrics_path_for_request) | function | `_metrics_path_for_request(request: Request) -> str` | Prefer route templates over raw URLs to keep metrics cardinality bounded. | [#L1613-L1626](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1613-L1626) |
| [`_metrics_middleware`](#contract-vllm_mlx.server._metrics_middleware) | function | `async _metrics_middleware(request: Request, call_next) -> not annotated` | Capture generic HTTP request metrics when enabled. | [#L1630-L1659](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1630-L1659) |
| [`RateLimiter`](#contract-vllm_mlx.server.RateLimiter) | class | `RateLimiter(requests_per_minute: int = 60, enabled: bool = False)` | Simple in-memory rate limiter using sliding window. | [#L1662-L1700](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1662-L1700) |
| [`RateLimiter.__init__`](#contract-vllm_mlx.server.RateLimiter.__init__) | method | `RateLimiter.__init__(requests_per_minute: int = 60, enabled: bool = False) -> not annotated` | Method `RateLimiter.__init__` updates `self.requests_per_minute`, `self.enabled`, `self.window_size`, `self._requests`; calls `defaultdict`, `threading.Lock`. | [#L1665-L1670](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1665-L1670) |
| [`RateLimiter.is_allowed`](#contract-vllm_mlx.server.RateLimiter.is_allowed) | method | `RateLimiter.is_allowed(client_id: str) -> tuple[bool, int]` | Check if request is allowed for client. | [#L1672-L1700](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1672-L1700) |
| [`check_rate_limit`](#contract-vllm_mlx.server.check_rate_limit) | function | `async check_rate_limit(request: Request) -> not annotated` | Rate limiting dependency. | [#L1707-L1720](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1707-L1720) |
| [`verify_api_key`](#contract-vllm_mlx.server.verify_api_key) | function | `async verify_api_key(credentials: HTTPAuthorizationCredentials = Depends(security)) -> not annotated` | Verify API key if authentication is enabled. | [#L1723-L1742](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1723-L1742) |
| [`get_engine`](#contract-vllm_mlx.server.get_engine) | function | `get_engine() -> BaseEngine` | Get the loaded engine, raising error if not loaded. | [#L1745-L1749](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1745-L1749) |
| [`_coerce_tool_arguments`](#contract-vllm_mlx.server._coerce_tool_arguments) | function | `_coerce_tool_arguments(arguments_json: str, tool_name: str, tools: list[dict] \| None) -> str` | Coerce tool call arguments to match the tool schema. | [#L1752-L1796](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1752-L1796) |
| [`_validate_model_name`](#contract-vllm_mlx.server._validate_model_name) | function | `_validate_model_name(request_model: str) -> None` | Validate that the request model name matches the served model. | [#L1799-L1818](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1799-L1818) |
| [`_get_engine_tokenizer`](#contract-vllm_mlx.server._get_engine_tokenizer) | function | `_get_engine_tokenizer(engine: BaseEngine \| None) -> object \| None` | Return tokenizer-like parser state from the active engine. | [#L1821-L1828](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1821-L1828) |
| [`_get_or_init_tool_parser`](#contract-vllm_mlx.server._get_or_init_tool_parser) | function | `_get_or_init_tool_parser(engine: BaseEngine \| None = None) -> not annotated` | Return the cached tool parser, initializing it from the given engine. | [#L1831-L1841](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1831-L1841) |
| [`_parse_tool_calls_with_parser`](#contract-vllm_mlx.server._parse_tool_calls_with_parser) | function | `_parse_tool_calls_with_parser(output_text: str, request: ChatCompletionRequest \| None = None, engine: BaseEngine \| None = None) -> tuple[str, list \| None]` | Parse tool calls from model output using the configured parser. | [#L1844-L1930](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1844-L1930) |
| [`_apply_response_format_or_raise`](#contract-vllm_mlx.server._apply_response_format_or_raise) | function | `_apply_response_format_or_raise(text: str, response_format: object, *, ensure_ascii: bool = False) -> str` | Return validated JSON content or fail before returning a success response. | [#L1933-L1952](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1933-L1952) |
| [`_response_format_type`](#contract-vllm_mlx.server._response_format_type) | function | `_response_format_type(response_format: object \| None) -> str \| None` | Function `_response_format_type` calls `isinstance`, `response_format.get`, `getattr`; has 3 explicit return paths. | [#L1955-L1960](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1955-L1960) |
| [`_promote_streaming_response_format_delta`](#contract-vllm_mlx.server._promote_streaming_response_format_delta) | function | `_promote_streaming_response_format_delta(content: str \| None, reasoning: str \| None, request: ChatCompletionRequest) -> tuple[str \| None, str \| None]` | Keep response_format JSON on the streaming content channel. | [#L1963-L1981](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1963-L1981) |
| [`_new_response_item_id`](#contract-vllm_mlx.server._new_response_item_id) | function | `_new_response_item_id(prefix: str) -> str` | Generate stable OpenAI-style item ids. | [#L1984-L1986](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1984-L1986) |
| [`_response_content_to_text`](#contract-vllm_mlx.server._response_content_to_text) | function | `_response_content_to_text(content) -> str` | Normalize Responses API content items into plain text. | [#L1989-L2006](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L1989-L2006) |
| [`_responses_tools_to_chat_tools`](#contract-vllm_mlx.server._responses_tools_to_chat_tools) | function | `_responses_tools_to_chat_tools(tools: list[ResponseFunctionTool \| dict]) -> tuple[list[dict] \| None, list[str]]` | Convert supported Responses tools and report unsupported tool types. | [#L2009-L2049](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2009-L2049) |
| [`_responses_input_to_chat_messages`](#contract-vllm_mlx.server._responses_input_to_chat_messages) | function | `_responses_input_to_chat_messages(request: ResponsesRequest) -> list[dict]` | Convert Responses API input items into chat-completions-style messages. | [#L2052-L2170](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2052-L2170) |
| [`_responses_request_to_new_persisted_messages`](#contract-vllm_mlx.server._responses_request_to_new_persisted_messages) | function | `_responses_request_to_new_persisted_messages(request: ResponsesRequest) -> list[dict]` | Persist only the current request's replayable input items. | [#L2173-L2181](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2173-L2181) |
| [`_responses_request_to_persisted_messages`](#contract-vllm_mlx.server._responses_request_to_persisted_messages) | function | `_responses_request_to_persisted_messages(request: ResponsesRequest) -> list[dict]` | Persist replayable history for chained previous_response_id requests. | [#L2184-L2200](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2184-L2200) |
| [`_responses_request_to_chat_request`](#contract-vllm_mlx.server._responses_request_to_chat_request) | function | `_responses_request_to_chat_request(request: ResponsesRequest) -> ChatCompletionRequest` | Build a ChatCompletionRequest from a ResponsesRequest. | [#L2203-L2253](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2203-L2253) |
| [`_build_responses_output_items`](#contract-vllm_mlx.server._build_responses_output_items) | function | `_build_responses_output_items(text: str \| None, reasoning: str \| None, tool_calls: list[ToolCall] \| None) -> list[ResponseMessageItem \| ResponseReasoningItem \| ResponseFunctionCallItem]` | Convert parsed assistant output into Responses API output items. | [#L2256-L2293](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2256-L2293) |
| [`_response_output_items_to_chat_messages`](#contract-vllm_mlx.server._response_output_items_to_chat_messages) | function | `_response_output_items_to_chat_messages(output_items: list) -> list[dict]` | Persist assistant output in chat-completions form for previous_response_id. | [#L2296-L2325](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2296-L2325) |
| [`_build_response_object`](#contract-vllm_mlx.server._build_response_object) | function | `_build_response_object(request: ResponsesRequest, output_items: list[ResponseMessageItem \| ResponseReasoningItem \| ResponseFunctionCallItem], prompt_tokens: int, completion_tokens: int, finish_reason: str \| None, response_id: str \| None = None) -> ResponseObject` | Build a full Responses API object. | [#L2328-L2367](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2328-L2367) |
| [`_prepare_responses_request`](#contract-vllm_mlx.server._prepare_responses_request) | function | `_prepare_responses_request(request: ResponsesRequest, *, validate_remote_media: bool = True) -> tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]` | Prepare a Responses request for execution on the chat engine. | [#L2370-L2414](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2370-L2414) |
| [`_prepare_streaming_responses_request`](#contract-vllm_mlx.server._prepare_streaming_responses_request) | function | `_prepare_streaming_responses_request(request: ResponsesRequest) -> tuple[BaseEngine, ChatCompletionRequest, list[dict], dict]` | Prepare a streaming Responses request after eager URL validation. | [#L2417-L2421](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2417-L2421) |
| [`_run_responses_request`](#contract-vllm_mlx.server._run_responses_request) | function | `async _run_responses_request(request: ResponsesRequest, raw_request: Request) -> tuple[ResponseObject \| None, list[dict]]` | Execute a Responses API request against the backend chat engine. | [#L2424-L2477](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2424-L2477) |
| [`_stream_responses_request`](#contract-vllm_mlx.server._stream_responses_request) | function | `async _stream_responses_request(request: ResponsesRequest) -> AsyncIterator[str]` | Execute a Responses API request and stream SSE events incrementally. | [#L2480-L2868](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2480-L2868) |
| [`_stream_responses_request._start_text_item`](#contract-vllm_mlx.server._stream_responses_request._start_text_item) | nested function | `_stream_responses_request._start_text_item() -> list[str]` | Nested Function `_stream_responses_request._start_text_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`. | [#L2525-L2561](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2525-L2561) |
| [`_stream_responses_request._start_reasoning_item`](#contract-vllm_mlx.server._stream_responses_request._start_reasoning_item) | nested function | `_stream_responses_request._start_reasoning_item() -> list[str]` | Nested Function `_stream_responses_request._start_reasoning_item` calls `_new_response_item_id`, `events.append`, `_responses_sse_event`, `ResponseOutputItemAddedEvent`; returns `events`. | [#L2563-L2598](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2563-L2598) |
| [`_responses_sse_event`](#contract-vllm_mlx.server._responses_sse_event) | function | `_responses_sse_event(event_type: str, payload: BaseModel \| dict) -> str` | Encode a Responses API SSE event. | [#L2871-L2878](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2871-L2878) |
| [`_strip_harmony_analysis_blocks`](#contract-vllm_mlx.server._strip_harmony_analysis_blocks) | function | `_strip_harmony_analysis_blocks(text: str) -> str` | Remove harmony analysis-channel blocks (and their content) so reasoning text is never handed to the tool parser, while commentary/final text is preserved. | [#L2888-L2892](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2888-L2892) |
| [`_extract_reasoning_and_tool_calls`](#contract-vllm_mlx.server._extract_reasoning_and_tool_calls) | function | `_extract_reasoning_and_tool_calls(output_text: str, request: ChatCompletionRequest \| None = None, *, allow_reasoning: bool = True, engine: BaseEngine \| None = None) -> tuple[str \| None, str \| None, list[ToolCall] \| None]` | Extract reasoning first, then parse tool calls from the cleaned content. | [#L2895-L2951](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2895-L2951) |
| [`_detect_native_tool_support`](#contract-vllm_mlx.server._detect_native_tool_support) | function | `_detect_native_tool_support() -> bool` | Detect if the active tool parser supports native tool format. | [#L2954-L2983](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2954-L2983) |
| [`_detect_harmony_rendering`](#contract-vllm_mlx.server._detect_harmony_rendering) | function | `_detect_harmony_rendering() -> bool` | Detect whether the harmony rendering path should handle prompt building. | [#L2986-L3019](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L2986-L3019) |
| [`_tool_choice_disabled`](#contract-vllm_mlx.server._tool_choice_disabled) | function | `_tool_choice_disabled(request: ChatCompletionRequest \| None) -> bool` | Return True when tool_choice explicitly disables tool calling. | [#L3022-L3031](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3022-L3031) |
| [`_get_streaming_tool_parser`](#contract-vllm_mlx.server._get_streaming_tool_parser) | function | `_get_streaming_tool_parser(request: ChatCompletionRequest \| None, engine: BaseEngine \| None = None) -> not annotated` | Get a streaming-capable tool parser for this request. | [#L3034-L3071](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3034-L3071) |
| [`_extract_streaming_tool_delta`](#contract-vllm_mlx.server._extract_streaming_tool_delta) | function | `_extract_streaming_tool_delta(parser, previous_text: str, delta_text: str, request_context: dict) -> tuple[str, dict \| None]` | Parse one request-local streaming delta and return new accumulated text. | [#L3074-L3088](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3074-L3088) |
| [`_stream_request_metadata`](#contract-vllm_mlx.server._stream_request_metadata) | function | `_stream_request_metadata(request: ChatCompletionRequest) -> tuple[dict, list \| None, bool]` | Function `_stream_request_metadata` calls `request.model_dump(include={'tools'}).get`, `request.model_dump`, `bool`; returns `({'tools': tools or []}, tools, include_usage)`. | [#L3091-L3100](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3091-L3100) |
| [`_parse_streaming_tool_content`](#contract-vllm_mlx.server._parse_streaming_tool_content) | function | `_parse_streaming_tool_content(parser, accumulated_text: str, delta_text: str, request_context: dict) -> tuple[str, dict \| None, bool]` | Function `_parse_streaming_tool_content` calls `_extract_streaming_tool_delta`; returns `(accumulated_text, result, suppress)`. | [#L3103-L3116](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3103-L3116) |
| [`_streaming_tool_markup_possible`](#contract-vllm_mlx.server._streaming_tool_markup_possible) | function | `_streaming_tool_markup_possible(text: str) -> bool` | Heuristic marker check to avoid parser work on ordinary text chunks. | [#L3119-L3125](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3119-L3125) |
| [`_streaming_tool_markup_possible_after_delta`](#contract-vllm_mlx.server._streaming_tool_markup_possible_after_delta) | function | `_streaming_tool_markup_possible_after_delta(accumulated_text: str, delta_text: str) -> bool` | Check only the boundary window needed to detect newly appearing tool markup. | [#L3128-L3143](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3128-L3143) |
| [`load_embedding_model`](#contract-vllm_mlx.server.load_embedding_model) | function | `load_embedding_model(model_name: str \| None, *, lock: bool = False, reuse_existing: bool = True) -> None` | Load or reuse the embedding model engine when configured. | [#L3146-L3171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3146-L3171) |
| [`load_reranker_model`](#contract-vllm_mlx.server.load_reranker_model) | function | `load_reranker_model(model_name: str \| None, *, lock: bool = False, reuse_existing: bool = True) -> None` | Load or reuse the reranker model engine when configured. | [#L3174-L3199](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3174-L3199) |
| [`load_model`](#contract-vllm_mlx.server.load_model) | function | `load_model(model_name: str, use_batching: bool = False, scheduler_config = None, stream_interval: int = 1, max_tokens: int = 32768, max_request_tokens: int = 32768, force_mllm: bool = False, gpu_memory_utilization: float = 0.9, served_model_name: str \| None = None, trust_remote_code: bool = False, mtp: bool = False, prefill_step_size: int = 2048, specprefill_enabled: bool = False, specprefill_threshold: int = 8192, specprefill_keep_pct: float = 0.3, specprefill_backbone_pct: float = 0.0, specprefill_draft_model: str = None, mllm_draft_model: str \| None = None, mllm_draft_kind: str \| None = None, mllm_draft_block_size: int \| None = None, warm_prompts_path: str \| None = None, auto_unload_idle_seconds: float = 0.0, lazy_load_model: bool = False) -> not annotated` | Load a model (auto-detects MLLM vs LLM). | [#L3202-L3431](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3202-L3431) |
| [`load_model_registry`](#contract-vllm_mlx.server.load_model_registry) | function | `load_model_registry(config_path: str, *, defaults: RegistryServeDefaults) -> None` | Load a registry-backed model manager from YAML configuration. | [#L3434-L3457](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3434-L3457) |
| [`get_usage`](#contract-vllm_mlx.server.get_usage) | function | `get_usage(output: GenerationOutput) -> Usage` | Extract usage metrics from GenerationOutput. | [#L3460-L3472](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3460-L3472) |
| [`metrics`](#contract-vllm_mlx.server.metrics) | function | `async metrics() -> not annotated` | Prometheus scrape endpoint (disabled by default). | [#L3476-L3485](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3476-L3485) |
| [`health`](#contract-vllm_mlx.server.health) | function | `async health() -> not annotated` | Health check endpoint. | [#L3489-L3544](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3489-L3544) |
| [`status`](#contract-vllm_mlx.server.status) | function | `async status() -> not annotated` | Real-time status with per-request details for debugging and monitoring. | [#L3548-L3597](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3548-L3597) |
| [`cache_stats`](#contract-vllm_mlx.server.cache_stats) | function | `async cache_stats() -> not annotated` | Get cache statistics for debugging and monitoring. | [#L3601-L3627](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3601-L3627) |
| [`clear_cache`](#contract-vllm_mlx.server.clear_cache) | function | `async clear_cache() -> not annotated` | Clear all caches. | [#L3631-L3659](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3631-L3659) |
| [`clear_prefix_cache`](#contract-vllm_mlx.server.clear_prefix_cache) | function | `async clear_prefix_cache() -> not annotated` | Clear the text prefix cache used for KV reuse in continuous batching. | [#L3663-L3713](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3663-L3713) |
| [`clear_prefix_cache._rewarm`](#contract-vllm_mlx.server.clear_prefix_cache._rewarm) | nested function | `async clear_prefix_cache._rewarm() -> not annotated` | Nested Function `clear_prefix_cache._rewarm` calls `load_warmup_file`, `warm_prefix_cache`, `logger.info`, `logger.warning`; awaits asynchronous work. | [#L3688-L3707](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3688-L3707) |
| [`cancel_request`](#contract-vllm_mlx.server.cancel_request) | function | `async cancel_request(request_id: str) -> not annotated` | Cancel an active or queued request. | [#L3720-L3747](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3720-L3747) |
| [`delete_request`](#contract-vllm_mlx.server.delete_request) | function | `async delete_request(request_id: str) -> not annotated` | OpenAI-style alias for cancelling an active or queued request. | [#L3754-L3756](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3754-L3756) |
| [`list_models`](#contract-vllm_mlx.server.list_models) | function | `async list_models() -> ModelsResponse` | List available models. | [#L3760-L3775](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3760-L3775) |
| [`create_embeddings`](#contract-vllm_mlx.server.create_embeddings) | function | `async create_embeddings(request: EmbeddingRequest) -> EmbeddingResponse` | Create embeddings for the given input text(s). | [#L3787-L3908](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3787-L3908) |
| [`rerank_documents`](#contract-vllm_mlx.server.rerank_documents) | function | `async rerank_documents(request: RerankRequest) -> RerankResponse` | Rerank documents against a query using a cross-encoder model. | [#L3920-L4038](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L3920-L4038) |
| [`list_mcp_tools`](#contract-vllm_mlx.server.list_mcp_tools) | function | `async list_mcp_tools() -> MCPToolsResponse` | List all available MCP tools. | [#L4047-L4063](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4047-L4063) |
| [`list_mcp_servers`](#contract-vllm_mlx.server.list_mcp_servers) | function | `async list_mcp_servers() -> MCPServersResponse` | Get status of all MCP servers. | [#L4067-L4084](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4067-L4084) |
| [`execute_mcp_tool`](#contract-vllm_mlx.server.execute_mcp_tool) | function | `async execute_mcp_tool(request: MCPExecuteRequest) -> MCPExecuteResponse` | Execute an MCP tool. | [#L4088-L4117](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4088-L4117) |
| [`create_transcription`](#contract-vllm_mlx.server.create_transcription) | function | `async create_transcription(file: UploadFile, model: str = 'whisper-large-v3', language: str \| None = None, response_format: str = 'json') -> not annotated` | Transcribe audio to text (OpenAI Whisper API compatible). | [#L4130-L4196](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4130-L4196) |
| [`create_speech`](#contract-vllm_mlx.server.create_speech) | function | `async create_speech(model: str = 'kokoro', input: str = '', voice: str = 'af_heart', speed: float = 1.0, response_format: str = 'wav') -> not annotated` | Generate speech from text (OpenAI TTS API compatible). | [#L4200-L4254](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4200-L4254) |
| [`list_voices`](#contract-vllm_mlx.server.list_voices) | function | `async list_voices(model: str = 'kokoro') -> not annotated` | List available voices for a TTS model. | [#L4258-L4267](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4258-L4267) |
| [`_ensure_sse_terminal`](#contract-vllm_mlx.server._ensure_sse_terminal) | function | `async _ensure_sse_terminal(generator: AsyncIterator[str], terminal_frame: str) -> AsyncIterator[str]` | Guarantee that *terminal_frame* is emitted exactly once at the end of *generator*, even if the generator raises mid-stream. | [#L4275-L4296](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4275-L4296) |
| [`_find_uvicorn_cycle`](#contract-vllm_mlx.server._find_uvicorn_cycle) | function | `_find_uvicorn_cycle(obj, depth = 0, visited = None) -> not annotated` | Walk through middleware wrappers to find uvicorn's RequestResponseCycle. | [#L4299-L4346](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4299-L4346) |
| [`_is_client_disconnected`](#contract-vllm_mlx.server._is_client_disconnected) | function | `_is_client_disconnected(raw_request: Request) -> bool` | Reliable client disconnect check. | [#L4349-L4374](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4349-L4374) |
| [`_disconnect_guard`](#contract-vllm_mlx.server._disconnect_guard) | function | `async _disconnect_guard(generator: AsyncIterator[str], raw_request: Request, poll_interval: float = 0.5, heartbeat_interval: float = 5.0, cleanup = None, timeout: float \| None = None) -> AsyncIterator[str]` | Wrap streaming generator to abort on client disconnect. | [#L4377-L4546](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4377-L4546) |
| [`_disconnect_guard._elapsed`](#contract-vllm_mlx.server._disconnect_guard._elapsed) | nested function | `_disconnect_guard._elapsed() -> not annotated` | Nested Function `_disconnect_guard._elapsed` calls `_time.monotonic`; returns `f'{_time.monotonic() - _t0:.1f}s'`. | [#L4407-L4408](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4407-L4408) |
| [`_disconnect_guard._wait_disconnect`](#contract-vllm_mlx.server._disconnect_guard._wait_disconnect) | nested function | `async _disconnect_guard._wait_disconnect() -> not annotated` | Nested Function `_disconnect_guard._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_elapsed`; awaits asynchronous work; returns `None`. | [#L4417-L4429](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4417-L4429) |
| [`_disconnect_guard._deferred_generator_close`](#contract-vllm_mlx.server._disconnect_guard._deferred_generator_close) | nested function | `async _disconnect_guard._deferred_generator_close() -> not annotated` | Nested Function `_disconnect_guard._deferred_generator_close` calls `asyncio.sleep`, `_gen_to_close.aclose`, `logger.debug`, `type`; awaits asynchronous work. | [#L4528-L4536](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4528-L4536) |
| [`_wait_with_disconnect`](#contract-vllm_mlx.server._wait_with_disconnect) | function | `async _wait_with_disconnect(coro, raw_request: Request, timeout: float, poll_interval: float = 0.5, timeout_detail_seconds: float \| None = None, cleanup_result = None) -> not annotated` | Run a coroutine with both timeout and client disconnect detection. | [#L4549-L4638](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4549-L4638) |
| [`_wait_with_disconnect._wait_disconnect`](#contract-vllm_mlx.server._wait_with_disconnect._wait_disconnect) | nested function | `async _wait_with_disconnect._wait_disconnect() -> not annotated` | Nested Function `_wait_with_disconnect._wait_disconnect` calls `asyncio.sleep`, `_is_client_disconnected`, `logger.info`, `_time.monotonic`; awaits asynchronous work; returns `None`. | [#L4569-L4581](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4569-L4581) |
| [`_start_request_budget`](#contract-vllm_mlx.server._start_request_budget) | function | `_start_request_budget(timeout: float \| None) -> tuple[float, float]` | Return the total timeout and absolute deadline for a request. | [#L4641-L4644](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4641-L4644) |
| [`_remaining_request_timeout`](#contract-vllm_mlx.server._remaining_request_timeout) | function | `_remaining_request_timeout(total_timeout: float, deadline: float) -> float` | Compute remaining request budget or raise the standard timeout error. | [#L4647-L4655](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4647-L4655) |
| [`_acquire_default_engine_for_request`](#contract-vllm_mlx.server._acquire_default_engine_for_request) | function | `async _acquire_default_engine_for_request(raw_request: Request, *, total_timeout: float, deadline: float, count_activity: bool = True, model: str \| None = None) -> BaseEngine \| None` | Acquire the engine for a request, using the model registry when active. | [#L4661-L4719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4661-L4719) |
| [`_acquire_default_engine_for_request._registry_acquire`](#contract-vllm_mlx.server._acquire_default_engine_for_request._registry_acquire) | nested function | `async _acquire_default_engine_for_request._registry_acquire() -> not annotated` | Nested Function `_acquire_default_engine_for_request._registry_acquire` calls `_acquire_request_model`, `id`; awaits asynchronous work; returns `ctx.engine`. | [#L4681-L4685](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4681-L4685) |
| [`_acquire_default_engine_for_request._registry_cleanup`](#contract-vllm_mlx.server._acquire_default_engine_for_request._registry_cleanup) | nested function | `async _acquire_default_engine_for_request._registry_cleanup(_result) -> not annotated` | Nested Function `_acquire_default_engine_for_request._registry_cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`; awaits asynchronous work. | [#L4687-L4690](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4687-L4690) |
| [`_release_engine_for_request`](#contract-vllm_mlx.server._release_engine_for_request) | function | `async _release_engine_for_request(raw_request: Request \| None, *, count_activity: bool = True) -> None` | Release the engine acquired for this request. | [#L4722-L4737](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4722-L4737) |
| [`_make_release_cleanup`](#contract-vllm_mlx.server._make_release_cleanup) | function | `_make_release_cleanup(raw_request: Request \| None) -> not annotated` | Return a cleanup callable suitable for ``_disconnect_guard``. | [#L4740-L4752](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4740-L4752) |
| [`_make_release_cleanup._cleanup`](#contract-vllm_mlx.server._make_release_cleanup._cleanup) | nested function | `async _make_release_cleanup._cleanup() -> not annotated` | Nested Function `_make_release_cleanup._cleanup` calls `_active_request_contexts.pop`, `id`, `ctx.release`, `_release_default_engine`; awaits asynchronous work. | [#L4744-L4749](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4744-L4749) |
| [`create_completion`](#contract-vllm_mlx.server.create_completion) | function | `async create_completion(request: CompletionRequest, raw_request: Request) -> not annotated` | Create a text completion. | [#L4763-L4909](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4763-L4909) |
| [`create_chat_completion`](#contract-vllm_mlx.server.create_chat_completion) | function | `async create_chat_completion(request: ChatCompletionRequest, raw_request: Request) -> not annotated` | Create a chat completion (supports multimodal content for VLM models). | [#L4916-L5114](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L4916-L5114) |
| [`_normalize_messages`](#contract-vllm_mlx.server._normalize_messages) | function | `_normalize_messages(messages: list[dict]) -> list[dict]` | Normalize message roles and merge consecutive same-role messages. | [#L5117-L5172](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5117-L5172) |
| [`_get_engine_tokenizer`](#contract-vllm_mlx.server._get_engine_tokenizer) | function | `_get_engine_tokenizer(engine) -> object \| None` | Return the tokenizer backing ``engine``, if exposed. | [#L5175-L5187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5175-L5187) |
| [`create_response`](#contract-vllm_mlx.server.create_response) | function | `async create_response(request: ResponsesRequest, raw_request: Request) -> not annotated` | Create a Responses API response. | [#L5194-L5214](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5194-L5214) |
| [`_get_forced_tool_name`](#contract-vllm_mlx.server._get_forced_tool_name) | function | `_get_forced_tool_name(tool_choice) -> str \| None` | Extract forced tool name from tool_choice, if any. | [#L5217-L5230](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5217-L5230) |
| [`_apply_forced_tool_choice`](#contract-vllm_mlx.server._apply_forced_tool_choice) | function | `_apply_forced_tool_choice(tool_choice, tools, messages, chat_kwargs = None) -> not annotated` | Apply forced tool_choice by filtering tools and injecting instructions. | [#L5233-L5279](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5233-L5279) |
| [`_tool_name`](#contract-vllm_mlx.server._tool_name) | function | `_tool_name(tool: dict) -> str \| None` | Extract function name from a tool definition dict. | [#L5282-L5287](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5282-L5287) |
| [`_inject_json_instruction`](#contract-vllm_mlx.server._inject_json_instruction) | function | `_inject_json_instruction(messages: list, instruction: str) -> list` | Inject JSON instruction into messages. | [#L5290-L5319](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5290-L5319) |
| [`_convert_anthropic_stop_reason`](#contract-vllm_mlx.server._convert_anthropic_stop_reason) | function | `_convert_anthropic_stop_reason(openai_reason: str \| None) -> str` | Convert OpenAI finish_reason to Anthropic stop_reason. | [#L5327-L5335](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5327-L5335) |
| [`_prepare_anthropic_endpoint_invocation`](#contract-vllm_mlx.server._prepare_anthropic_endpoint_invocation) | function | `_prepare_anthropic_endpoint_invocation(engine: BaseEngine, openai_request: ChatCompletionRequest, effective_max_tokens: int) -> PreparedChatInvocation` | Prepare Anthropic invocation and convert URL-safety errors to 400s. | [#L5338-L5351](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5338-L5351) |
| [`create_anthropic_message`](#contract-vllm_mlx.server.create_anthropic_message) | function | `async create_anthropic_message(request: Request) -> not annotated` | Anthropic Messages API endpoint. | [#L5357-L5578](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5357-L5578) |
| [`count_anthropic_tokens`](#contract-vllm_mlx.server.count_anthropic_tokens) | function | `async count_anthropic_tokens(request: Request) -> not annotated` | Count tokens for an Anthropic Messages API request. | [#L5585-L5666](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5585-L5666) |
| [`_emit_content_pieces`](#contract-vllm_mlx.server._emit_content_pieces) | function | `_emit_content_pieces(pieces: list[tuple[str, str]], current_block_type: str \| None, block_index: int) -> tuple[list[str], str \| None, int]` | Emit Anthropic SSE events for content pieces from the think router. | [#L5669-L5719](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5669-L5719) |
| [`_stream_anthropic_messages`](#contract-vllm_mlx.server._stream_anthropic_messages) | function | `async _stream_anthropic_messages(engine: BaseEngine, openai_request: ChatCompletionRequest, anthropic_request: AnthropicRequest, prepared: PreparedChatInvocation, metrics_tracker = None) -> AsyncIterator[str]` | Stream Anthropic Messages API SSE events. | [#L5722-L5995](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L5722-L5995) |
| [`stream_completion`](#contract-vllm_mlx.server.stream_completion) | function | `async stream_completion(engine: BaseEngine, prompt: str, request: CompletionRequest, max_tokens: int, repetition_penalty: float \| None = None, metrics_tracker = None) -> AsyncIterator[str]` | Stream completion response. | [#L6003-L6084](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6003-L6084) |
| [`stream_chat_completion`](#contract-vllm_mlx.server.stream_chat_completion) | function | `async stream_chat_completion(engine: BaseEngine, messages: list, request: ChatCompletionRequest, metrics_tracker = None, **kwargs) -> AsyncIterator[str]` | Stream chat completion response. | [#L6087-L6512](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6087-L6512) |
| [`init_mcp`](#contract-vllm_mlx.server.init_mcp) | function | `async init_mcp(config_path: str) -> not annotated` | Initialize MCP manager from config file. | [#L6520-L6546](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6520-L6546) |
| [`_make_keepalive_http_protocol`](#contract-vllm_mlx.server._make_keepalive_http_protocol) | function | `_make_keepalive_http_protocol(idle = 10, interval = 5, count = 3) -> not annotated` | Create a uvicorn HTTP protocol class with aggressive TCP keepalive. | [#L6554-L6589](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6554-L6589) |
| [`_make_keepalive_http_protocol._KeepaliveProtocol`](#contract-vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol) | nested class | `_make_keepalive_http_protocol._KeepaliveProtocol()` | Nested Class `_make_keepalive_http_protocol._KeepaliveProtocol` derives from `_Base` and declares 1 direct member(s). | [#L6567-L6587](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6567-L6587) |
| [`_make_keepalive_http_protocol._KeepaliveProtocol.connection_made`](#contract-vllm_mlx.server._make_keepalive_http_protocol._KeepaliveProtocol.connection_made) | nested function | `_make_keepalive_http_protocol._KeepaliveProtocol.connection_made(transport) -> not annotated` | Nested Function `_make_keepalive_http_protocol._KeepaliveProtocol.connection_made` calls `super().connection_made`, `super`, `transport.get_extra_info`, `sock.setsockopt`; returns `None`. | [#L6568-L6587](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6568-L6587) |
| [`main`](#contract-vllm_mlx.server.main) | function | `main() -> not annotated` | Run the server. | [#L6597-L6708](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6597-L6708) |
| [`create_parser`](#contract-vllm_mlx.server.create_parser) | function | `create_parser() -> argparse.ArgumentParser` | Create the standalone server CLI parser. | [#L6711-L6912](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6711-L6912) |