Skip to content

vllm_mlx.endpoint_model_policies

Request-time model resolution policies for optional endpoints.

View the complete module source at #L1-L118.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.endpoint_model_policies

Request-time model resolution policies for optional endpoints.

These endpoints intentionally do not expose arbitrary Hugging Face loading from user-controlled request bodies. Unknown model names must be rejected before any engine instantiation or download path is reached.

vllm_mlx.endpoint_model_policies._EMBEDDING_MODELS module-attribute 

_EMBEDDING_MODELS = frozenset({'mlx-community/ModernBERT-base-mlx', 'mlx-community/all-MiniLM-L6-v2-4bit', 'mlx-community/bert-base-uncased-mlx', 'mlx-community/bge-large-en-v1.5-4bit', 'mlx-community/embeddinggemma-300m-6bit', 'mlx-community/multilingual-e5-large-mlx', 'mlx-community/multilingual-e5-small-mlx'})

vllm_mlx.endpoint_model_policies._STT_MODEL_ALIASES module-attribute 

_STT_MODEL_ALIASES = {'whisper-large-v3': 'mlx-community/whisper-large-v3-mlx', 'whisper-large-v3-turbo': 'mlx-community/whisper-large-v3-turbo', 'whisper-medium': 'mlx-community/whisper-medium-mlx', 'whisper-small': 'mlx-community/whisper-small-mlx', 'parakeet': 'mlx-community/parakeet-tdt-0.6b-v2', 'parakeet-v3': 'mlx-community/parakeet-tdt-0.6b-v3'}

vllm_mlx.endpoint_model_policies._TTS_MODEL_ALIASES module-attribute 

_TTS_MODEL_ALIASES = {'kokoro': 'mlx-community/Kokoro-82M-bf16', 'kokoro-4bit': 'mlx-community/Kokoro-82M-4bit', 'chatterbox': 'mlx-community/chatterbox-turbo-fp16', 'chatterbox-4bit': 'mlx-community/chatterbox-turbo-4bit', 'vibevoice': 'mlx-community/VibeVoice-Realtime-0.5B-4bit', 'voxcpm': 'mlx-community/VoxCPM1.5'}

vllm_mlx.endpoint_model_policies._STT_MODEL_MAP module-attribute 

_STT_MODEL_MAP = _with_identity_aliases(_STT_MODEL_ALIASES)

vllm_mlx.endpoint_model_policies._TTS_MODEL_MAP module-attribute 

_TTS_MODEL_MAP = _with_identity_aliases(_TTS_MODEL_ALIASES)

vllm_mlx.endpoint_model_policies._with_identity_aliases 

_with_identity_aliases(model_map: dict[str, str]) -> dict[str, str]
Source code in vllm_mlx/endpoint_model_policies.py 
42
43
44
45
46
def _with_identity_aliases(model_map: dict[str, str]) -> dict[str, str]:
    expanded = dict(model_map)
    for model_name in model_map.values():
        expanded[model_name] = model_name
    return expanded

vllm_mlx.endpoint_model_policies._reject_unknown_embedding_model 

_reject_unknown_embedding_model(requested_model: str) -> None
Source code in vllm_mlx/endpoint_model_policies.py 
53
54
55
56
57
58
59
60
61
62
63
def _reject_unknown_embedding_model(requested_model: str) -> None:
    supported = ", ".join(sorted(_EMBEDDING_MODELS))
    raise HTTPException(
        status_code=400,
        detail=(
            f"Embedding model '{requested_model}' is not available. "
            "Request-time embedding model loading is limited to the supported "
            f"allowlist: {supported}. To use a different embedding model, start "
            "the server with --embedding-model <model>."
        ),
    )

vllm_mlx.endpoint_model_policies._reject_unknown_audio_model 

_reject_unknown_audio_model(endpoint: str, requested_model: str, supported_aliases: dict[str, str]) -> None
Source code in vllm_mlx/endpoint_model_policies.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
def _reject_unknown_audio_model(
    endpoint: str,
    requested_model: str,
    supported_aliases: dict[str, str],
) -> None:
    aliases = ", ".join(sorted(supported_aliases))
    raise HTTPException(
        status_code=400,
        detail=(
            f"{endpoint} model '{requested_model}' is not available. "
            f"Supported request models are: {aliases}. Exact configured model IDs "
            "for those aliases are also accepted."
        ),
    )

vllm_mlx.endpoint_model_policies.resolve_embedding_model_name 

resolve_embedding_model_name(requested_model: str, *, locked_model: str | None = None) -> str

Resolve the embedding model for a request or raise HTTP 400.

Source code in vllm_mlx/endpoint_model_policies.py 
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
def resolve_embedding_model_name(
    requested_model: str,
    *,
    locked_model: str | None = None,
) -> str:
    """Resolve the embedding model for a request or raise HTTP 400."""
    if locked_model is not None:
        if requested_model == locked_model:
            return locked_model
        raise HTTPException(
            status_code=400,
            detail=(
                f"Embedding model '{requested_model}' is not available. "
                f"This server was started with --embedding-model {locked_model}. "
                f"Only '{locked_model}' can be used for embeddings. Restart the "
                f"server with a different --embedding-model to use '{requested_model}'."
            ),
        )

    if requested_model in _EMBEDDING_MODELS:
        return requested_model

    _reject_unknown_embedding_model(requested_model)

vllm_mlx.endpoint_model_policies.resolve_stt_model_name 

resolve_stt_model_name(requested_model: str) -> str

Resolve an STT request model alias or configured model ID.

Source code in vllm_mlx/endpoint_model_policies.py 
107
108
109
110
111
def resolve_stt_model_name(requested_model: str) -> str:
    """Resolve an STT request model alias or configured model ID."""
    if requested_model in _STT_MODEL_MAP:
        return _STT_MODEL_MAP[requested_model]
    _reject_unknown_audio_model("Transcription", requested_model, _STT_MODEL_ALIASES)

vllm_mlx.endpoint_model_policies.resolve_tts_model_name 

resolve_tts_model_name(requested_model: str) -> str

Resolve a TTS request model alias or configured model ID.

Source code in vllm_mlx/endpoint_model_policies.py 
114
115
116
117
118
def resolve_tts_model_name(requested_model: str) -> str:
    """Resolve a TTS request model alias or configured model ID."""
    if requested_model in _TTS_MODEL_MAP:
        return _TTS_MODEL_MAP[requested_model]
    _reject_unknown_audio_model("Speech", requested_model, _TTS_MODEL_ALIASES)

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.endpoint_model_policies._with_identity_aliases · function 
vllm_mlx.endpoint_model_policies._with_identity_aliases(model_map: dict[str, str]) -> dict[str, str]

Function _with_identity_aliases calls dict, model_map.values; returns expanded.

Parameters

Name Type Required Default Description
model_map dict[str, str] yes none Required positional or keyword input.

Returns

  • Type: dict[str, str]
  • Direct return expressions: expanded

Exceptions and behavior

Function _with_identity_aliases calls dict, model_map.values; returns expanded. No direct raise statement appears in this definition.

View source #L42-L46.

vllm_mlx.endpoint_model_policies._reject_unknown_embedding_model · function 
vllm_mlx.endpoint_model_policies._reject_unknown_embedding_model(requested_model: str) -> None

Function _reject_unknown_embedding_model calls ', '.join, sorted, HTTPException; can raise HTTPException.

Parameters

Name Type Required Default Description
requested_model str yes none Required positional or keyword input.

Returns

  • Type: None

Exceptions and behavior

Function _reject_unknown_embedding_model calls ', '.join, sorted, HTTPException; can raise HTTPException. Directly raised exceptions: HTTPException.

View source #L53-L63.

vllm_mlx.endpoint_model_policies._reject_unknown_audio_model · function 
vllm_mlx.endpoint_model_policies._reject_unknown_audio_model(endpoint: str, requested_model: str, supported_aliases: dict[str, str]) -> None

Function _reject_unknown_audio_model calls ', '.join, sorted, HTTPException; can raise HTTPException.

Parameters

Name Type Required Default Description
endpoint str yes none Required positional or keyword input.
requested_model str yes none Required positional or keyword input.
supported_aliases dict[str, str] yes none Required positional or keyword input.

Returns

  • Type: None

Exceptions and behavior

Function _reject_unknown_audio_model calls ', '.join, sorted, HTTPException; can raise HTTPException. Directly raised exceptions: HTTPException.

View source #L66-L79.

vllm_mlx.endpoint_model_policies.resolve_embedding_model_name · function 
vllm_mlx.endpoint_model_policies.resolve_embedding_model_name(requested_model: str, *, locked_model: str | None = None) -> str

Resolve the embedding model for a request or raise HTTP 400.

Parameters

Name Type Required Default Description
requested_model str yes none Required positional or keyword input.
locked_model str \| None no None Optional keyword-only input; defaults to None.

Returns

  • Type: str
  • Direct return expressions: locked_model; requested_model

Exceptions and behavior

Function resolve_embedding_model_name calls HTTPException, _reject_unknown_embedding_model; can raise HTTPException; has 2 explicit return paths. Directly raised exceptions: HTTPException.

View source #L82-L104.

vllm_mlx.endpoint_model_policies.resolve_stt_model_name · function 
vllm_mlx.endpoint_model_policies.resolve_stt_model_name(requested_model: str) -> str

Resolve an STT request model alias or configured model ID.

Parameters

Name Type Required Default Description
requested_model str yes none Required positional or keyword input.

Returns

  • Type: str
  • Direct return expressions: _STT_MODEL_MAP[requested_model]

Exceptions and behavior

Function resolve_stt_model_name calls _reject_unknown_audio_model; returns _STT_MODEL_MAP[requested_model]. No direct raise statement appears in this definition.

View source #L107-L111.

vllm_mlx.endpoint_model_policies.resolve_tts_model_name · function 
vllm_mlx.endpoint_model_policies.resolve_tts_model_name(requested_model: str) -> str

Resolve a TTS request model alias or configured model ID.

Parameters

Name Type Required Default Description
requested_model str yes none Required positional or keyword input.

Returns

  • Type: str
  • Direct return expressions: _TTS_MODEL_MAP[requested_model]

Exceptions and behavior

Function resolve_tts_model_name calls _reject_unknown_audio_model; returns _TTS_MODEL_MAP[requested_model]. No direct raise statement appears in this definition.

View source #L114-L118.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
_with_identity_aliases function _with_identity_aliases(model_map: dict[str, str]) -> dict[str, str] Function _with_identity_aliases calls dict, model_map.values; returns expanded. #L42-L46
_reject_unknown_embedding_model function _reject_unknown_embedding_model(requested_model: str) -> None Function _reject_unknown_embedding_model calls ', '.join, sorted, HTTPException; can raise HTTPException. #L53-L63
_reject_unknown_audio_model function _reject_unknown_audio_model(endpoint: str, requested_model: str, supported_aliases: dict[str, str]) -> None Function _reject_unknown_audio_model calls ', '.join, sorted, HTTPException; can raise HTTPException. #L66-L79
resolve_embedding_model_name function resolve_embedding_model_name(requested_model: str, *, locked_model: str \| None = None) -> str Resolve the embedding model for a request or raise HTTP 400. #L82-L104
resolve_stt_model_name function resolve_stt_model_name(requested_model: str) -> str Resolve an STT request model alias or configured model ID. #L107-L111
resolve_tts_model_name function resolve_tts_model_name(requested_model: str) -> str Resolve a TTS request model alias or configured model ID. #L114-L118