vllm_mlx.audio.tts

Text-to-Speech (TTS) engine using mlx-audio.

View the complete module source at #L1-L315.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.audio.tts

Text-to-Speech (TTS) engine using mlx-audio.

Supports: - Kokoro (fast, lightweight) - Chatterbox (multilingual, expressive) - VibeVoice (realtime, low latency) - VoxCPM (Chinese/English, high quality)

vllm_mlx.audio.tts.logger module-attribute

logger = logging.getLogger(__name__)

vllm_mlx.audio.tts.DEFAULT_TTS_MODEL module-attribute

DEFAULT_TTS_MODEL = 'mlx-community/Kokoro-82M-bf16'

vllm_mlx.audio.tts.KOKORO_VOICES module-attribute

KOKORO_VOICES = ['af_heart', 'af_bella', 'af_nicole', 'af_sarah', 'af_sky', 'am_adam', 'am_michael', 'bf_emma', 'bf_isabella', 'bm_george', 'bm_lewis']

vllm_mlx.audio.tts.CHATTERBOX_VOICES module-attribute

CHATTERBOX_VOICES = ['default']

vllm_mlx.audio.tts.AudioOutput dataclass

AudioOutput(audio: ndarray, sample_rate: int, duration: float)

Output from TTS generation.

vllm_mlx.audio.tts.AudioOutput.audio instance-attribute

audio: ndarray

vllm_mlx.audio.tts.AudioOutput.sample_rate instance-attribute

sample_rate: int

vllm_mlx.audio.tts.AudioOutput.duration instance-attribute

duration: float

vllm_mlx.audio.tts.TTSEngine

TTSEngine(model_name: str = DEFAULT_TTS_MODEL)

Text-to-Speech engine supporting multiple model families.

Usage

engine = TTSEngine("mlx-community/Kokoro-82M-bf16") engine.load() audio = engine.generate("Hello world!", voice="af_heart") engine.save(audio, "output.wav")

Initialize TTS engine.

Parameters:

• model_name (str, default: DEFAULT_TTS_MODEL ) –

  HuggingFace model name. Supported families: - Kokoro: mlx-community/Kokoro-82M-bf16, Kokoro-82M-4bit - Chatterbox: mlx-community/chatterbox-turbo-fp16 - VibeVoice: mlx-community/VibeVoice-Realtime-0.5B-4bit - VoxCPM: mlx-community/VoxCPM1.5

Source code in vllm_mlx/audio/tts.py

def __init__(
    self,
    model_name: str = DEFAULT_TTS_MODEL,
):
    """
    Initialize TTS engine.

    Args:
        model_name: HuggingFace model name. Supported families:
            - Kokoro: mlx-community/Kokoro-82M-bf16, Kokoro-82M-4bit
            - Chatterbox: mlx-community/chatterbox-turbo-fp16
            - VibeVoice: mlx-community/VibeVoice-Realtime-0.5B-4bit
            - VoxCPM: mlx-community/VoxCPM1.5
    """
    self.model_name = model_name
    self.model = None
    self._loaded = False
    self._model_family = self._detect_family(model_name)

vllm_mlx.audio.tts.TTSEngine.model_name instance-attribute

model_name = model_name

vllm_mlx.audio.tts.TTSEngine.model instance-attribute

model = None

vllm_mlx.audio.tts.TTSEngine._loaded instance-attribute

_loaded = False

vllm_mlx.audio.tts.TTSEngine._model_family instance-attribute

_model_family = self._detect_family(model_name)

vllm_mlx.audio.tts.TTSEngine._detect_family

_detect_family(model_name: str) -> str

Detect model family from name.

Source code in vllm_mlx/audio/tts.py

def _detect_family(self, model_name: str) -> str:
    """Detect model family from name."""
    name_lower = model_name.lower()
    if "kokoro" in name_lower:
        return "kokoro"
    elif "chatterbox" in name_lower:
        return "chatterbox"
    elif "vibevoice" in name_lower:
        return "vibevoice"
    elif "voxcpm" in name_lower:
        return "voxcpm"
    elif "csm" in name_lower:
        return "csm"
    elif "cosyvoice" in name_lower:
        return "cosyvoice"
    else:
        return "kokoro"  # Default

vllm_mlx.audio.tts.TTSEngine.load

load() -> None

Load the TTS model.

Source code in vllm_mlx/audio/tts.py

def load(self) -> None:
    """Load the TTS model."""
    if self._loaded:
        return

    try:
        from mlx_audio.tts.generate import load_model

        self.model = load_model(self.model_name)
        self._loaded = True
        logger.info(
            f"TTS model loaded: {self.model_name} (family: {self._model_family})"
        )
    except ImportError as e:
        logger.error(f"mlx-audio not installed: {e}")
        raise ImportError(
            "mlx-audio is required for TTS. Install with: pip install mlx-audio"
        ) from e

vllm_mlx.audio.tts.TTSEngine.generate

generate(text: str, voice: str = 'af_heart', speed: float = 1.0, lang_code: str = 'a') -> AudioOutput

Generate speech from text.

Parameters:

• text (str) –

  Text to synthesize

• voice (str, default: 'af_heart' ) –

  Voice ID (model-specific)

• speed (float, default: 1.0 ) –

  Speech speed (0.5 to 2.0)

• lang_code (str, default: 'a' ) –

  Language code (a=English, e=Spanish, f=French, etc.)

Returns:

• AudioOutput –

  AudioOutput with audio data and metadata

Source code in vllm_mlx/audio/tts.py

def generate(
    self,
    text: str,
    voice: str = "af_heart",
    speed: float = 1.0,
    lang_code: str = "a",
) -> AudioOutput:
    """
    Generate speech from text.

    Args:
        text: Text to synthesize
        voice: Voice ID (model-specific)
        speed: Speech speed (0.5 to 2.0)
        lang_code: Language code (a=English, e=Spanish, f=French, etc.)

    Returns:
        AudioOutput with audio data and metadata
    """
    if not self._loaded:
        self.load()

    try:
        import mlx.core as mx

        audio_chunks = []
        sample_rate = 24000  # Default for most models

        for result in self.model.generate(
            text=text,
            voice=voice,
            speed=speed,
            lang_code=lang_code,
        ):
            audio_data = result.audio
            if hasattr(result, "sample_rate"):
                sample_rate = result.sample_rate

            # Convert mlx array to numpy
            if isinstance(audio_data, mx.array):
                audio_np = np.array(audio_data.tolist(), dtype=np.float32)
            elif hasattr(audio_data, "tolist"):
                audio_np = np.array(audio_data.tolist(), dtype=np.float32)
            else:
                audio_np = np.array(audio_data, dtype=np.float32)

            audio_chunks.append(audio_np)

        if not audio_chunks:
            raise RuntimeError("No audio generated")

        # Concatenate all chunks
        full_audio = (
            np.concatenate(audio_chunks)
            if len(audio_chunks) > 1
            else audio_chunks[0]
        )
        duration = len(full_audio) / sample_rate

        return AudioOutput(
            audio=full_audio,
            sample_rate=sample_rate,
            duration=duration,
        )
    except Exception as e:
        logger.error(f"TTS generation failed: {e}")
        raise

vllm_mlx.audio.tts.TTSEngine.stream_generate

stream_generate(text: str, voice: str = 'af_heart', speed: float = 1.0) -> Iterator[AudioOutput]

Stream speech generation chunk by chunk.

Parameters:

• text (str) –

  Text to synthesize

• voice (str, default: 'af_heart' ) –

  Voice ID

• speed (float, default: 1.0 ) –

  Speech speed

Yields:

• AudioOutput –

  AudioOutput chunks

Source code in vllm_mlx/audio/tts.py

def stream_generate(
    self,
    text: str,
    voice: str = "af_heart",
    speed: float = 1.0,
) -> Iterator[AudioOutput]:
    """
    Stream speech generation chunk by chunk.

    Args:
        text: Text to synthesize
        voice: Voice ID
        speed: Speech speed

    Yields:
        AudioOutput chunks
    """
    if not self._loaded:
        self.load()

    sample_rate = 24000

    for result in self.model.generate(
        text=text,
        voice=voice,
        speed=speed,
    ):
        audio_data = result.audio
        if hasattr(result, "sample_rate"):
            sample_rate = result.sample_rate

        if hasattr(audio_data, "tolist"):
            audio_np = np.array(audio_data.tolist(), dtype=np.float32)
        else:
            audio_np = np.array(audio_data, dtype=np.float32)

        yield AudioOutput(
            audio=audio_np,
            sample_rate=sample_rate,
            duration=len(audio_np) / sample_rate,
        )

vllm_mlx.audio.tts.TTSEngine.save

save(audio: AudioOutput, path: Union[str, Path], format: str = 'wav') -> None

Save audio to file.

Parameters:

• audio (AudioOutput) –

  AudioOutput to save

• path (Union[str, Path]) –

  Output file path

• format (str, default: 'wav' ) –

  Output format (wav, mp3)

Source code in vllm_mlx/audio/tts.py

def save(
    self,
    audio: AudioOutput,
    path: Union[str, Path],
    format: str = "wav",
) -> None:
    """
    Save audio to file.

    Args:
        audio: AudioOutput to save
        path: Output file path
        format: Output format (wav, mp3)
    """
    try:
        from mlx_audio.tts import save_audio

        save_audio(audio.audio, str(path), sample_rate=audio.sample_rate)
        logger.info(f"Audio saved to {path}")
    except ImportError:
        # Fallback to scipy
        import scipy.io.wavfile as wav

        # Ensure audio is in correct format
        audio_int16 = (audio.audio * 32767).astype(np.int16)
        wav.write(str(path), audio.sample_rate, audio_int16)
        logger.info(f"Audio saved to {path} (scipy fallback)")

vllm_mlx.audio.tts.TTSEngine.to_bytes

to_bytes(audio: AudioOutput, format: str = 'wav') -> bytes

Convert audio to bytes.

Parameters:

• audio (AudioOutput) –

  AudioOutput to convert

• format (str, default: 'wav' ) –

  Output format (wav, mp3)

Returns:

• bytes –

  Audio data as bytes

Source code in vllm_mlx/audio/tts.py

def to_bytes(
    self,
    audio: AudioOutput,
    format: str = "wav",
) -> bytes:
    """
    Convert audio to bytes.

    Args:
        audio: AudioOutput to convert
        format: Output format (wav, mp3)

    Returns:
        Audio data as bytes
    """
    import scipy.io.wavfile as wav

    buffer = io.BytesIO()
    audio_int16 = (audio.audio * 32767).astype(np.int16)
    wav.write(buffer, audio.sample_rate, audio_int16)
    return buffer.getvalue()

vllm_mlx.audio.tts.TTSEngine.get_voices

get_voices() -> list

Get available voices for current model.

Source code in vllm_mlx/audio/tts.py

def get_voices(self) -> list:
    """Get available voices for current model."""
    if self._model_family == "kokoro":
        return KOKORO_VOICES
    elif self._model_family == "chatterbox":
        return CHATTERBOX_VOICES
    else:
        return ["default"]

vllm_mlx.audio.tts.TTSEngine.unload

unload() -> None

Unload model to free memory.

Source code in vllm_mlx/audio/tts.py

def unload(self) -> None:
    """Unload model to free memory."""
    self.model = None
    self._loaded = False
    logger.info("TTS model unloaded")

vllm_mlx.audio.tts.generate_speech

generate_speech(text: str, model_name: str = DEFAULT_TTS_MODEL, voice: str = 'af_heart', speed: float = 1.0) -> AudioOutput

Convenience function to generate speech without managing engine.

Parameters:

• text (str) –

  Text to synthesize

• model_name (str, default: DEFAULT_TTS_MODEL ) –

  Model to use

• voice (str, default: 'af_heart' ) –

  Voice ID

• speed (float, default: 1.0 ) –

  Speech speed

Returns:

• AudioOutput –

  AudioOutput

Source code in vllm_mlx/audio/tts.py

def generate_speech(
    text: str,
    model_name: str = DEFAULT_TTS_MODEL,
    voice: str = "af_heart",
    speed: float = 1.0,
) -> AudioOutput:
    """
    Convenience function to generate speech without managing engine.

    Args:
        text: Text to synthesize
        model_name: Model to use
        voice: Voice ID
        speed: Speech speed

    Returns:
        AudioOutput
    """
    engine = TTSEngine(model_name)
    engine.load()
    return engine.generate(text, voice=voice, speed=speed)

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.audio.tts.AudioOutput · class

vllm_mlx.audio.tts.AudioOutput(audio: np.ndarray, sample_rate: int, duration: float)

Output from TTS generation.

Parameters

Name	Type	Required	Default	Description
audio	np.ndarray	yes	none	Required constructor field.
sample_rate	int	yes	none	Required constructor field.
duration	float	yes	none	Required constructor field.

Returns

• Constructs: vllm_mlx.audio.tts.AudioOutput

Exceptions and behavior

Class AudioOutput declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L44-L49.

vllm_mlx.audio.tts.TTSEngine · class

vllm_mlx.audio.tts.TTSEngine(model_name: str = DEFAULT_TTS_MODEL)

Text-to-Speech engine supporting multiple model families.

Parameters

Name	Type	Required	Default	Description
model_name	str	no	DEFAULT_TTS_MODEL	HuggingFace model name. Supported families: - Kokoro: mlx-community/Kokoro-82M-bf16, Kokoro-82M-4bit - Chatterbox: mlx-community/chatterbox-turbo-fp16 - VibeVoice: mlx-community/VibeVoice-Realtime-0.5B-4bit - VoxCPM: mlx-community/VoxCPM1.5

Returns

• Constructs: vllm_mlx.audio.tts.TTSEngine

Exceptions and behavior

Class TTSEngine declares 9 direct member(s). No direct raise statement appears in this definition.

View source #L52-L292.

vllm_mlx.audio.tts.TTSEngine.__init__ · method

vllm_mlx.audio.tts.TTSEngine.__init__(model_name: str = DEFAULT_TTS_MODEL) -> not annotated

Initialize TTS engine.

Parameters

Name	Type	Required	Default	Description
model_name	str	no	DEFAULT_TTS_MODEL	HuggingFace model name. Supported families: - Kokoro: mlx-community/Kokoro-82M-bf16, Kokoro-82M-4bit - Chatterbox: mlx-community/chatterbox-turbo-fp16 - VibeVoice: mlx-community/VibeVoice-Realtime-0.5B-4bit - VoxCPM: mlx-community/VoxCPM1.5

Returns

• Type: not annotated

Exceptions and behavior

Method TTSEngine.__init__ updates self.model_name, self.model, self._loaded, self._model_family; calls self._detect_family. No direct raise statement appears in this definition.

View source #L63-L80.

vllm_mlx.audio.tts.TTSEngine._detect_family · method

vllm_mlx.audio.tts.TTSEngine._detect_family(model_name: str) -> str

Detect model family from name.

Parameters

Name	Type	Required	Default	Description
model_name	str	yes	none	Required positional or keyword input.

Returns

• Type: str
• Direct return expressions: 'kokoro'; 'chatterbox'; 'vibevoice'; 'voxcpm'; 'csm'; 'cosyvoice'

Exceptions and behavior

Method TTSEngine._detect_family calls model_name.lower; has 6 explicit return paths. No direct raise statement appears in this definition.

View source #L82-L98.

vllm_mlx.audio.tts.TTSEngine.load · method

vllm_mlx.audio.tts.TTSEngine.load() -> None

Load the TTS model.

Parameters

This callable has no explicit inputs.

Returns

• Type: None
• Direct return expressions: None

Exceptions and behavior

Method TTSEngine.load updates self.model, self._loaded; calls load_model, logger.info, logger.error, ImportError; can raise ImportError; returns None. Directly raised exceptions: ImportError.

View source #L100-L117.

vllm_mlx.audio.tts.TTSEngine.generate · method

vllm_mlx.audio.tts.TTSEngine.generate(text: str, voice: str = 'af_heart', speed: float = 1.0, lang_code: str = 'a') -> AudioOutput

Generate speech from text.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Text to synthesize
voice	str	no	'af_heart'	Voice ID (model-specific)
speed	float	no	1.0	Speech speed (0.5 to 2.0)
lang_code	str	no	'a'	Language code (a=English, e=Spanish, f=French, etc.)

Returns

• Type: AudioOutput
• Direct return expressions: AudioOutput(audio=full_audio, sample_rate=sample_rate, duration=duration)

Exceptions and behavior

Method TTSEngine.generate calls self.load, self.model.generate, hasattr, isinstance; can raise RuntimeError; returns AudioOutput(audio=full_audio, sample_rate=sample_rate, duration=duration). Directly raised exceptions: RuntimeError.

View source #L119-L185.

vllm_mlx.audio.tts.TTSEngine.stream_generate · method

vllm_mlx.audio.tts.TTSEngine.stream_generate(text: str, voice: str = 'af_heart', speed: float = 1.0) -> Iterator[AudioOutput]

Stream speech generation chunk by chunk.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Text to synthesize
voice	str	no	'af_heart'	Voice ID
speed	float	no	1.0	Speech speed

Returns

• Type: Iterator[AudioOutput]
• Yields values incrementally.

Exceptions and behavior

Method TTSEngine.stream_generate calls self.load, self.model.generate, hasattr, np.array; yields values incrementally. No direct raise statement appears in this definition.

View source #L187-L227.

vllm_mlx.audio.tts.TTSEngine.save · method

vllm_mlx.audio.tts.TTSEngine.save(audio: AudioOutput, path: Union[str, Path], format: str = 'wav') -> None

Save audio to file.

Parameters

Name	Type	Required	Default	Description
audio	AudioOutput	yes	none	AudioOutput to save
path	Union[str, Path]	yes	none	Output file path
format	str	no	'wav'	Output format (wav, mp3)

Returns

• Type: None

Exceptions and behavior

Method TTSEngine.save calls save_audio, str, logger.info, (audio.audio * 32767).astype. No direct raise statement appears in this definition.

View source #L229-L255.

vllm_mlx.audio.tts.TTSEngine.to_bytes · method

vllm_mlx.audio.tts.TTSEngine.to_bytes(audio: AudioOutput, format: str = 'wav') -> bytes

Convert audio to bytes.

Parameters

Name	Type	Required	Default	Description
audio	AudioOutput	yes	none	AudioOutput to convert
format	str	no	'wav'	Output format (wav, mp3)

Returns

• Type: bytes
• Direct return expressions: buffer.getvalue()

Exceptions and behavior

Method TTSEngine.to_bytes calls io.BytesIO, (audio.audio * 32767).astype, wav.write, buffer.getvalue; returns buffer.getvalue(). No direct raise statement appears in this definition.

View source #L257-L277.

vllm_mlx.audio.tts.TTSEngine.get_voices · method

vllm_mlx.audio.tts.TTSEngine.get_voices() -> list

Get available voices for current model.

Parameters

This callable has no explicit inputs.

Returns

• Type: list
• Direct return expressions: KOKORO_VOICES; CHATTERBOX_VOICES; ['default']

Exceptions and behavior

Method TTSEngine.get_voices has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L279-L286.

vllm_mlx.audio.tts.TTSEngine.unload · method

vllm_mlx.audio.tts.TTSEngine.unload() -> None

Unload model to free memory.

Parameters

This callable has no explicit inputs.

Returns

• Type: None

Exceptions and behavior

Method TTSEngine.unload updates self.model, self._loaded; calls logger.info. No direct raise statement appears in this definition.

View source #L288-L292.

vllm_mlx.audio.tts.generate_speech · function

vllm_mlx.audio.tts.generate_speech(text: str, model_name: str = DEFAULT_TTS_MODEL, voice: str = 'af_heart', speed: float = 1.0) -> AudioOutput

Convenience function to generate speech without managing engine.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Text to synthesize
model_name	str	no	DEFAULT_TTS_MODEL	Model to use
voice	str	no	'af_heart'	Voice ID
speed	float	no	1.0	Speech speed

Returns

• Type: AudioOutput
• Direct return expressions: engine.generate(text, voice=voice, speed=speed)

Exceptions and behavior

Function generate_speech calls TTSEngine, engine.load, engine.generate; returns engine.generate(text, voice=voice, speed=speed). No direct raise statement appears in this definition.

View source #L295-L315.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
AudioOutput	class	AudioOutput(audio: np.ndarray, sample_rate: int, duration: float)	Output from TTS generation.	#L44-L49
TTSEngine	class	TTSEngine(model_name: str = DEFAULT_TTS_MODEL)	Text-to-Speech engine supporting multiple model families.	#L52-L292
TTSEngine.__init__	method	TTSEngine.__init__(model_name: str = DEFAULT_TTS_MODEL) -> not annotated	Initialize TTS engine.	#L63-L80
TTSEngine._detect_family	method	TTSEngine._detect_family(model_name: str) -> str	Detect model family from name.	#L82-L98
TTSEngine.load	method	TTSEngine.load() -> None	Load the TTS model.	#L100-L117
TTSEngine.generate	method	TTSEngine.generate(text: str, voice: str = 'af_heart', speed: float = 1.0, lang_code: str = 'a') -> AudioOutput	Generate speech from text.	#L119-L185
TTSEngine.stream_generate	method	TTSEngine.stream_generate(text: str, voice: str = 'af_heart', speed: float = 1.0) -> Iterator[AudioOutput]	Stream speech generation chunk by chunk.	#L187-L227
TTSEngine.save	method	TTSEngine.save(audio: AudioOutput, path: Union[str, Path], format: str = 'wav') -> None	Save audio to file.	#L229-L255
TTSEngine.to_bytes	method	TTSEngine.to_bytes(audio: AudioOutput, format: str = 'wav') -> bytes	Convert audio to bytes.	#L257-L277
TTSEngine.get_voices	method	TTSEngine.get_voices() -> list	Get available voices for current model.	#L279-L286
TTSEngine.unload	method	TTSEngine.unload() -> None	Unload model to free memory.	#L288-L292
generate_speech	function	generate_speech(text: str, model_name: str = DEFAULT_TTS_MODEL, voice: str = 'af_heart', speed: float = 1.0) -> AudioOutput	Convenience function to generate speech without managing engine.	#L295-L315