Skip to content

vllm_mlx.audio.processor

Audio processing using mlx-audio.

View the complete module source at #L1-L214.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.audio.processor

Audio processing using mlx-audio.

Supports: - SAM-Audio: Text-guided source separation (isolate voice from background) - MossFormer2: Speech enhancement (noise removal)

vllm_mlx.audio.processor.logger module-attribute 

logger = logging.getLogger(__name__)

vllm_mlx.audio.processor.DEFAULT_SAM_MODEL module-attribute 

DEFAULT_SAM_MODEL = 'mlx-community/sam-audio-large-fp16'

vllm_mlx.audio.processor.SeparationResult dataclass 

SeparationResult(target: ndarray, residual: ndarray, sample_rate: int, peak_memory: float)

Result from audio separation.

vllm_mlx.audio.processor.SeparationResult.target instance-attribute 

target: ndarray

vllm_mlx.audio.processor.SeparationResult.residual instance-attribute 

residual: ndarray

vllm_mlx.audio.processor.SeparationResult.sample_rate instance-attribute 

sample_rate: int

vllm_mlx.audio.processor.SeparationResult.peak_memory instance-attribute 

peak_memory: float

vllm_mlx.audio.processor.AudioProcessor 

AudioProcessor(model_name: str = DEFAULT_SAM_MODEL)

Audio processor for voice separation and enhancement.

Uses SAM-Audio for text-guided source separation: - Isolate speech from music/noise - Extract specific sounds by description

Usage

processor = AudioProcessor() processor.load() result = processor.separate("meeting.mp3", description="speech") processor.save(result.target, "voice_only.wav")

Initialize audio processor.

Parameters:

  • model_name (str, default: DEFAULT_SAM_MODEL ) –

    HuggingFace model name. Supported: - mlx-community/sam-audio-large-fp16 (best quality) - mlx-community/sam-audio-large - mlx-community/sam-audio-small-fp16 (faster) - mlx-community/sam-audio-small

Source code in vllm_mlx/audio/processor.py 
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
def __init__(
    self,
    model_name: str = DEFAULT_SAM_MODEL,
):
    """
    Initialize audio processor.

    Args:
        model_name: HuggingFace model name. Supported:
            - mlx-community/sam-audio-large-fp16 (best quality)
            - mlx-community/sam-audio-large
            - mlx-community/sam-audio-small-fp16 (faster)
            - mlx-community/sam-audio-small
    """
    self.model_name = model_name
    self.model = None
    self.processor = None
    self._loaded = False
    self.sample_rate = 44100  # SAM-Audio default

vllm_mlx.audio.processor.AudioProcessor.model_name instance-attribute 

model_name = model_name

vllm_mlx.audio.processor.AudioProcessor.model instance-attribute 

model = None

vllm_mlx.audio.processor.AudioProcessor.processor instance-attribute 

processor = None

vllm_mlx.audio.processor.AudioProcessor._loaded instance-attribute 

_loaded = False

vllm_mlx.audio.processor.AudioProcessor.sample_rate instance-attribute 

sample_rate = 44100

vllm_mlx.audio.processor.AudioProcessor.load 

load() -> None

Load the SAM-Audio model.

Source code in vllm_mlx/audio/processor.py 
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def load(self) -> None:
    """Load the SAM-Audio model."""
    if self._loaded:
        return

    try:
        from mlx_audio.sts import SAMAudio, SAMAudioProcessor

        self.model = SAMAudio.from_pretrained(self.model_name)
        self.processor = SAMAudioProcessor.from_pretrained(self.model_name)

        if hasattr(self.model, "sample_rate"):
            self.sample_rate = self.model.sample_rate

        self._loaded = True
        logger.info(f"Audio processor loaded: {self.model_name}")
    except ImportError as e:
        logger.error(f"mlx-audio not installed: {e}")
        raise ImportError(
            "mlx-audio is required. Install with: pip install mlx-audio"
        ) from e

vllm_mlx.audio.processor.AudioProcessor.separate 

separate(audio_path: Union[str, Path], description: str = 'speech', chunk_seconds: Optional[float] = None) -> SeparationResult

Separate audio based on text description.

Parameters:

  • audio_path (Union[str, Path]) –

    Path to audio file

  • description (str, default: 'speech' ) –

    What to isolate (e.g., "speech", "music", "a person speaking")

  • chunk_seconds (Optional[float], default: None ) –

    Process in chunks for long audio (memory efficient)

Returns:

  • SeparationResult –

    SeparationResult with target (isolated) and residual (background) audio

Source code in vllm_mlx/audio/processor.py 
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
def separate(
    self,
    audio_path: Union[str, Path],
    description: str = "speech",
    chunk_seconds: Optional[float] = None,
) -> SeparationResult:
    """
    Separate audio based on text description.

    Args:
        audio_path: Path to audio file
        description: What to isolate (e.g., "speech", "music", "a person speaking")
        chunk_seconds: Process in chunks for long audio (memory efficient)

    Returns:
        SeparationResult with target (isolated) and residual (background) audio
    """
    if not self._loaded:
        self.load()

    audio_path = str(audio_path)

    try:
        # Process input
        batch = self.processor(
            descriptions=[description],
            audios=[audio_path],
        )

        # Separate
        if chunk_seconds:
            # Memory-efficient for long audio
            result = self.model.separate_long(
                audios=batch.audios,
                descriptions=batch.descriptions,
                chunk_seconds=chunk_seconds,
                overlap_seconds=chunk_seconds / 3,
                anchor_ids=getattr(batch, "anchor_ids", None),
                anchor_alignment=getattr(batch, "anchor_alignment", None),
            )
        else:
            result = self.model.separate(
                audios=batch.audios,
                descriptions=batch.descriptions,
                sizes=getattr(batch, "sizes", None),
                anchor_ids=getattr(batch, "anchor_ids", None),
                anchor_alignment=getattr(batch, "anchor_alignment", None),
            )

        # Convert to numpy
        target = self._to_numpy(result.target[0])
        residual = self._to_numpy(result.residual[0])

        return SeparationResult(
            target=target,
            residual=residual,
            sample_rate=self.sample_rate,
            peak_memory=getattr(result, "peak_memory", 0.0),
        )
    except Exception as e:
        logger.error(f"Audio separation failed: {e}")
        raise

vllm_mlx.audio.processor.AudioProcessor._to_numpy 

_to_numpy(audio) -> ndarray

Convert audio to numpy array.

Source code in vllm_mlx/audio/processor.py 
153
154
155
156
157
def _to_numpy(self, audio) -> np.ndarray:
    """Convert audio to numpy array."""
    if hasattr(audio, "tolist"):
        return np.array(audio.tolist(), dtype=np.float32)
    return np.array(audio, dtype=np.float32)

vllm_mlx.audio.processor.AudioProcessor.save 

save(audio: ndarray, path: Union[str, Path], sample_rate: Optional[int] = None) -> None

Save audio to file.

Parameters:

  • audio (ndarray) –

    Audio data as numpy array

  • path (Union[str, Path]) –

    Output file path

  • sample_rate (Optional[int], default: None ) –

    Sample rate (uses model default if None)

Source code in vllm_mlx/audio/processor.py 
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
def save(
    self,
    audio: np.ndarray,
    path: Union[str, Path],
    sample_rate: Optional[int] = None,
) -> None:
    """
    Save audio to file.

    Args:
        audio: Audio data as numpy array
        path: Output file path
        sample_rate: Sample rate (uses model default if None)
    """
    sr = sample_rate or self.sample_rate

    try:
        from mlx_audio.sts import save_audio

        save_audio(audio, str(path), sample_rate=sr)
    except ImportError:
        import scipy.io.wavfile as wav

        audio_int16 = (audio * 32767).astype(np.int16)
        wav.write(str(path), sr, audio_int16)

    logger.info(f"Audio saved to {path}")

vllm_mlx.audio.processor.AudioProcessor.unload 

unload() -> None

Unload model to free memory.

Source code in vllm_mlx/audio/processor.py 
187
188
189
190
191
192
def unload(self) -> None:
    """Unload model to free memory."""
    self.model = None
    self.processor = None
    self._loaded = False
    logger.info("Audio processor unloaded")

vllm_mlx.audio.processor.separate_voice 

separate_voice(audio_path: Union[str, Path], model_name: str = DEFAULT_SAM_MODEL, description: str = 'speech') -> Tuple[ndarray, ndarray]

Convenience function to separate voice from audio.

Parameters:

  • audio_path (Union[str, Path]) –

    Path to audio file

  • model_name (str, default: DEFAULT_SAM_MODEL ) –

    Model to use

  • description (str, default: 'speech' ) –

    What to isolate

Returns:

  • Tuple[ndarray, ndarray] –

    Tuple of (voice_audio, background_audio) as numpy arrays

Source code in vllm_mlx/audio/processor.py 
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
def separate_voice(
    audio_path: Union[str, Path],
    model_name: str = DEFAULT_SAM_MODEL,
    description: str = "speech",
) -> Tuple[np.ndarray, np.ndarray]:
    """
    Convenience function to separate voice from audio.

    Args:
        audio_path: Path to audio file
        model_name: Model to use
        description: What to isolate

    Returns:
        Tuple of (voice_audio, background_audio) as numpy arrays
    """
    processor = AudioProcessor(model_name)
    processor.load()
    result = processor.separate(audio_path, description=description)
    return result.target, result.residual

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.audio.processor.SeparationResult · class 
vllm_mlx.audio.processor.SeparationResult(target: np.ndarray, residual: np.ndarray, sample_rate: int, peak_memory: float)

Result from audio separation.

Parameters

Name Type Required Default Description
target np.ndarray yes none Required constructor field.
residual np.ndarray yes none Required constructor field.
sample_rate int yes none Required constructor field.
peak_memory float yes none Required constructor field.

Returns

  • Constructs: vllm_mlx.audio.processor.SeparationResult

Exceptions and behavior

Class SeparationResult declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L24-L30.

vllm_mlx.audio.processor.AudioProcessor · class 
vllm_mlx.audio.processor.AudioProcessor(model_name: str = DEFAULT_SAM_MODEL)

Audio processor for voice separation and enhancement.

Parameters

Name Type Required Default Description
model_name str no DEFAULT_SAM_MODEL HuggingFace model name. Supported: - mlx-community/sam-audio-large-fp16 (best quality) - mlx-community/sam-audio-large - mlx-community/sam-audio-small-fp16 (faster) - mlx-community/sam-audio-small

Returns

  • Constructs: vllm_mlx.audio.processor.AudioProcessor

Exceptions and behavior

Class AudioProcessor declares 6 direct member(s). No direct raise statement appears in this definition.

View source #L33-L192.

vllm_mlx.audio.processor.AudioProcessor.__init__ · method 
vllm_mlx.audio.processor.AudioProcessor.__init__(model_name: str = DEFAULT_SAM_MODEL) -> not annotated

Initialize audio processor.

Parameters

Name Type Required Default Description
model_name str no DEFAULT_SAM_MODEL HuggingFace model name. Supported: - mlx-community/sam-audio-large-fp16 (best quality) - mlx-community/sam-audio-large - mlx-community/sam-audio-small-fp16 (faster) - mlx-community/sam-audio-small

Returns

  • Type: not annotated

Exceptions and behavior

Method AudioProcessor.__init__ updates self.model_name, self.model, self.processor, self._loaded. No direct raise statement appears in this definition.

View source #L48-L66.

vllm_mlx.audio.processor.AudioProcessor.load · method 
vllm_mlx.audio.processor.AudioProcessor.load() -> None

Load the SAM-Audio model.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None
  • Direct return expressions: None

Exceptions and behavior

Method AudioProcessor.load updates self.model, self.processor, self.sample_rate, self._loaded; calls SAMAudio.from_pretrained, SAMAudioProcessor.from_pretrained, hasattr, logger.info; can raise ImportError; returns None. Directly raised exceptions: ImportError.

View source #L68-L88.

vllm_mlx.audio.processor.AudioProcessor.separate · method 
vllm_mlx.audio.processor.AudioProcessor.separate(audio_path: Union[str, Path], description: str = 'speech', chunk_seconds: Optional[float] = None) -> SeparationResult

Separate audio based on text description.

Parameters

Name Type Required Default Description
audio_path Union[str, Path] yes none Path to audio file
description str no 'speech' What to isolate (e.g., "speech", "music", "a person speaking")
chunk_seconds Optional[float] no None Process in chunks for long audio (memory efficient)

Returns

  • Type: SeparationResult
  • Direct return expressions: SeparationResult(target=target, residual=residual, sample_rate=self.sample_rate, peak_memory=getattr(result, 'peak_memo…

Exceptions and behavior

Method AudioProcessor.separate calls self.load, str, self.processor, self.model.separate_long; returns SeparationResult(target=target, residual=residual, sample_rate=self.sample_rate, peak_memory=getattr(result, 'peak_memo…. No direct raise statement appears in this definition.

View source #L90-L151.

vllm_mlx.audio.processor.AudioProcessor._to_numpy · method 
vllm_mlx.audio.processor.AudioProcessor._to_numpy(audio) -> np.ndarray

Convert audio to numpy array.

Parameters

Name Type Required Default Description
audio not annotated yes none Required positional or keyword input.

Returns

  • Type: np.ndarray
  • Direct return expressions: np.array(audio.tolist(), dtype=np.float32); np.array(audio, dtype=np.float32)

Exceptions and behavior

Method AudioProcessor._to_numpy calls hasattr, np.array, audio.tolist; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L153-L157.

vllm_mlx.audio.processor.AudioProcessor.save · method 
vllm_mlx.audio.processor.AudioProcessor.save(audio: np.ndarray, path: Union[str, Path], sample_rate: Optional[int] = None) -> None

Save audio to file.

Parameters

Name Type Required Default Description
audio np.ndarray yes none Audio data as numpy array
path Union[str, Path] yes none Output file path
sample_rate Optional[int] no None Sample rate (uses model default if None)

Returns

  • Type: None

Exceptions and behavior

Method AudioProcessor.save calls save_audio, str, (audio * 32767).astype, wav.write. No direct raise statement appears in this definition.

View source #L159-L185.

vllm_mlx.audio.processor.AudioProcessor.unload · method 
vllm_mlx.audio.processor.AudioProcessor.unload() -> None

Unload model to free memory.

Parameters

This callable has no explicit inputs.

Returns

  • Type: None

Exceptions and behavior

Method AudioProcessor.unload updates self.model, self.processor, self._loaded; calls logger.info. No direct raise statement appears in this definition.

View source #L187-L192.

vllm_mlx.audio.processor.separate_voice · function 
vllm_mlx.audio.processor.separate_voice(audio_path: Union[str, Path], model_name: str = DEFAULT_SAM_MODEL, description: str = 'speech') -> Tuple[np.ndarray, np.ndarray]

Convenience function to separate voice from audio.

Parameters

Name Type Required Default Description
audio_path Union[str, Path] yes none Path to audio file
model_name str no DEFAULT_SAM_MODEL Model to use
description str no 'speech' What to isolate

Returns

  • Type: Tuple[np.ndarray, np.ndarray]
  • Direct return expressions: (result.target, result.residual)

Exceptions and behavior

Function separate_voice calls AudioProcessor, processor.load, processor.separate; returns (result.target, result.residual). No direct raise statement appears in this definition.

View source #L195-L214.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
SeparationResult class SeparationResult(target: np.ndarray, residual: np.ndarray, sample_rate: int, peak_memory: float) Result from audio separation. #L24-L30
AudioProcessor class AudioProcessor(model_name: str = DEFAULT_SAM_MODEL) Audio processor for voice separation and enhancement. #L33-L192
AudioProcessor.__init__ method AudioProcessor.__init__(model_name: str = DEFAULT_SAM_MODEL) -> not annotated Initialize audio processor. #L48-L66
AudioProcessor.load method AudioProcessor.load() -> None Load the SAM-Audio model. #L68-L88
AudioProcessor.separate method AudioProcessor.separate(audio_path: Union[str, Path], description: str = 'speech', chunk_seconds: Optional[float] = None) -> SeparationResult Separate audio based on text description. #L90-L151
AudioProcessor._to_numpy method AudioProcessor._to_numpy(audio) -> np.ndarray Convert audio to numpy array. #L153-L157
AudioProcessor.save method AudioProcessor.save(audio: np.ndarray, path: Union[str, Path], sample_rate: Optional[int] = None) -> None Save audio to file. #L159-L185
AudioProcessor.unload method AudioProcessor.unload() -> None Unload model to free memory. #L187-L192
separate_voice function separate_voice(audio_path: Union[str, Path], model_name: str = DEFAULT_SAM_MODEL, description: str = 'speech') -> Tuple[np.ndarray, np.ndarray] Convenience function to separate voice from audio. #L195-L214