# Complete CLI option inventory This page is generated from every `add_argument` declaration in the runtime, maintenance scripts, and runnable examples. The hand-written [CLI guide](cli.md) explains supported workflows. ## `examples.audio_separation_example.main` ### Parser `parser` #### `audio` Input audio file (mp3, wav, etc.) - Destination: `audio` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L37-L37](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L37-L37) #### `--description`, `-d` What to isolate: speech, music, singing, etc. (default: speech) - Destination: `description` - Required: `false` - Default: `speech` - Choices: `not restricted` - Action: `store` - Source: [L38-L39](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L38-L39) #### `--output`, `-o` Output file for isolated audio (default: input_voice.wav) - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L40-L41](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L40-L41) #### `--background`, `-b` Output file for background audio (optional) - Destination: `background` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L42-L43](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L42-L43) #### `--model`, `-m` SAM-Audio model to use - Destination: `model` - Required: `false` - Default: `mlx-community/sam-audio-large-fp16` - Choices: `not restricted` - Action: `store` - Source: [L44-L45](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L44-L45) #### `--chunk`, `-c` Process in chunks of N seconds (for long audio) - Destination: `chunk` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L46-L47](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L46-L47) #### `--play`, `-p` Play result after processing (macOS) - Destination: `play` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L48-L49](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/audio_separation_example.py#L48-L49) ## `examples.benchmark_audio.main` ### Parser `parser` #### `--tts` Run TTS benchmarks - Destination: `tts` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L302-L302](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_audio.py#L302-L302) #### `--stt` Run STT benchmarks - Destination: `stt` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L303-L303](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_audio.py#L303-L303) #### `--audio` Audio file for STT benchmark - Destination: `audio` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L304-L304](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_audio.py#L304-L304) #### `--all` Run all benchmarks - Destination: `all` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L305-L305](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_audio.py#L305-L305) ## `examples.closed_captions.main` ### Parser `parser` #### `--model`, `-m` No argparse help text is declared. - Destination: `model` - Required: `false` - Default: `whisper-large-v3` - Choices: `not restricted` - Action: `store` - Source: [L150-L150](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/closed_captions.py#L150-L150) #### `--language`, `-l` es, en, etc. - Destination: `language` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L151-L151](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/closed_captions.py#L151-L151) #### `--chunk`, `-c` Chunk size (default: 3.0s) - Destination: `chunk` - Required: `false` - Default: `3.0` - Choices: `not restricted` - Action: `store` - Source: [L152-L152](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/closed_captions.py#L152-L152) ## `examples.mic_live.main` ### Parser `parser` #### `--model`, `-m` Model (whisper-small, whisper-medium, parakeet) - Destination: `model` - Required: `false` - Default: `whisper-small` - Choices: `not restricted` - Action: `store` - Source: [L206-L207](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_live.py#L206-L207) #### `--language`, `-l` Language code (en, es, etc.) - Destination: `language` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L208-L208](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_live.py#L208-L208) #### `--sensitivity`, `-s` Mic sensitivity 0.01-0.05 (default: 0.015) - Destination: `sensitivity` - Required: `false` - Default: `0.015` - Choices: `not restricted` - Action: `store` - Source: [L209-L210](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_live.py#L209-L210) ## `examples.mic_realtime.main` ### Parser `parser` #### `--model`, `-m` Model to use (default: whisper-small) - Destination: `model` - Required: `false` - Default: `whisper-small` - Choices: `not restricted` - Action: `store` - Source: [L186-L187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_realtime.py#L186-L187) #### `--chunk`, `-c` Chunk duration in seconds (default: 3.0) - Destination: `chunk` - Required: `false` - Default: `3.0` - Choices: `not restricted` - Action: `store` - Source: [L188-L189](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_realtime.py#L188-L189) #### `--language`, `-l` Language code (e.g., en, es) - Destination: `language` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L190-L190](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_realtime.py#L190-L190) #### `--list-models` List available models - Destination: `list_models` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L191-L191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_realtime.py#L191-L191) ## `examples.mic_transcribe.main` ### Parser `parser` #### `--duration`, `-d` Recording duration in seconds - Destination: `duration` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L115-L115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L115-L115) #### `--model`, `-m` Model: whisper-small, whisper-medium, whisper-large-v3, parakeet - Destination: `model` - Required: `false` - Default: `whisper-small` - Choices: `not restricted` - Action: `store` - Source: [L116-L117](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L116-L117) #### `--language`, `-l` Language code (e.g., en, es). Auto-detect if not set - Destination: `language` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L118-L118](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L118-L118) #### `--continuous`, `-c` Continuous mode: keep recording and transcribing - Destination: `continuous` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L119-L120](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L119-L120) #### `--save`, `-s` Save recorded audio to this file - Destination: `save` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L121-L121](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L121-L121) #### `--list-models` List available models - Destination: `list_models` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L122-L122](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L122-L122) #### `--list-devices` List audio input devices - Destination: `list_devices` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L123-L123](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mic_transcribe.py#L123-L123) ## `examples.mllm_benchmark.main` ### Parser `parser` #### `--server-url` URL of the vllm-mlx server - Destination: `server_url` - Required: `false` - Default: `http://localhost:8000` - Choices: `not restricted` - Action: `store` - Source: [L390-L395](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mllm_benchmark.py#L390-L395) #### `--output` Save results to JSON file - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L396-L401](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mllm_benchmark.py#L396-L401) #### `--warmup` Number of warmup runs (default: 1) - Destination: `warmup` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L402-L407](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mllm_benchmark.py#L402-L407) #### `--quick` Run quick benchmark with fewer resolutions - Destination: `quick` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L408-L412](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/mllm_benchmark.py#L408-L412) ## `examples.test_batching.main` ### Parser `parser` #### `--model` Model to use - Destination: `model` - Required: `false` - Default: `mlx-community/Llama-3.2-1B-Instruct-4bit` - Choices: `not restricted` - Action: `store` - Source: [L118-L123](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L118-L123) #### `--num-requests` Number of concurrent requests - Destination: `num_requests` - Required: `false` - Default: `5` - Choices: `not restricted` - Action: `store` - Source: [L124-L129](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L124-L129) #### `--max-tokens` Max tokens per request - Destination: `max_tokens` - Required: `false` - Default: `30` - Choices: `not restricted` - Action: `store` - Source: [L130-L135](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L130-L135) #### `--temperature` Sampling temperature - Destination: `temperature` - Required: `false` - Default: `0.7` - Choices: `not restricted` - Action: `store` - Source: [L136-L141](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L136-L141) ## `examples.test_openai_compatibility.main` ### Parser `parser` #### `--server-url` URL of the vllm-mlx server (default: http://localhost:8000) - Destination: `server_url` - Required: `false` - Default: `http://localhost:8000` - Choices: `not restricted` - Action: `store` - Source: [L703-L708](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_openai_compatibility.py#L703-L708) #### `--no-image` Skip image tests - Destination: `no_image` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L709-L713](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_openai_compatibility.py#L709-L713) #### `--no-video` Skip video tests - Destination: `no_video` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L714-L718](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_openai_compatibility.py#L714-L718) ## `examples.test_video.main` ### Parser `parser` #### `--video` Path to video file (will create test video if not provided) - Destination: `video` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L271-L275](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L271-L275) #### `--video-url` URL to a video file to test URL support - Destination: `video_url` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L276-L280](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L276-L280) #### `--model` VLM model to use - Destination: `model` - Required: `false` - Default: `mlx-community/Qwen3-VL-4B-Instruct-3bit` - Choices: `not restricted` - Action: `store` - Source: [L281-L286](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L281-L286) #### `--extract-only` Only test frame extraction (no model loading) - Destination: `extract_only` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L287-L291](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L287-L291) #### `--create-test-video` Create synthetic test video instead of downloading - Destination: `create_test_video` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L292-L296](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L292-L296) #### `--url-only` Only test video URL support (requires --video-url) - Destination: `url_only` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L297-L301](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_video.py#L297-L301) ## `examples.tts_example.main` ### Parser `parser` #### `text` Text to synthesize - Destination: `text` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L48-L48](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L48-L48) #### `--voice`, `-v` Voice ID (default: af_heart) - Destination: `voice` - Required: `false` - Default: `af_heart` - Choices: `not restricted` - Action: `store` - Source: [L49-L49](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L49-L49) #### `--lang`, `-l` Language code: a=English, e/es=Spanish, f=French, etc. - Destination: `lang` - Required: `false` - Default: `a` - Choices: `not restricted` - Action: `store` - Source: [L50-L50](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L50-L50) #### `--speed`, `-s` Speech speed 0.5-2.0 (default: 1.0) - Destination: `speed` - Required: `false` - Default: `1.0` - Choices: `not restricted` - Action: `store` - Source: [L51-L51](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L51-L51) #### `--output`, `-o` Output file (default: output.wav) - Destination: `output` - Required: `false` - Default: `output.wav` - Choices: `not restricted` - Action: `store` - Source: [L52-L52](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L52-L52) #### `--model`, `-m` TTS model - Destination: `model` - Required: `false` - Default: `mlx-community/Kokoro-82M-bf16` - Choices: `not restricted` - Action: `store` - Source: [L53-L53](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L53-L53) #### `--list-voices` List available voices - Destination: `list_voices` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L54-L54](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L54-L54) #### `--list-languages` List available languages - Destination: `list_languages` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L55-L55](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L55-L55) #### `--play`, `-p` Play audio after generation (macOS) - Destination: `play` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L56-L56](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_example.py#L56-L56) ## `examples.tts_multilingual.main` ### Parser `parser` #### `text` Text to synthesize - Destination: `text` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L261-L261](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L261-L261) #### `--model`, `-m` Model: kokoro, chatterbox, vibevoice, voxcpm, or 'auto' - Destination: `model` - Required: `false` - Default: `auto` - Choices: `not restricted` - Action: `store` - Source: [L262-L263](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L262-L263) #### `--lang`, `-l` Language code: en, es, fr, ja, zh, etc. - Destination: `lang` - Required: `false` - Default: `en` - Choices: `not restricted` - Action: `store` - Source: [L264-L265](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L264-L265) #### `--voice`, `-v` Voice ID (model-specific) - Destination: `voice` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L266-L267](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L266-L267) #### `--speed`, `-s` Speech speed 0.5-2.0 - Destination: `speed` - Required: `false` - Default: `1.0` - Choices: `not restricted` - Action: `store` - Source: [L268-L269](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L268-L269) #### `--output`, `-o` Output file - Destination: `output` - Required: `false` - Default: `output.wav` - Choices: `not restricted` - Action: `store` - Source: [L270-L271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L270-L271) #### `--play`, `-p` Play audio after generation (macOS) - Destination: `play` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L272-L273](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L272-L273) #### `--list-models` List available models - Destination: `list_models` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L274-L275](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L274-L275) #### `--list-languages` List supported languages - Destination: `list_languages` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L276-L277](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/tts_multilingual.py#L276-L277) ## `examples.video_benchmark.main` ### Parser `parser` #### `--model` VLM model to use - Destination: `model` - Required: `false` - Default: `mlx-community/Qwen3-VL-4B-Instruct-3bit` - Choices: `not restricted` - Action: `store` - Source: [L500-L505](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L500-L505) #### `--video` Path to local video file - Destination: `video` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L506-L511](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L506-L511) #### `--video-url` URL to download video from - Destination: `video_url` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L512-L517](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L512-L517) #### `--duration` Duration of synthetic test video (seconds) - Destination: `duration` - Required: `false` - Default: `10.0` - Choices: `not restricted` - Action: `store` - Source: [L518-L523](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L518-L523) #### `--warmup` Number of warmup runs - Destination: `warmup` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L524-L529](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L524-L529) #### `--quick` Run quick benchmark with fewer configurations - Destination: `quick` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L530-L534](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L530-L534) #### `--output` Save results to JSON file - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L535-L540](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/video_benchmark.py#L535-L540) ## `scripts.add_mtp_weights.main` ### Parser `parser` #### `--mlx-model-path` f'Path to MLX model directory (default: {DEFAULT_MLX_MODEL})' - Destination: `mlx_model_path` - Required: `false` - Default: `DEFAULT_MLX_MODEL` - Choices: `not restricted` - Action: `store` - Source: [L243-L248](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights.py#L243-L248) #### `--source-model` f'HuggingFace model to download MTP shard from (default: {DEFAULT_SOURCE_MODEL})' - Destination: `source_model` - Required: `false` - Default: `DEFAULT_SOURCE_MODEL` - Choices: `not restricted` - Action: `store` - Source: [L249-L254](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights.py#L249-L254) #### `--download-dir` Directory to download MTP shard to (default: temp dir) - Destination: `download_dir` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L255-L260](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights.py#L255-L260) #### `--bits` Quantization bits (default: 6, matching 6-bit model) - Destination: `bits` - Required: `false` - Default: `6` - Choices: `not restricted` - Action: `store` - Source: [L261-L266](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights.py#L261-L266) #### `--skip-download` Skip download (use existing shard) - Destination: `skip_download` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L267-L271](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights.py#L267-L271) ## `scripts.add_mtp_weights_qwen35.main` ### Parser `parser` #### `--mlx-model-path` Path to MLX model directory (HF cache or direct path) - Destination: `mlx_model_path` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L327-L332](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L327-L332) #### `--source-model` HuggingFace BF16 model to download MTP shards from (e.g., Qwen/Qwen3.5-122B-A10B) - Destination: `source_model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L333-L338](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L333-L338) #### `--download-dir` Directory to download shards to (default: temp dir) - Destination: `download_dir` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L339-L344](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L339-L344) #### `--skip-download` Skip download (use existing shards in download-dir) - Destination: `skip_download` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L345-L349](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L345-L349) #### `--keep-shards` Don't delete downloaded BF16 shards after extraction - Destination: `keep_shards` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L350-L354](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L350-L354) #### `--no-quantize` Save MTP weights in BF16 (no quantization). Required for correct MTP predictions. - Destination: `no_quantize` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L355-L359](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/add_mtp_weights_qwen35.py#L355-L359) ## `scripts.gen_api_reference.main` ### Parser `parser` #### `--check` fail instead of writing when generated references are stale - Destination: `check` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L93-L97](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/scripts/gen_api_reference.py#L93-L97) ## `vllm_mlx.benchmark.main` ### Parser `parser` #### `--model` Model name (HuggingFace model name or local path) - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1468-L1473](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1468-L1473) #### `--prompts` Number of prompts to benchmark for LLM (default: 5) - Destination: `prompts` - Required: `false` - Default: `5` - Choices: `not restricted` - Action: `store` - Source: [L1474-L1479](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1474-L1479) #### `--max-tokens` Maximum tokens to generate per prompt (default: 256) - Destination: `max_tokens` - Required: `false` - Default: `256` - Choices: `not restricted` - Action: `store` - Source: [L1480-L1485](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1480-L1485) #### `--temperature` Sampling temperature (default: 0.7) - Destination: `temperature` - Required: `false` - Default: `0.7` - Choices: `not restricted` - Action: `store` - Source: [L1486-L1491](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1486-L1491) #### `--warmup` Number of warmup runs (default: 1) - Destination: `warmup` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L1492-L1497](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1492-L1497) #### `--output` Output file for JSON results - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1498-L1503](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1498-L1503) #### `--mllm` Force MLLM benchmark mode (auto-detected by default) - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1504-L1508](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1504-L1508) #### `--quick` Quick benchmark with fewer configurations - Destination: `quick` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1509-L1513](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1509-L1513) #### `--video` Run video benchmark instead of image benchmark (for MLLM models) - Destination: `video` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1515-L1519](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1515-L1519) #### `--video-url` URL of video to use for benchmark (default: Big Buck Bunny 10s) - Destination: `video_url` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1520-L1525](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1520-L1525) #### `--video-path` Local path to video file for benchmark - Destination: `video_path` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1526-L1531](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/benchmark.py#L1526-L1531) ## `vllm_mlx.cli.create_parser` ### Parser `serve_parser` #### `model` Model to serve - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1008-L1008](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1008-L1008) #### `--models-config` YAML file describing a registry of models for lazy multi-model serving - Destination: `models_config` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1009-L1014](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1009-L1014) #### `--served-model-name` The model name used in the API. If not specified, the model argument is used. - Destination: `served_model_name` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1015-L1020](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1015-L1020) #### `--host` Host to bind (default: localhost; use 0.0.0.0 to expose externally) - Destination: `host` - Required: `false` - Default: `127.0.0.1` - Choices: `not restricted` - Action: `store` - Source: [L1021-L1026](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1021-L1026) #### `--port` Port to bind - Destination: `port` - Required: `false` - Default: `8000` - Choices: `not restricted` - Action: `store` - Source: [L1027-L1027](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1027-L1027) #### `--max-num-seqs` Max concurrent sequences - Destination: `max_num_seqs` - Required: `false` - Default: `256` - Choices: `not restricted` - Action: `store` - Source: [L1028-L1030](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1028-L1030) #### `--prefill-batch-size` Prefill batch size - Destination: `prefill_batch_size` - Required: `false` - Default: `8` - Choices: `not restricted` - Action: `store` - Source: [L1031-L1033](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1031-L1033) #### `--completion-batch-size` Completion batch size - Destination: `completion_batch_size` - Required: `false` - Default: `32` - Choices: `not restricted` - Action: `store` - Source: [L1034-L1036](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1034-L1036) #### `--mllm-prefill-step-size` Override MLLM prefill-step guard (0=use MLLM default: 1024) - Destination: `mllm_prefill_step_size` - Required: `false` - Default: `0` - Choices: `not restricted` - Action: `store` - Source: [L1037-L1042](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1037-L1042) #### `--enable-prefix-cache` Enable prefix caching for repeated prompts (default: enabled) - Destination: `enable_prefix_cache` - Required: `false` - Default: `True` - Choices: `not restricted` - Action: `store_true` - Source: [L1043-L1048](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1043-L1048) #### `--disable-prefix-cache` Disable prefix caching - Destination: `disable_prefix_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1049-L1053](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1049-L1053) #### `--prefix-cache-size` Max entries in prefix cache (default: 100, legacy mode only) - Destination: `prefix_cache_size` - Required: `false` - Default: `100` - Choices: `not restricted` - Action: `store` - Source: [L1054-L1059](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1054-L1059) #### `--cache-memory-mb` Cache memory limit in MB (default: auto-detect ~20%% of RAM) - Destination: `cache_memory_mb` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1061-L1066](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1061-L1066) #### `--cache-memory-percent` Fraction of available RAM for cache if auto-detecting (default: 0.20) - Destination: `cache_memory_percent` - Required: `false` - Default: `0.2` - Choices: `not restricted` - Action: `store` - Source: [L1067-L1072](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1067-L1072) #### `--no-memory-aware-cache` Disable memory-aware cache, use legacy entry-count based cache - Destination: `no_memory_aware_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1073-L1077](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1073-L1077) #### `--kv-cache-quantization` Quantize stored KV caches to reduce memory (8-bit by default) - Destination: `kv_cache_quantization` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1079-L1083](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1079-L1083) #### `--kv-cache-quantization-bits` Bit width for KV cache quantization (default: 8) - Destination: `kv_cache_quantization_bits` - Required: `false` - Default: `8` - Choices: `[4, 8]` - Action: `store` - Source: [L1084-L1090](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1084-L1090) #### `--kv-cache-quantization-group-size` Group size for KV cache quantization (default: 64) - Destination: `kv_cache_quantization_group_size` - Required: `false` - Default: `64` - Choices: `not restricted` - Action: `store` - Source: [L1091-L1096](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1091-L1096) #### `--kv-cache-min-quantize-tokens` Minimum tokens for quantization to apply (default: 256) - Destination: `kv_cache_min_quantize_tokens` - Required: `false` - Default: `256` - Choices: `not restricted` - Action: `store` - Source: [L1097-L1102](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1097-L1102) #### `--ssd-cache-dir` Directory for SSD KV cache tier (default: disabled) - Destination: `ssd_cache_dir` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1104-L1109](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1104-L1109) #### `--ssd-cache-max-gb` Maximum SSD cache size in GB (default: 10.0) - Destination: `ssd_cache_max_gb` - Required: `false` - Default: `10.0` - Choices: `not restricted` - Action: `store` - Source: [L1110-L1115](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1110-L1115) #### `--warm-prompts` Path to a JSON file with prompts to pre-run at startup. Populates the prefix cache so the first real request hits warm (cold TTFT drops 1.3-2.3x on agent workloads). File format is a list of message arrays, same shape as /v1/chat/completions messages. Prompts are warmed concurrently — keep the file small (1-3 entries for typical agent deployments) to avoid memory pressure at boot. - Destination: `warm_prompts` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1117-L1129](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1117-L1129) #### `--stream-interval` Tokens to batch before streaming (1=smooth, higher=throughput) - Destination: `stream_interval` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L1130-L1135](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1130-L1135) #### `--max-kv-size` Maximum KV cache size per sequence. When set, uses RotatingKVCache which bounds memory at the cost of losing early context. Reasoning models (e.g. Qwen3, DeepSeek-R1) should use >= 32768 to avoid evicting the think block mid-generation. - Destination: `max_kv_size` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1136-L1144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1136-L1144) #### `--max-tokens` Default max tokens for generation (default: 32768) - Destination: `max_tokens` - Required: `false` - Default: `32768` - Choices: `not restricted` - Action: `store` - Source: [L1145-L1150](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1145-L1150) #### `--max-request-tokens` Maximum max_tokens accepted from API clients (default: 32768) - Destination: `max_request_tokens` - Required: `false` - Default: `32768` - Choices: `not restricted` - Action: `store` - Source: [L1151-L1156](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1151-L1156) #### `--continuous-batching` Enable continuous batching for multiple concurrent users (slower for single user) - Destination: `continuous_batching` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1157-L1161](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1157-L1161) #### `--gpu-memory-utilization` Fraction of device memory for Metal allocation limit and emergency cache clear threshold (0.0-1.0, default: 0.90). Increase to 0.95 for large models (200GB+) that need more memory headroom. - Destination: `gpu_memory_utilization` - Required: `false` - Default: `0.9` - Choices: `not restricted` - Action: `store` - Source: [L1162-L1169](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1162-L1169) #### `--use-paged-cache` Use paged KV cache for memory efficiency (experimental) - Destination: `use_paged_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1171-L1175](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1171-L1175) #### `--paged-cache-block-size` Tokens per cache block (default: 64) - Destination: `paged_cache_block_size` - Required: `false` - Default: `64` - Choices: `not restricted` - Action: `store` - Source: [L1176-L1181](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1176-L1181) #### `--max-cache-blocks` Maximum number of cache blocks (default: 1000) - Destination: `max_cache_blocks` - Required: `false` - Default: `1000` - Choices: `not restricted` - Action: `store` - Source: [L1182-L1187](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1182-L1187) #### `--chunked-prefill-tokens` Max prefill tokens per scheduler step (0=disabled). Prevents starvation of active requests during long prefills. - Destination: `chunked_prefill_tokens` - Required: `false` - Default: `0` - Choices: `not restricted` - Action: `store` - Source: [L1189-L1195](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1189-L1195) #### `--enable-mtp` Enable MTP (Multi-Token Prediction) for models with built-in MTP heads. Uses cache snapshot/restore for speculative generation. - Destination: `enable_mtp` - Required: `false` - Default: `False` - Choices: `not restricted` - Action: `store_true` - Source: [L1197-L1203](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1197-L1203) #### `--mtp-num-draft-tokens` Number of draft tokens per MTP step (default: 1) - Destination: `mtp_num_draft_tokens` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L1204-L1209](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1204-L1209) #### `--mtp-optimistic` Skip MTP acceptance check for maximum speed. ~5-10%% wrong tokens. Best for chat, not for code. - Destination: `mtp_optimistic` - Required: `false` - Default: `False` - Choices: `not restricted` - Action: `store_true` - Source: [L1210-L1216](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1210-L1216) #### `--prefill-step-size` Chunk size for prompt prefill processing. Larger values use more memory but can improve prefill throughput. (default: 2048) - Destination: `prefill_step_size` - Required: `false` - Default: `2048` - Choices: `not restricted` - Action: `store` - Source: [L1218-L1224](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1218-L1224) #### `--specprefill` Enable SpecPrefill: use a small draft model to score token importance, then sparse-prefill only the important tokens on the target model. Reduces TTFT on long prompts. Requires --specprefill-draft-model. - Destination: `specprefill` - Required: `false` - Default: `False` - Choices: `not restricted` - Action: `store_true` - Source: [L1226-L1233](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1226-L1233) #### `--specprefill-threshold` Minimum suffix tokens to trigger SpecPrefill (default: 8192). Shorter prompts use full prefill (scoring overhead > savings). - Destination: `specprefill_threshold` - Required: `false` - Default: `8192` - Choices: `not restricted` - Action: `store` - Source: [L1234-L1240](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1234-L1240) #### `--specprefill-keep-pct` Fraction of tokens to keep during sparse prefill (default: 0.3). Lower = faster prefill but more quality loss. - Destination: `specprefill_keep_pct` - Required: `false` - Default: `0.3` - Choices: `not restricted` - Action: `store` - Source: [L1241-L1247](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1241-L1247) #### `--specprefill-backbone-pct` Fraction of chunks reserved for evenly spaced sparse-prefill coverage (default: 0.0). - Destination: `specprefill_backbone_pct` - Required: `false` - Default: `0.0` - Choices: `not restricted` - Action: `store` - Source: [L1248-L1254](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1248-L1254) #### `--specprefill-draft-model` Path to small draft model for SpecPrefill importance scoring. Must share the same tokenizer as the target model. - Destination: `specprefill_draft_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1255-L1261](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1255-L1261) #### `--mllm-draft-model` Path to an mlx-vlm MLLM draft/assistant model. For Gemma 4 assistant drafters, use with --mllm-draft-kind mtp. - Destination: `mllm_draft_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1263-L1269](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1263-L1269) #### `--mllm-draft-kind` mlx-vlm draft kind for --mllm-draft-model. - Destination: `mllm_draft_kind` - Required: `false` - Default: `None` - Choices: `['mtp']` - Action: `store` - Source: [L1270-L1276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1270-L1276) #### `--mllm-draft-block-size` Draft block size passed to mlx-vlm for --mllm-draft-model. - Destination: `mllm_draft_block_size` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1277-L1282](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1277-L1282) #### `--mcp-config` Path to MCP configuration file (JSON/YAML) for tool integration - Destination: `mcp_config` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1284-L1289](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1284-L1289) #### `--api-key` API key for authentication (if not set, no auth required) - Destination: `api_key` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1291-L1296](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1291-L1296) #### `--rate-limit` Rate limit requests per minute per client (0 = disabled) - Destination: `rate_limit` - Required: `false` - Default: `0` - Choices: `not restricted` - Action: `store` - Source: [L1297-L1302](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1297-L1302) #### `--timeout` Default request timeout in seconds (default: 300) - Destination: `timeout` - Required: `false` - Default: `300.0` - Choices: `not restricted` - Action: `store` - Source: [L1303-L1308](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1303-L1308) #### `--enable-metrics` Expose Prometheus metrics on /metrics (disabled by default) - Destination: `enable_metrics` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1309-L1313](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1309-L1313) #### `--auto-unload-idle-seconds` Unload the main model after this many idle seconds (0 = disabled) - Destination: `auto_unload_idle_seconds` - Required: `false` - Default: `0.0` - Choices: `not restricted` - Action: `store` - Source: [L1314-L1319](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1314-L1319) #### `--lazy-load-model` Register the main model at startup but defer loading until first request - Destination: `lazy_load_model` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1320-L1324](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1320-L1324) #### `--max-audio-upload-mb` Maximum size of uploaded audio files in MiB (default: 25) - Destination: `max_audio_upload_mb` - Required: `false` - Default: `25` - Choices: `not restricted` - Action: `store` - Source: [L1325-L1330](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1325-L1330) #### `--max-tts-input-chars` Maximum number of characters accepted by /v1/audio/speech (default: 4096) - Destination: `max_tts_input_chars` - Required: `false` - Default: `4096` - Choices: `not restricted` - Action: `store` - Source: [L1331-L1336](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1331-L1336) #### `--enable-auto-tool-choice` Enable auto tool choice for supported models. Use --tool-call-parser to specify which parser to use. - Destination: `enable_auto_tool_choice` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1338-L1342](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1338-L1342) #### `--tool-call-parser` Select the tool call parser for the model. Options: auto (auto-detect), mistral, qwen, qwen3_coder, llama, hermes, harmony, gpt-oss, deepseek, gemma4, kimi, granite, nemotron, xlam, functionary, glm47, minimax. Required for --enable-auto-tool-choice. - Destination: `tool_call_parser` - Required: `false` - Default: `None` - Choices: `['auto', 'mistral', 'qwen', 'qwen3_coder', 'llama', 'hermes', 'harmony', 'gpt-oss', 'deepseek', 'kimi', 'granite', 'nemotron', 'xlam', 'functionary', 'gemma4', 'glm47', 'minimax']` - Action: `store` - Source: [L1343-L1373](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1343-L1373) #### `--reasoning-parser` f"Enable reasoning content extraction with specified parser. Extracts ... tags into reasoning_content field. Options: {', '.join(reasoning_choices)}." - Destination: `reasoning_parser` - Required: `false` - Default: `None` - Choices: `reasoning_choices` - Action: `store` - Source: [L1378-L1388](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1378-L1388) #### `--mllm` Force load model as multimodal (vision) even if name doesn't match auto-detection patterns - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1390-L1394](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1390-L1394) #### `--trust-remote-code` Allow HuggingFace remote code execution during model/tokenizer loading - Destination: `trust_remote_code` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1395-L1399](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1395-L1399) #### `--default-temperature` Override default temperature for all requests (default: use model default) - Destination: `default_temperature` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1401-L1406](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1401-L1406) #### `--default-top-p` Override default top_p for all requests (default: use model default) - Destination: `default_top_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1407-L1412](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1407-L1412) #### `--default-thinking-token-budget` Default thinking token budget for reasoning models. Caps reasoning tokens by forcing the end-think sequence when the budget is exhausted. Per-request thinking_token_budget overrides this. (default: None = unlimited) - Destination: `default_thinking_token_budget` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1413-L1422](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1413-L1422) #### `--default-chat-template-kwargs` Default chat template kwargs to apply to all requests when request chat_template_kwargs is omitted or empty; empty request kwargs use existing server defaults (JSON object, e.g. {"enable_thinking": true}) - Destination: `default_chat_template_kwargs` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1423-L1432](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1423-L1432) #### `--default-top-k` Override default top_k for all requests (default: use model default) - Destination: `default_top_k` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1433-L1438](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1433-L1438) #### `--default-min-p` Override default min_p for all requests (default: use model default) - Destination: `default_min_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1439-L1444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1439-L1444) #### `--default-presence-penalty` Override default presence_penalty for all requests (default: use model default) - Destination: `default_presence_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1445-L1453](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1445-L1453) #### `--default-repetition-penalty` Override default repetition_penalty for all requests (default: use model default) - Destination: `default_repetition_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1454-L1462](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1454-L1462) #### `--embedding-model` Pre-load an embedding model at startup (e.g. mlx-community/embeddinggemma-300m-6bit) - Destination: `embedding_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1464-L1469](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1464-L1469) #### `--rerank-model` Pre-load a reranker model at startup (e.g. mlx-community/jina-reranker-v2-base-multilingual) - Destination: `rerank_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1471-L1476](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1471-L1476) #### `--download-timeout` Per-file download timeout in seconds (default: 300) - Destination: `download_timeout` - Required: `false` - Default: `300` - Choices: `not restricted` - Action: `store` - Source: [L1478-L1483](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1478-L1483) #### `--download-retries` Number of download retry attempts (default: 3) - Destination: `download_retries` - Required: `false` - Default: `3` - Choices: `not restricted` - Action: `store` - Source: [L1484-L1489](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1484-L1489) #### `--offline` Offline mode — only use locally cached models - Destination: `offline` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1490-L1494](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1490-L1494) ### Parser `bench_parser` #### `model` Model to benchmark - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1497-L1497](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1497-L1497) #### `--num-prompts` Number of prompts - Destination: `num_prompts` - Required: `false` - Default: `10` - Choices: `not restricted` - Action: `store` - Source: [L1498-L1500](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1498-L1500) #### `--max-tokens` Max tokens per prompt - Destination: `max_tokens` - Required: `false` - Default: `100` - Choices: `not restricted` - Action: `store` - Source: [L1501-L1503](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1501-L1503) #### `--max-num-seqs` Max concurrent sequences - Destination: `max_num_seqs` - Required: `false` - Default: `32` - Choices: `not restricted` - Action: `store` - Source: [L1504-L1506](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1504-L1506) #### `--prefill-batch-size` Prefill batch size - Destination: `prefill_batch_size` - Required: `false` - Default: `8` - Choices: `not restricted` - Action: `store` - Source: [L1507-L1509](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1507-L1509) #### `--completion-batch-size` Completion batch size - Destination: `completion_batch_size` - Required: `false` - Default: `16` - Choices: `not restricted` - Action: `store` - Source: [L1510-L1512](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1510-L1512) #### `--enable-prefix-cache` Enable prefix caching (default: enabled) - Destination: `enable_prefix_cache` - Required: `false` - Default: `True` - Choices: `not restricted` - Action: `store_true` - Source: [L1513-L1518](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1513-L1518) #### `--disable-prefix-cache` Disable prefix caching - Destination: `disable_prefix_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1519-L1523](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1519-L1523) #### `--prefix-cache-size` Max entries in prefix cache (default: 100, legacy mode only) - Destination: `prefix_cache_size` - Required: `false` - Default: `100` - Choices: `not restricted` - Action: `store` - Source: [L1524-L1529](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1524-L1529) #### `--cache-memory-mb` Cache memory limit in MB (default: auto-detect ~20%% of RAM) - Destination: `cache_memory_mb` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1531-L1536](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1531-L1536) #### `--cache-memory-percent` Fraction of available RAM for cache if auto-detecting (default: 0.20) - Destination: `cache_memory_percent` - Required: `false` - Default: `0.2` - Choices: `not restricted` - Action: `store` - Source: [L1537-L1542](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1537-L1542) #### `--no-memory-aware-cache` Disable memory-aware cache, use legacy entry-count based cache - Destination: `no_memory_aware_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1543-L1547](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1543-L1547) #### `--kv-cache-quantization` Quantize stored KV caches to reduce memory (8-bit by default) - Destination: `kv_cache_quantization` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1549-L1553](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1549-L1553) #### `--kv-cache-quantization-bits` Bit width for KV cache quantization (default: 8) - Destination: `kv_cache_quantization_bits` - Required: `false` - Default: `8` - Choices: `[4, 8]` - Action: `store` - Source: [L1554-L1560](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1554-L1560) #### `--kv-cache-quantization-group-size` Group size for KV cache quantization (default: 64) - Destination: `kv_cache_quantization_group_size` - Required: `false` - Default: `64` - Choices: `not restricted` - Action: `store` - Source: [L1561-L1566](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1561-L1566) #### `--kv-cache-min-quantize-tokens` Minimum tokens for quantization to apply (default: 256) - Destination: `kv_cache_min_quantize_tokens` - Required: `false` - Default: `256` - Choices: `not restricted` - Action: `store` - Source: [L1567-L1572](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1567-L1572) #### `--use-paged-cache` Use paged KV cache for memory efficiency (experimental) - Destination: `use_paged_cache` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1574-L1578](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1574-L1578) #### `--paged-cache-block-size` Tokens per cache block (default: 64) - Destination: `paged_cache_block_size` - Required: `false` - Default: `64` - Choices: `not restricted` - Action: `store` - Source: [L1579-L1584](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1579-L1584) #### `--max-cache-blocks` Maximum number of cache blocks (default: 1000) - Destination: `max_cache_blocks` - Required: `false` - Default: `1000` - Choices: `not restricted` - Action: `store` - Source: [L1585-L1590](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1585-L1590) ### Parser `detok_parser` #### `model` Model to use for tokenizer (default: mlx-community/Qwen3-0.6B-8bit) - Destination: `model` - Required: `true` - Default: `mlx-community/Qwen3-0.6B-8bit` - Choices: `not restricted` - Action: `store` - Source: [L1596-L1602](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1596-L1602) #### `--iterations` Benchmark iterations (default: 5) - Destination: `iterations` - Required: `false` - Default: `5` - Choices: `not restricted` - Action: `store` - Source: [L1603-L1605](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1603-L1605) ### Parser `kv_cache_parser` #### `--layers` Number of layers (default: 32) - Destination: `layers` - Required: `false` - Default: `32` - Choices: `not restricted` - Action: `store` - Source: [L1611-L1613](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1611-L1613) #### `--seq-len` Sequence length (default: 512) - Destination: `seq_len` - Required: `false` - Default: `512` - Choices: `not restricted` - Action: `store` - Source: [L1614-L1616](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1614-L1616) #### `--heads` Number of attention heads (default: 32) - Destination: `heads` - Required: `false` - Default: `32` - Choices: `not restricted` - Action: `store` - Source: [L1617-L1619](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1617-L1619) #### `--head-dim` Head dimension (default: 128) - Destination: `head_dim` - Required: `false` - Default: `128` - Choices: `not restricted` - Action: `store` - Source: [L1620-L1622](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1620-L1622) #### `--group-size` Quantization group size (default: 64) - Destination: `group_size` - Required: `false` - Default: `64` - Choices: `not restricted` - Action: `store` - Source: [L1623-L1628](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1623-L1628) ### Parser `download_parser` #### `model` Model to download - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1634-L1634](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1634-L1634) #### `--timeout` Per-file download timeout in seconds (default: 300) - Destination: `timeout` - Required: `false` - Default: `300` - Choices: `not restricted` - Action: `store` - Source: [L1635-L1640](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1635-L1640) #### `--retries` Number of retry attempts (default: 3) - Destination: `retries` - Required: `false` - Default: `3` - Choices: `not restricted` - Action: `store` - Source: [L1641-L1646](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1641-L1646) #### `--mllm` Download as multimodal model (broader file patterns) - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1647-L1651](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1647-L1651) ### Parser `model_inspect_parser` #### `model` Local model path or Hugging Face model id - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1666-L1670](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1666-L1670) #### `--revision` Hugging Face revision to inspect - Destination: `revision` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1671-L1676](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1671-L1676) #### `--local-files-only` Use only local Hugging Face cache files - Destination: `local_files_only` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1677-L1681](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1677-L1681) ### Parser `model_acquire_parser` #### `model` Hugging Face model id - Destination: `model` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1687-L1687](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1687-L1687) #### `--revision` Hugging Face revision to download - Destination: `revision` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1688-L1693](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1688-L1693) #### `--target-dir` Final local directory. Defaults to Hugging Face cache. - Destination: `target_dir` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1694-L1699](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1694-L1699) #### `--staging-dir` Directory for temporary staged downloads before finalizing target-dir - Destination: `staging_dir` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1700-L1705](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1700-L1705) #### `--mllm` Acquire multimodal model files using broader allow patterns - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1706-L1710](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1706-L1710) #### `--no-fast-transfer` Do not set HF_HUB_ENABLE_HF_TRANSFER=1 during download - Destination: `no_fast_transfer` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1711-L1715](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1711-L1715) #### `--local-files-only` Use only local Hugging Face cache files - Destination: `local_files_only` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1716-L1720](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1716-L1720) ### Parser `model_convert_parser` #### `source` Hugging Face model id or local source path - Destination: `source` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1726-L1730](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1726-L1730) #### `--output` Output directory for the converted MLX model - Destination: `output` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1731-L1736](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1731-L1736) #### `--quantize` Generate a quantized MLX model - Destination: `quantize` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1737-L1741](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1737-L1741) #### `--q-bits` Quantization bit width (e.g. 3, 4, 8) - Destination: `q_bits` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1742-L1747](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1742-L1747) #### `--q-group-size` Quantization group size (default: mlx-lm default) - Destination: `q_group_size` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1748-L1753](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1748-L1753) #### `--q-mode` No argparse help text is declared. - Destination: `q_mode` - Required: `false` - Default: `None` - Choices: `['affine', 'mxfp4', 'nvfp4', 'mxfp8']` - Action: `store` - Source: [L1754-L1758](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1754-L1758) #### `--quant-predicate` mlx-lm mixed-bit quantization recipe - Destination: `quant_predicate` - Required: `false` - Default: `None` - Choices: `['mixed_2_6', 'mixed_3_4', 'mixed_3_6', 'mixed_4_6']` - Action: `store` - Source: [L1759-L1764](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1759-L1764) #### `--dtype` Non-quantized parameter dtype - Destination: `dtype` - Required: `false` - Default: `None` - Choices: `['float16', 'bfloat16', 'float32']` - Action: `store` - Source: [L1765-L1770](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1765-L1770) #### `--trust-remote-code` Allow Hugging Face remote code during mlx-lm conversion - Destination: `trust_remote_code` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1771-L1775](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1771-L1775) #### `--dry-run` Print the conversion command and manifest without executing - Destination: `dry_run` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1776-L1780](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1776-L1780) ### Parser `model_register_parser` #### `artifact` Finalized local model artifact directory - Destination: `artifact` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1786-L1790](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1786-L1790) #### `--model-id` Override model ID (default: directory name of artifact) - Destination: `model_id` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1791-L1796](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1791-L1796) #### `--served-model-name` Model name exposed by the API (default: model-id) - Destination: `served_model_name` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1797-L1802](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1797-L1802) #### `--preset-alias` Optional alias for preset lookup in registry - Destination: `preset_alias` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1803-L1808](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1803-L1808) #### `--output` Manifest path. Defaults to artifact/vllm_mlx_registration_manifest.json - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1809-L1814](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1809-L1814) ### Parser `mllm_group` #### `--mllm` Mark the artifact as an MLLM serving candidate - Destination: `mllm` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store_true` - Source: [L1816-L1821](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1816-L1821) #### `--no-mllm` Explicitly mark the artifact as text-only - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_false` - Source: [L1822-L1827](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1822-L1827) ### Parser `model_register_parser` #### `--tool-call-parser` Tool call parser name for the model (e.g. qwen3_coder, mistral) - Destination: `tool_call_parser` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1828-L1833](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1828-L1833) #### `--reasoning-parser` Reasoning parser name for thinking models (e.g. qwen3) - Destination: `reasoning_parser` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1834-L1839](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1834-L1839) #### `--default-temperature` Default temperature for all requests - Destination: `default_temperature` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1840-L1845](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1840-L1845) #### `--default-top-p` Default top_p for all requests - Destination: `default_top_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1846-L1851](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1846-L1851) #### `--default-top-k` Default top_k for all requests - Destination: `default_top_k` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1852-L1857](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1852-L1857) #### `--default-min-p` Default min_p for all requests - Destination: `default_min_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1858-L1863](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1858-L1863) #### `--default-presence-penalty` Default presence_penalty for all requests - Destination: `default_presence_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1864-L1869](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1864-L1869) #### `--default-repetition-penalty` Default repetition_penalty for all requests - Destination: `default_repetition_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1870-L1875](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1870-L1875) #### `--default-chat-template-kwargs` Default chat template kwargs as JSON, e.g. {"enable_thinking": true} - Destination: `default_chat_template_kwargs` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1876-L1881](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1876-L1881) #### `--feature-flag` Feature flag to record in the registration manifest. Repeatable. - Destination: `feature_flag` - Required: `false` - Default: `[]` - Choices: `not restricted` - Action: `append` - Source: [L1882-L1887](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1882-L1887) ### Parser `model_qualify_parser` #### `model_id` Model ID to qualify against the running server - Destination: `model_id` - Required: `true` - Default: `argparse default` - Choices: `not restricted` - Action: `store` - Source: [L1893-L1897](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1893-L1897) #### `--url` Running server URL for bench-serve - Destination: `url` - Required: `false` - Default: `http://127.0.0.1:8080` - Choices: `not restricted` - Action: `store` - Source: [L1898-L1903](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1898-L1903) #### `--workload` bench-serve workload contract path - Destination: `workload` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1904-L1909](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1904-L1909) #### `--output` Qualification request manifest path - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1910-L1915](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1910-L1915) #### `--result-output` Result output path passed to bench-serve - Destination: `result_output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1916-L1921](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1916-L1921) #### `--repetitions` Number of repetitions per benchmark sweep configuration - Destination: `repetitions` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1922-L1927](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1922-L1927) #### `--dry-run` Write or print the qualification command without running it - Destination: `dry_run` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1928-L1932](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1928-L1932) #### `--extra-arg` Extra argument passed through to bench-serve. Repeatable. - Destination: `extra_arg` - Required: `false` - Default: `[]` - Choices: `not restricted` - Action: `append` - Source: [L1933-L1938](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1933-L1938) ### Parser `bench_serve_parser` #### `--url` Base URL of the running server (default: http://127.0.0.1:8080) - Destination: `url` - Required: `false` - Default: `http://127.0.0.1:8080` - Choices: `not restricted` - Action: `store` - Source: [L1944-L1949](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1944-L1949) #### `--model` Model ID to benchmark (default: auto-detected from server) - Destination: `model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1950-L1955](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1950-L1955) #### `--workload` Path to a declarative workload JSON file. When set, bench-serve runs contract-style cases with per-case quality checks and comparison-only policy timeouts instead of the prompt sweep. - Destination: `workload` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1956-L1965](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1956-L1965) #### `--prompts` Comma-separated prompt set names or paths (default: short,medium,long) - Destination: `prompts` - Required: `false` - Default: `short,medium,long` - Choices: `not restricted` - Action: `store` - Source: [L1966-L1971](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1966-L1971) #### `--prompt-file` Path to an additional prompt file (JSON list of message dicts) - Destination: `prompt_file` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1972-L1977](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1972-L1977) #### `--system-prompt-file` Path to a text file whose contents are prepended as a system message to every prompt. Use this together with --warm-prompts to benchmark the warm-cache path (the warmup populates the prefix cache with this same system, so every request in the bench hits the cache). - Destination: `system_prompt_file` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L1978-L1989](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1978-L1989) #### `--skip-preflight-token-count` Skip the pre-flight max_tokens=1 request that counts prompt tokens per prompt set. That request populates the prefix cache with the full prompt, which defeats cold-vs-warm comparisons. Auto-enabled when --system-prompt-file is set; pass this flag explicitly to force-enable regardless. - Destination: `skip_preflight_token_count` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L1990-L2000](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1990-L2000) #### `--concurrency` Comma-separated concurrency levels to sweep (default: 1,4) - Destination: `concurrency` - Required: `false` - Default: `1,4` - Choices: `not restricted` - Action: `store` - Source: [L2001-L2006](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2001-L2006) #### `--max-tokens` Maximum tokens to generate per request (default: 256) - Destination: `max_tokens` - Required: `false` - Default: `256` - Choices: `not restricted` - Action: `store` - Source: [L2007-L2012](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2007-L2012) #### `--repetitions` Number of repetitions per sweep configuration or workload case (default: 3) - Destination: `repetitions` - Required: `false` - Default: `3` - Choices: `not restricted` - Action: `store` - Source: [L2013-L2018](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2013-L2018) #### `--warmup` Warmup rounds before the first measured repetition (default: 1) - Destination: `warmup` - Required: `false` - Default: `1` - Choices: `not restricted` - Action: `store` - Source: [L2019-L2024](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2019-L2024) #### `--enable-thinking` Enable thinking mode: "true", "false", or "true,false" to sweep both - Destination: `enable_thinking` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L2025-L2030](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2025-L2030) #### `--extra-body` Comma-separated JSON dicts to pass as extra body parameters - Destination: `extra_body` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L2031-L2036](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2031-L2036) #### `--output` File path to write results to (default: stdout) - Destination: `output` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L2037-L2042](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2037-L2042) #### `--format` Output format (auto = table for prompt sweeps, json for workloads; sqlite requires --output) - Destination: `format` - Required: `false` - Default: `auto` - Choices: `['auto', 'table', 'json', 'csv', 'sql', 'sqlite']` - Action: `store` - Source: [L2043-L2052](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2043-L2052) #### `--validate` Validate responses (default: true) - Destination: `validate` - Required: `false` - Default: `true` - Choices: `['true', 'false']` - Action: `store` - Source: [L2053-L2059](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2053-L2059) #### `--scrape-metrics` Scrape /metrics before and after each run (default: true) - Destination: `scrape_metrics` - Required: `false` - Default: `true` - Choices: `['true', 'false']` - Action: `store` - Source: [L2060-L2066](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2060-L2066) #### `--include-content` Include full generated content in workload JSON output - Destination: `include_content` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L2067-L2071](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2067-L2071) #### `--request-timeout-s` HTTP transport timeout for workload mode in seconds (default: 300). Use 0 to disable; product policy timeouts belong in the workload. - Destination: `request_timeout_s` - Required: `false` - Default: `300.0` - Choices: `not restricted` - Action: `store` - Source: [L2072-L2080](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2072-L2080) #### `--cache-policy` Workload cache handling (default: workload defaults or preserve). Use before-case for cold, uncontaminated per-case qualification. Workload JSON may also spell these with underscores. - Destination: `cache_policy` - Required: `false` - Default: `None` - Choices: `['preserve', 'before-run', 'before-case']` - Action: `store` - Source: [L2081-L2091](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2081-L2091) #### `--tag` Optional tag string stored in every result row - Destination: `tag` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L2092-L2097](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2092-L2097) #### `--override-field` Override result fields as key=value pairs (e.g. chip=M4Pro) - Destination: `override_field` - Required: `false` - Default: `[]` - Choices: `not restricted` - Action: `store` - Source: [L2098-L2103](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2098-L2103) ## `vllm_mlx.gradio_app.main` ### Parser `parser` #### `--server-url` URL of the vllm-mlx server (default: http://localhost:8000) - Destination: `server_url` - Required: `false` - Default: `http://localhost:8000` - Choices: `not restricted` - Action: `store` - Source: [L280-L285](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L280-L285) #### `--port` Port for Gradio interface (default: 7860) - Destination: `port` - Required: `false` - Default: `7860` - Choices: `not restricted` - Action: `store` - Source: [L286-L291](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L286-L291) #### `--share` Create a public share link - Destination: `share` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L292-L296](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L292-L296) #### `--max-tokens` Maximum tokens to generate (default: 2048) - Destination: `max_tokens` - Required: `false` - Default: `2048` - Choices: `not restricted` - Action: `store` - Source: [L297-L302](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L297-L302) #### `--temperature` Sampling temperature (default: 0.7) - Destination: `temperature` - Required: `false` - Default: `0.7` - Choices: `not restricted` - Action: `store` - Source: [L303-L308](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L303-L308) #### `--served-model-name` Model name to send in /v1/chat/completions requests (default: default) - Destination: `served_model_name` - Required: `false` - Default: `default` - Choices: `not restricted` - Action: `store` - Source: [L309-L316](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L309-L316) #### `--text-only` Use text-only mode (no image/video support, faster for LLM-only models) - Destination: `text_only` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L317-L321](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_app.py#L317-L321) ## `vllm_mlx.gradio_text_app.main` ### Parser `parser` #### `--server-url` URL of the vllm-mlx server (default: http://localhost:8000) - Destination: `server_url` - Required: `false` - Default: `http://localhost:8000` - Choices: `not restricted` - Action: `store` - Source: [L136-L141](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L136-L141) #### `--port` Port for Gradio interface (default: 7861) - Destination: `port` - Required: `false` - Default: `7861` - Choices: `not restricted` - Action: `store` - Source: [L142-L147](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L142-L147) #### `--share` Create a public share link - Destination: `share` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L148-L152](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L148-L152) #### `--max-tokens` Maximum tokens to generate (default: 512) - Destination: `max_tokens` - Required: `false` - Default: `512` - Choices: `not restricted` - Action: `store` - Source: [L153-L158](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L153-L158) #### `--temperature` Sampling temperature (default: 0.7) - Destination: `temperature` - Required: `false` - Default: `0.7` - Choices: `not restricted` - Action: `store` - Source: [L159-L164](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L159-L164) #### `--served-model-name` Model name to send in /v1/chat/completions requests (default: default) - Destination: `served_model_name` - Required: `false` - Default: `default` - Choices: `not restricted` - Action: `store` - Source: [L165-L172](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/gradio_text_app.py#L165-L172) ## `vllm_mlx.server.create_parser` ### Parser `parser` #### `--model` Model to load (HuggingFace model name or local path) - Destination: `model` - Required: `false` - Default: `mlx-community/Llama-3.2-3B-Instruct-4bit` - Choices: `not restricted` - Action: `store` - Source: [L6728-L6733](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6728-L6733) #### `--host` Host to bind to (default: localhost; use 0.0.0.0 to expose externally) - Destination: `host` - Required: `false` - Default: `127.0.0.1` - Choices: `not restricted` - Action: `store` - Source: [L6734-L6739](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6734-L6739) #### `--port` Port to bind to - Destination: `port` - Required: `false` - Default: `8000` - Choices: `not restricted` - Action: `store` - Source: [L6740-L6745](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6740-L6745) #### `--mllm` Force loading as MLLM (multimodal language model) - Destination: `mllm` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L6746-L6750](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6746-L6750) #### `--trust-remote-code` Allow HuggingFace remote code execution during model/tokenizer loading - Destination: `trust_remote_code` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L6751-L6755](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6751-L6755) #### `--continuous-batching` Enable continuous batching for multiple concurrent users - Destination: `continuous_batching` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L6756-L6760](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6756-L6760) #### `--mllm-draft-model` Path to an mlx-vlm MLLM draft/assistant model. - Destination: `mllm_draft_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6761-L6766](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6761-L6766) #### `--mllm-draft-kind` mlx-vlm draft kind for --mllm-draft-model. - Destination: `mllm_draft_kind` - Required: `false` - Default: `None` - Choices: `['mtp']` - Action: `store` - Source: [L6767-L6773](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6767-L6773) #### `--mllm-draft-block-size` Draft block size passed to mlx-vlm for --mllm-draft-model. - Destination: `mllm_draft_block_size` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6774-L6779](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6774-L6779) #### `--mcp-config` Path to MCP configuration file (JSON/YAML) - Destination: `mcp_config` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6780-L6785](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6780-L6785) #### `--max-tokens` Default max tokens for generation - Destination: `max_tokens` - Required: `false` - Default: `32768` - Choices: `not restricted` - Action: `store` - Source: [L6786-L6791](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6786-L6791) #### `--max-request-tokens` Maximum max_tokens accepted from API clients (default: 32768) - Destination: `max_request_tokens` - Required: `false` - Default: `32768` - Choices: `not restricted` - Action: `store` - Source: [L6792-L6797](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6792-L6797) #### `--api-key` API key for authentication (if not set, no auth required) - Destination: `api_key` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6798-L6803](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6798-L6803) #### `--timeout` Default request timeout in seconds (default: 300) - Destination: `timeout` - Required: `false` - Default: `300.0` - Choices: `not restricted` - Action: `store` - Source: [L6804-L6809](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6804-L6809) #### `--enable-metrics` Expose Prometheus metrics on /metrics (disabled by default) - Destination: `enable_metrics` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L6810-L6814](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6810-L6814) #### `--auto-unload-idle-seconds` Unload the main model after this many idle seconds (0 = disabled) - Destination: `auto_unload_idle_seconds` - Required: `false` - Default: `0.0` - Choices: `not restricted` - Action: `store` - Source: [L6815-L6820](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6815-L6820) #### `--lazy-load-model` Register the main model at startup but defer loading until first request - Destination: `lazy_load_model` - Required: `false` - Default: `argparse default` - Choices: `not restricted` - Action: `store_true` - Source: [L6821-L6825](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6821-L6825) #### `--rate-limit` Rate limit requests per minute per client (0 = disabled) - Destination: `rate_limit` - Required: `false` - Default: `0` - Choices: `not restricted` - Action: `store` - Source: [L6826-L6831](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6826-L6831) #### `--reasoning-parser` f"Enable reasoning content extraction with specified parser. Options: {', '.join(reasoning_choices)}." - Destination: `reasoning_parser` - Required: `false` - Default: `None` - Choices: `reasoning_choices` - Action: `store` - Source: [L6836-L6845](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6836-L6845) #### `--embedding-model` Pre-load an embedding model at startup (e.g. mlx-community/all-MiniLM-L6-v2-4bit) - Destination: `embedding_model` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6846-L6851](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6846-L6851) #### `--default-temperature` Default temperature for generation when not specified in request - Destination: `default_temperature` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6852-L6857](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6852-L6857) #### `--default-top-p` Default top_p for generation when not specified in request - Destination: `default_top_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6858-L6863](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6858-L6863) #### `--default-chat-template-kwargs` Default chat template kwargs to apply to all requests when request chat_template_kwargs is omitted or empty; empty request kwargs use existing server defaults (JSON object, e.g. {"enable_thinking": false}) - Destination: `default_chat_template_kwargs` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6864-L6873](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6864-L6873) #### `--default-top-k` Default top_k for generation when not specified in request - Destination: `default_top_k` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6874-L6879](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6874-L6879) #### `--default-min-p` Default min_p for generation when not specified in request - Destination: `default_min_p` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6880-L6885](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6880-L6885) #### `--default-presence-penalty` Default presence_penalty for generation when not specified in request - Destination: `default_presence_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6886-L6891](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6886-L6891) #### `--default-repetition-penalty` Default repetition_penalty for generation when not specified in request - Destination: `default_repetition_penalty` - Required: `false` - Default: `None` - Choices: `not restricted` - Action: `store` - Source: [L6892-L6899](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6892-L6899) #### `--max-audio-upload-mb` Maximum size of uploaded audio files in MiB (default: 25) - Destination: `max_audio_upload_mb` - Required: `false` - Default: `DEFAULT_MAX_AUDIO_UPLOAD_MB` - Choices: `not restricted` - Action: `store` - Source: [L6900-L6905](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6900-L6905) #### `--max-tts-input-chars` Maximum number of characters accepted by /v1/audio/speech (default: 4096) - Destination: `max_tts_input_chars` - Required: `false` - Default: `DEFAULT_MAX_TTS_INPUT_CHARS` - Choices: `not restricted` - Action: `store` - Source: [L6906-L6911](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/server.py#L6906-L6911)