# `vllm_mlx.cli`

CLI for vllm-mlx.

[View the complete module source at #L1-L2138](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L1-L2138).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: vllm_mlx.cli
    options:
      members:
        - serve_command
        - download_command
        - model_command
        - bench_command
        - bench_detok_command
        - bench_kv_cache_command
        - bench_serve_command
        - create_parser
        - build_parser
        - main
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-vllm_mlx.cli.serve_command" markdown="1">
<summary><code>vllm_mlx.cli.serve_command</code> · function</summary>

```python
vllm_mlx.cli.serve_command(args) -> not annotated
```

Start the OpenAI-compatible server.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `serve_command` calls `logging.getLogger`, `getattr`, `print`, `sys.exit`.
No direct `raise` statement appears in this definition.

[View source #L22-L393](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L22-L393).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.download_command" markdown="1">
<summary><code>vllm_mlx.cli.download_command</code> · function</summary>

```python
vllm_mlx.cli.download_command(args) -> not annotated
```

Download a model to local cache without starting a server.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `download_command` calls `DownloadConfig`, `print`, `ensure_model_downloaded`.
No direct `raise` statement appears in this definition.

[View source #L396-L410](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L396-L410).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.model_command" markdown="1">
<summary><code>vllm_mlx.cli.model_command</code> · function</summary>

```python
vllm_mlx.cli.model_command(args) -> not annotated
```

Run model lifecycle helper commands.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `model_command` calls `inspect_model`, `acquire_model`, `AcquisitionOptions`, `convert_model`; can raise `ValueError`.
Directly raised exceptions: `ValueError`.

[View source #L413-L503](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L413-L503).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_command" markdown="1">
<summary><code>vllm_mlx.cli.bench_command</code> · function</summary>

```python
vllm_mlx.cli.bench_command(args) -> not annotated
```

Run benchmark.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `bench_command` calls `asyncio.run`, `run_benchmark`.
No direct `raise` statement appears in this definition.

[View source #L506-L625](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L506-L625).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_command.run_benchmark" markdown="1">
<summary><code>vllm_mlx.cli.bench_command.run_benchmark</code> · nested function</summary>

```python
async vllm_mlx.cli.bench_command.run_benchmark() -> not annotated
```

Nested Function `bench_command.run_benchmark` calls `print`, `load`, `SchedulerConfig`, `EngineConfig`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Nested Function `bench_command.run_benchmark` calls `print`, `load`, `SchedulerConfig`, `EngineConfig`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L520-L623](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L520-L623).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_command.run_benchmark.get_output" markdown="1">
<summary><code>vllm_mlx.cli.bench_command.run_benchmark.get_output</code> · nested function</summary>

```python
async vllm_mlx.cli.bench_command.run_benchmark.get_output(rid) -> not annotated
```

Nested Function `bench_command.run_benchmark.get_output` calls `engine.stream_outputs`; has 2 explicit return paths.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `rid` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `out`; `None`

**Exceptions and behavior**

Nested Function `bench_command.run_benchmark.get_output` calls `engine.stream_outputs`; has 2 explicit return paths.
No direct `raise` statement appears in this definition.

[View source #L597-L601](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L597-L601).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_detok_command" markdown="1">
<summary><code>vllm_mlx.cli.bench_detok_command</code> · function</summary>

```python
vllm_mlx.cli.bench_detok_command(args) -> not annotated
```

Benchmark streaming detokenizer optimization.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `bench_detok_command` calls `print`, `load`, `generate`, `tokenizer.encode`.
No direct `raise` statement appears in this definition.

[View source #L628-L740](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L628-L740).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_kv_cache_command" markdown="1">
<summary><code>vllm_mlx.cli.bench_kv_cache_command</code> · function</summary>

```python
vllm_mlx.cli.bench_kv_cache_command(args) -> not annotated
```

Benchmark KV cache quantization memory savings and quality.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `bench_kv_cache_command` calls `print`, `range`, `KVCache`, `mx.random.normal`.
No direct `raise` statement appears in this definition.

[View source #L743-L886](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L743-L886).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.bench_serve_command" markdown="1">
<summary><code>vllm_mlx.cli.bench_serve_command</code> · function</summary>

```python
vllm_mlx.cli.bench_serve_command(args) -> not annotated
```

Run serving benchmark.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `args` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `None`

**Exceptions and behavior**

Function `bench_serve_command` calls `sweep_only_warnings.append`, `print`, `', '.join`, `asyncio.run`; returns `None`.
No direct `raise` statement appears in this definition.

[View source #L889-L990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L889-L990).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.create_parser" markdown="1">
<summary><code>vllm_mlx.cli.create_parser</code> · function</summary>

```python
vllm_mlx.cli.create_parser() -> argparse.ArgumentParser
```

Build the top-level CLI parser.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `argparse.ArgumentParser`
- Direct return expressions: `parser`

**Exceptions and behavior**

Function `create_parser` calls `argparse.ArgumentParser`, `parser.add_subparsers`, `subparsers.add_parser`, `serve_parser.add_argument`; returns `parser`.
No direct `raise` statement appears in this definition.

[View source #L993-L2105](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L993-L2105).

</details>

<details class="api-contract" id="contract-vllm_mlx.cli.main" markdown="1">
<summary><code>vllm_mlx.cli.main</code> · function</summary>

```python
vllm_mlx.cli.main() -> not annotated
```

Parse the command line and dispatch to the selected vllm-mlx command.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `main` calls `create_parser`, `parser.parse_args`, `serve_command`, `bench_command`.
No direct `raise` statement appears in this definition.

[View source #L2112-L2134](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2112-L2134).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`serve_command`](#contract-vllm_mlx.cli.serve_command) | function | `serve_command(args) -> not annotated` | Start the OpenAI-compatible server. | [#L22-L393](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L22-L393) |
| [`download_command`](#contract-vllm_mlx.cli.download_command) | function | `download_command(args) -> not annotated` | Download a model to local cache without starting a server. | [#L396-L410](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L396-L410) |
| [`model_command`](#contract-vllm_mlx.cli.model_command) | function | `model_command(args) -> not annotated` | Run model lifecycle helper commands. | [#L413-L503](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L413-L503) |
| [`bench_command`](#contract-vllm_mlx.cli.bench_command) | function | `bench_command(args) -> not annotated` | Run benchmark. | [#L506-L625](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L506-L625) |
| [`bench_command.run_benchmark`](#contract-vllm_mlx.cli.bench_command.run_benchmark) | nested function | `async bench_command.run_benchmark() -> not annotated` | Nested Function `bench_command.run_benchmark` calls `print`, `load`, `SchedulerConfig`, `EngineConfig`; awaits asynchronous work. | [#L520-L623](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L520-L623) |
| [`bench_command.run_benchmark.get_output`](#contract-vllm_mlx.cli.bench_command.run_benchmark.get_output) | nested function | `async bench_command.run_benchmark.get_output(rid) -> not annotated` | Nested Function `bench_command.run_benchmark.get_output` calls `engine.stream_outputs`; has 2 explicit return paths. | [#L597-L601](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L597-L601) |
| [`bench_detok_command`](#contract-vllm_mlx.cli.bench_detok_command) | function | `bench_detok_command(args) -> not annotated` | Benchmark streaming detokenizer optimization. | [#L628-L740](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L628-L740) |
| [`bench_kv_cache_command`](#contract-vllm_mlx.cli.bench_kv_cache_command) | function | `bench_kv_cache_command(args) -> not annotated` | Benchmark KV cache quantization memory savings and quality. | [#L743-L886](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L743-L886) |
| [`bench_serve_command`](#contract-vllm_mlx.cli.bench_serve_command) | function | `bench_serve_command(args) -> not annotated` | Run serving benchmark. | [#L889-L990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L889-L990) |
| [`create_parser`](#contract-vllm_mlx.cli.create_parser) | function | `create_parser() -> argparse.ArgumentParser` | Build the top-level CLI parser. | [#L993-L2105](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L993-L2105) |
| [`main`](#contract-vllm_mlx.cli.main) | function | `main() -> not annotated` | Parse the command line and dispatch to the selected vllm-mlx command. | [#L2112-L2134](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/cli.py#L2112-L2134) |