# `examples.benchmark_all_models`

Benchmark all text models for README.

[View the complete module source at #L1-L148](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L1-L148).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: examples.benchmark_all_models
    options:
      members:
        - benchmark_model
        - main
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-examples.benchmark_all_models.benchmark_model" markdown="1">
<summary><code>examples.benchmark_all_models.benchmark_model</code> · function</summary>

```python
examples.benchmark_all_models.benchmark_model(model_name: str) -> not annotated
```

Benchmark a single model and return results.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `model_name` | `str` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `{'model': model_name.split('/')[-1], 'single_tps': single_tps, 'batch_tps': batch_tps, 'speedup': speedup, 'ttft_ms': t…`

**Exceptions and behavior**

Function `benchmark_model` calls `SamplingParams`, `print`, `load`, `format_prompt`; returns `{'model': model_name.split('/')[-1], 'single_tps': single_tps, 'batch_tps': batch_tps, 'speedup': speedup, 'ttft_ms': t…`.
No direct `raise` statement appears in this definition.

[View source #L7-L107](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L7-L107).

</details>

<details class="api-contract" id="contract-examples.benchmark_all_models.benchmark_model.format_prompt" markdown="1">
<summary><code>examples.benchmark_all_models.benchmark_model.format_prompt</code> · nested function</summary>

```python
examples.benchmark_all_models.benchmark_model.format_prompt(p) -> not annotated
```

Nested Function `benchmark_model.format_prompt` calls `tokenizer.apply_chat_template`; returns `tokenizer.apply_chat_template([{'role': 'user', 'content': p}], tokenize=False, add_generation_prompt=True)`.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `p` | `not annotated` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`
- Direct return expressions: `tokenizer.apply_chat_template([{'role': 'user', 'content': p}], tokenize=False, add_generation_prompt=True)`

**Exceptions and behavior**

Nested Function `benchmark_model.format_prompt` calls `tokenizer.apply_chat_template`; returns `tokenizer.apply_chat_template([{'role': 'user', 'content': p}], tokenize=False, add_generation_prompt=True)`.
No direct `raise` statement appears in this definition.

[View source #L29-L34](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L29-L34).

</details>

<details class="api-contract" id="contract-examples.benchmark_all_models.main" markdown="1">
<summary><code>examples.benchmark_all_models.main</code> · function</summary>

```python
examples.benchmark_all_models.main() -> not annotated
```

Function `main` calls `benchmark_model`, `results.append`, `print`, `traceback.print_exc`.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `main` calls `benchmark_model`, `results.append`, `print`, `traceback.print_exc`.
No direct `raise` statement appears in this definition.

[View source #L110-L144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L110-L144).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`benchmark_model`](#contract-examples.benchmark_all_models.benchmark_model) | function | `benchmark_model(model_name: str) -> not annotated` | Benchmark a single model and return results. | [#L7-L107](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L7-L107) |
| [`benchmark_model.format_prompt`](#contract-examples.benchmark_all_models.benchmark_model.format_prompt) | nested function | `benchmark_model.format_prompt(p) -> not annotated` | Nested Function `benchmark_model.format_prompt` calls `tokenizer.apply_chat_template`; returns `tokenizer.apply_chat_template([{'role': 'user', 'content': p}], tokenize=False, add_generation_prompt=True)`. | [#L29-L34](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L29-L34) |
| [`main`](#contract-examples.benchmark_all_models.main) | function | `main() -> not annotated` | Function `main` calls `benchmark_model`, `results.append`, `print`, `traceback.print_exc`. | [#L110-L144](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/benchmark_all_models.py#L110-L144) |