# `examples.test_batching`

Example: Test continuous batching with vllm-mlx.

[View the complete module source at #L1-L195](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L1-L195).

## API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

::: examples.test_batching
    options:
      members:
        - run_single_request
        - run_concurrent_requests
        - print_results
        - main
      filters: []
      show_if_no_docstring: true

## Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

<details class="api-contract" id="contract-examples.test_batching.run_single_request" markdown="1">
<summary><code>examples.test_batching.run_single_request</code> · function</summary>

```python
async examples.test_batching.run_single_request(engine: AsyncEngineCore, request_id: str, prompt: str, sampling_params: SamplingParams) -> dict
```

Run a single request and collect timing.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `AsyncEngineCore` | `yes` | `none` | Required positional or keyword input. |
| `request_id` | `str` | `yes` | `none` | Required positional or keyword input. |
| `prompt` | `str` | `yes` | `none` | Required positional or keyword input. |
| `sampling_params` | `SamplingParams` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `dict`
- Direct return expressions: `{'request_id': request_id, 'prompt_length': len(prompt.split()), 'num_tokens': len(tokens), 'ttft': ttft, 'total_time':…`

**Exceptions and behavior**

Function `run_single_request` calls `time.perf_counter`, `engine.add_request`, `engine.stream_outputs`, `tokens.extend`; awaits asynchronous work; returns `{'request_id': request_id, 'prompt_length': len(prompt.split()), 'num_tokens': len(tokens), 'ttft': ttft, 'total_time':…`.
No direct `raise` statement appears in this definition.

[View source #L29-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L29-L65).

</details>

<details class="api-contract" id="contract-examples.test_batching.run_concurrent_requests" markdown="1">
<summary><code>examples.test_batching.run_concurrent_requests</code> · function</summary>

```python
async examples.test_batching.run_concurrent_requests(engine: AsyncEngineCore, prompts: List[str], sampling_params: SamplingParams) -> List[dict]
```

Run multiple requests concurrently.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `engine` | `AsyncEngineCore` | `yes` | `none` | Required positional or keyword input. |
| `prompts` | `List[str]` | `yes` | `none` | Required positional or keyword input. |
| `sampling_params` | `SamplingParams` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `List[dict]`
- Direct return expressions: `await asyncio.gather(*tasks)`

**Exceptions and behavior**

Function `run_concurrent_requests` calls `enumerate`, `run_single_request`, `tasks.append`, `asyncio.gather`; awaits asynchronous work; returns `await asyncio.gather(*tasks)`.
No direct `raise` statement appears in this definition.

[View source #L68-L79](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L68-L79).

</details>

<details class="api-contract" id="contract-examples.test_batching.print_results" markdown="1">
<summary><code>examples.test_batching.print_results</code> · function</summary>

```python
examples.test_batching.print_results(results: List[dict], total_time: float) -> not annotated
```

Print benchmark results.

**Parameters**

| Name | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `results` | `List[dict]` | `yes` | `none` | Required positional or keyword input. |
| `total_time` | `float` | `yes` | `none` | Required positional or keyword input. |

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `print_results` calls `print`, `sum`, `len`.
No direct `raise` statement appears in this definition.

[View source #L82-L113](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L82-L113).

</details>

<details class="api-contract" id="contract-examples.test_batching.main" markdown="1">
<summary><code>examples.test_batching.main</code> · function</summary>

```python
async examples.test_batching.main() -> not annotated
```

Function `main` calls `argparse.ArgumentParser`, `parser.add_argument`, `parser.parse_args`, `print`; awaits asynchronous work.

**Parameters**

This callable has no explicit inputs.

**Returns**

- Type: `not annotated`

**Exceptions and behavior**

Function `main` calls `argparse.ArgumentParser`, `parser.add_argument`, `parser.parse_args`, `print`; awaits asynchronous work.
No direct `raise` statement appears in this definition.

[View source #L116-L191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L116-L191).

</details>

## Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

| Symbol | Kind | Signature and inputs | What it does | Source |
| --- | --- | --- | --- | --- |
| [`run_single_request`](#contract-examples.test_batching.run_single_request) | function | `async run_single_request(engine: AsyncEngineCore, request_id: str, prompt: str, sampling_params: SamplingParams) -> dict` | Run a single request and collect timing. | [#L29-L65](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L29-L65) |
| [`run_concurrent_requests`](#contract-examples.test_batching.run_concurrent_requests) | function | `async run_concurrent_requests(engine: AsyncEngineCore, prompts: List[str], sampling_params: SamplingParams) -> List[dict]` | Run multiple requests concurrently. | [#L68-L79](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L68-L79) |
| [`print_results`](#contract-examples.test_batching.print_results) | function | `print_results(results: List[dict], total_time: float) -> not annotated` | Print benchmark results. | [#L82-L113](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L82-L113) |
| [`main`](#contract-examples.test_batching.main) | function | `async main() -> not annotated` | Function `main` calls `argparse.ArgumentParser`, `parser.add_argument`, `parser.parse_args`, `print`; awaits asynchronous work. | [#L116-L191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/examples/test_batching.py#L116-L191) |