# `vllm_mlx.bench_serve` Serving benchmark for vllm-mlx. [View the complete module source at #L1-L2638](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1-L2638). ## API details Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them. ::: vllm_mlx.bench_serve options: members: - _BUILTIN_DIR - _BUILTIN_NAMES - _SQL_IDENTIFIER_RE - WorkloadCase - Workload - load_prompt_set - _require_message_list - _load_case_request - _request_extra_body - _first_not_none - _normalize_tags - _merge_case_checks - _build_workload_case - load_workload - BenchServeResult - SweepConfig - expand_sweep - parse_health_response - parse_status_response - parse_metrics_text - detect_hardware_fingerprint - auto_detect_runtime - scrape_metrics - clear_runtime_cache - _normalize_cache_policy - parse_sse_line - _cancel_server_request - accumulate_tool_calls - finalize_tool_calls - compute_request_metrics - count_prompt_tokens - stream_chat_completion - validate_response - _check_finish_reason - _check_length_bounds - _check_regex_patterns - _check_json_content - _check_tool_call_count_and_names - _check_tool_call_args - validate_quality_checks - compute_summary_stats - run_concurrent_requests - _summary_or_empty - _resolve_max_tokens - _assemble_case_request_kwargs - _empty_completion_result - _fetch_post_run_status - _compute_within_policy_timeout - _build_tool_calls_summary - _build_workload_record - run_workload_case - _group_results_by_case_id - _summarize_case - summarize_workload_results - run_bench_serve_workload - RESULT_COLUMNS - _TABLE_COLUMNS - _result_to_dict - format_table - format_json - format_csv - _sql_escape - _SQL_SCHEMA - format_sql - _write_sqlite_rows - _validate_sql_identifier - write_sqlite - WORKLOAD_RESULT_COLUMNS - _WORKLOAD_TABLE_COLUMNS - _workload_record_to_row - format_workload_table - format_workload_json - format_workload_csv - _WORKLOAD_SQL_SCHEMA - format_workload_sql - write_workload_sqlite - format_workload_payload - logger - run_bench_serve filters: [] show_if_no_docstring: true ## Complete contract reference Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.
vllm_mlx.bench_serve.WorkloadCase · class ```python vllm_mlx.bench_serve.WorkloadCase(case_id: str, messages: list[dict], request_path: Optional[str] = None, max_tokens: Optional[int] = None, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, policy_timeout_ms: Optional[int] = None, checks: Optional[dict] = None, tags: tuple[str, ...] = ()) ``` One declarative benchmark case for contract-style serving tests. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `case_id` | `str` | `yes` | `none` | Required constructor field. | | `messages` | `list[dict]` | `yes` | `none` | Required constructor field. | | `request_path` | `Optional[str]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `max_tokens` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `enable_thinking` | `Optional[bool]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `extra_body` | `Optional[dict]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `policy_timeout_ms` | `Optional[int]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `checks` | `Optional[dict]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `tags` | `tuple[str, ...]` | `no` | `()` | Optional constructor field; defaults to `()`. | **Returns** - Constructs: `vllm_mlx.bench_serve.WorkloadCase` **Exceptions and behavior** Class `WorkloadCase` declares 0 direct member(s). No direct `raise` statement appears in this definition. [View source #L51-L62](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L51-L62).
vllm_mlx.bench_serve.Workload · class ```python vllm_mlx.bench_serve.Workload(name: str, description: str, defaults: dict, cases: list[WorkloadCase]) ``` Normalized bench-serve workload manifest. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `name` | `str` | `yes` | `none` | Required constructor field. | | `description` | `str` | `yes` | `none` | Required constructor field. | | `defaults` | `dict` | `yes` | `none` | Required constructor field. | | `cases` | `list[WorkloadCase]` | `yes` | `none` | Required constructor field. | **Returns** - Constructs: `vllm_mlx.bench_serve.Workload` **Exceptions and behavior** Class `Workload` declares 0 direct member(s). No direct `raise` statement appears in this definition. [View source #L66-L72](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L66-L72).
vllm_mlx.bench_serve.load_prompt_set · function ```python vllm_mlx.bench_serve.load_prompt_set(name_or_path: str) -> list[list[dict]] ``` Load a prompt set by builtin name or file path. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `name_or_path` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[list[dict]]` - Direct return expressions: `[[msg] for msg in raw]`; `raw` **Exceptions and behavior** Function `load_prompt_set` calls `target.exists`, `FileNotFoundError`, `target.open`, `json.load`; can raise `FileNotFoundError`, `ValueError`; has 2 explicit return paths. Directly raised exceptions: `FileNotFoundError`, `ValueError`. [View source #L75-L137](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L75-L137).
vllm_mlx.bench_serve._require_message_list · function ```python vllm_mlx.bench_serve._require_message_list(value: Any, *, label: str) -> list[dict] ``` Function `_require_message_list` calls `isinstance`, `ValueError`, `enumerate`; can raise `ValueError`; returns `value`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `value` | `Any` | `yes` | `none` | Required positional or keyword input. | | `label` | `str` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `list[dict]` - Direct return expressions: `value` **Exceptions and behavior** Function `_require_message_list` calls `isinstance`, `ValueError`, `enumerate`; can raise `ValueError`; returns `value`. Directly raised exceptions: `ValueError`. [View source #L140-L148](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L140-L148).
vllm_mlx.bench_serve._load_case_request · function ```python vllm_mlx.bench_serve._load_case_request(path: str, *, workload_path: Path, case_id: str) -> dict ``` Function `_load_case_request` calls `Path(path).expanduser`, `Path`, `request_path.is_absolute`, `request_path.open`; can raise `ValueError`; returns `request`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `path` | `str` | `yes` | `none` | Required positional or keyword input. | | `workload_path` | `Path` | `yes` | `none` | Required keyword-only input. | | `case_id` | `str` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `dict` - Direct return expressions: `request` **Exceptions and behavior** Function `_load_case_request` calls `Path(path).expanduser`, `Path`, `request_path.is_absolute`, `request_path.open`; can raise `ValueError`; returns `request`. Directly raised exceptions: `ValueError`. [View source #L151-L159](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L151-L159).
vllm_mlx.bench_serve._request_extra_body · function ```python vllm_mlx.bench_serve._request_extra_body(request: dict) -> dict ``` Function `_request_extra_body` calls `request.items`; returns `{key: value for key, value in request.items() if key not in reserved}`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `request` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{key: value for key, value in request.items() if key not in reserved}` **Exceptions and behavior** Function `_request_extra_body` calls `request.items`; returns `{key: value for key, value in request.items() if key not in reserved}`. No direct `raise` statement appears in this definition. [View source #L162-L171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L162-L171).
vllm_mlx.bench_serve._first_not_none · function ```python vllm_mlx.bench_serve._first_not_none(*values: Any) -> Any ``` Function `_first_not_none` has 2 explicit return paths. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `*values` | `Any` | `no` | `none` | Additional variadic positional inputs accepted by this callable. | **Returns** - Type: `Any` - Direct return expressions: `value`; `None` **Exceptions and behavior** Function `_first_not_none` has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L174-L178](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L174-L178).
vllm_mlx.bench_serve._normalize_tags · function ```python vllm_mlx.bench_serve._normalize_tags(tags: Any, *, case_id: str) -> tuple[str, ...] ``` Coerce a workload case's ``tags`` field to a tuple of strings. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `tags` | `Any` | `yes` | `none` | Required positional or keyword input. | | `case_id` | `str` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `tuple[str, ...]` - Direct return expressions: `tuple((str(tag) for tag in tags))` **Exceptions and behavior** Function `_normalize_tags` calls `isinstance`, `ValueError`, `tuple`, `str`; can raise `ValueError`; returns `tuple((str(tag) for tag in tags))`. Directly raised exceptions: `ValueError`. [View source #L181-L191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L181-L191).
vllm_mlx.bench_serve._merge_case_checks · function ```python vllm_mlx.bench_serve._merge_case_checks(default_checks: Any, case_checks: Any, *, case_id: str) -> Optional[dict] ``` Merge a case's ``checks`` over the workload defaults. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `default_checks` | `Any` | `yes` | `none` | Required positional or keyword input. | | `case_checks` | `Any` | `yes` | `none` | Required positional or keyword input. | | `case_id` | `str` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `Optional[dict]` - Direct return expressions: `merged or None` **Exceptions and behavior** Function `_merge_case_checks` calls `dict`, `isinstance`, `ValueError`, `case_checks.items`; can raise `ValueError`; returns `merged or None`. Directly raised exceptions: `ValueError`. [View source #L194-L227](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L194-L227).
vllm_mlx.bench_serve._build_workload_case · function ```python vllm_mlx.bench_serve._build_workload_case(item: Any, idx: int, *, defaults: dict, workload_path: Path) -> WorkloadCase ``` Construct one ``WorkloadCase`` from a raw workload entry. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `item` | `Any` | `yes` | `none` | Required positional or keyword input. | | `idx` | `int` | `yes` | `none` | Required positional or keyword input. | | `defaults` | `dict` | `yes` | `none` | Required keyword-only input. | | `workload_path` | `Path` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `WorkloadCase` - Direct return expressions: `WorkloadCase(case_id=case_id, messages=messages, request_path=str(request_path) if request_path is not None else None, …` **Exceptions and behavior** Function `_build_workload_case` calls `isinstance`, `ValueError`, `str`, `item.get`; can raise `ValueError`; returns `WorkloadCase(case_id=case_id, messages=messages, request_path=str(request_path) if request_path is not None else None, …`. Directly raised exceptions: `ValueError`. [View source #L230-L300](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L230-L300).
vllm_mlx.bench_serve.load_workload · function ```python vllm_mlx.bench_serve.load_workload(path: str | Path) -> Workload ``` Load a declarative serving benchmark workload. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `path` | `str \| Path` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `Workload` - Direct return expressions: `Workload(name=str(raw.get('name') or workload_path.stem), description=str(raw.get('description') or ''), defaults=defau…` **Exceptions and behavior** Function `load_workload` calls `Path(path).expanduser`, `Path`, `workload_path.open`, `json.load`; can raise `ValueError`; returns `Workload(name=str(raw.get('name') or workload_path.stem), description=str(raw.get('description') or ''), defaults=defau…`. Directly raised exceptions: `ValueError`. [View source #L303-L335](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L303-L335).
vllm_mlx.bench_serve.BenchServeResult · class ```python vllm_mlx.bench_serve.BenchServeResult(run_id: str = '', timestamp: str = '', tag: str = '', chip: str = '', gpu_cores: int = 0, memory_gb: float = 0.0, bandwidth_gbs: float = 0.0, os_version: str = '', model_id: str = '', model_type: str = '', engine_type: str = '', mtp_enabled: bool = False, specprefill: bool = False, kv_quant: str = '', cache_type: str = '', prompt_set: str = '', concurrency: int = 1, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: str = '', repetition: int = 0, prompt_tokens: int = 0, ttft_ms: float = 0.0, tpot_ms: float = 0.0, e2e_latency_ms: float = 0.0, gen_tps: float = 0.0, prompt_tps: float = 0.0, throughput_tps: float = 0.0, requests_per_s: float = 0.0, metal_active_gb: float = 0.0, metal_peak_gb: float = 0.0, metal_cache_gb: float = 0.0, cache_hits: int = 0, cache_misses: int = 0, cache_hit_rate: float = 0.0, tokens_saved: int = 0, validated: bool = True) ``` Aggregated results from a single bench-serve run configuration. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `run_id` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `timestamp` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `tag` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `chip` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `gpu_cores` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `memory_gb` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `bandwidth_gbs` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `os_version` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `model_id` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `model_type` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `engine_type` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `mtp_enabled` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. | | `specprefill` | `bool` | `no` | `False` | Optional constructor field; defaults to `False`. | | `kv_quant` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `cache_type` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `prompt_set` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `concurrency` | `int` | `no` | `1` | Optional constructor field; defaults to `1`. | | `max_tokens` | `int` | `no` | `256` | Optional constructor field; defaults to `256`. | | `enable_thinking` | `Optional[bool]` | `no` | `None` | Optional constructor field; defaults to `None`. | | `extra_body` | `str` | `no` | `''` | Optional constructor field; defaults to `''`. | | `repetition` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `prompt_tokens` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `ttft_ms` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `tpot_ms` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `e2e_latency_ms` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `gen_tps` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `prompt_tps` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `throughput_tps` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `requests_per_s` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `metal_active_gb` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `metal_peak_gb` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `metal_cache_gb` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `cache_hits` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `cache_misses` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `cache_hit_rate` | `float` | `no` | `0.0` | Optional constructor field; defaults to `0.0`. | | `tokens_saved` | `int` | `no` | `0` | Optional constructor field; defaults to `0`. | | `validated` | `bool` | `no` | `True` | Optional constructor field; defaults to `True`. | **Returns** - Constructs: `vllm_mlx.bench_serve.BenchServeResult` **Exceptions and behavior** Class `BenchServeResult` declares 0 direct member(s). No direct `raise` statement appears in this definition. [View source #L344-L400](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L344-L400).
vllm_mlx.bench_serve.expand_sweep · function ```python vllm_mlx.bench_serve.expand_sweep(prompt_sets: list[str], concurrencies: list[int], thinking_values: list[Optional[bool]], extra_bodies: list[str], repetitions: int) -> list[SweepConfig] ``` Expand sweep parameters into a flat list of configurations. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `prompt_sets` | `list[str]` | `yes` | `none` | Names or paths of prompt sets to include. | | `concurrencies` | `list[int]` | `yes` | `none` | Concurrency levels to test (e.g. ``[1, 4, 16]``). | | `thinking_values` | `list[Optional[bool]]` | `yes` | `none` | Values for ``enable_thinking`` (e.g. ``[None, True, False]``). | | `extra_bodies` | `list[str]` | `yes` | `none` | JSON strings (or empty string) to pass as extra body parameters on each request. | | `repetitions` | `int` | `yes` | `none` | Number of times to repeat each unique combination. Each repeat gets a distinct 0-based repetition index. | **Returns** - Type: `list[SweepConfig]` - Direct return expressions: `configs` **Exceptions and behavior** Function `expand_sweep` calls `itertools.product`, `range`, `configs.append`; returns `configs`. No direct `raise` statement appears in this definition. [View source #L411-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L411-L444).
vllm_mlx.bench_serve.parse_health_response · function ```python vllm_mlx.bench_serve.parse_health_response(data: dict) -> dict ``` Extract model identity fields from a GET /health response. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `data` | `dict` | `yes` | `none` | Parsed JSON body from the /health endpoint. Expected shape:: {"status": "healthy", "model_loaded": True, "model_name": "...", "model_type": "llm"\|"mllm"} | **Returns** - Type: `dict` - Direct return expressions: `{'model_name': data.get('model_name', ''), 'model_type': data.get('model_type', '')}` **Exceptions and behavior** Function `parse_health_response` calls `data.get`; returns `{'model_name': data.get('model_name', ''), 'model_type': data.get('model_type', '')}`. No direct `raise` statement appears in this definition. [View source #L452-L467](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L452-L467).
vllm_mlx.bench_serve.parse_status_response · function ```python vllm_mlx.bench_serve.parse_status_response(data: dict) -> dict ``` Extract metal and cache info from a GET /v1/status response. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `data` | `dict` | `yes` | `none` | Parsed JSON body from the /v1/status endpoint. Metal info is expected under ``data["metal"]`` and cache info under ``data["cache"]``. Missing keys are handled gracefully. | **Returns** - Type: `dict` - Direct return expressions: `{'model': data.get('model', ''), 'metal_active_gb': float(metal.get('active_memory_gb') or metal.get('active_gb') or 0.…` **Exceptions and behavior** Function `parse_status_response` calls `data.get`, `float`, `metal.get`, `cache.get`; returns `{'model': data.get('model', ''), 'metal_active_gb': float(metal.get('active_memory_gb') or metal.get('active_gb') or 0.…`. No direct `raise` statement appears in this definition. [View source #L470-L496](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L470-L496).
vllm_mlx.bench_serve.parse_metrics_text · function ```python vllm_mlx.bench_serve.parse_metrics_text(text: str) -> dict ``` Parse Prometheus text exposition format from GET /metrics. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `text` | `str` | `yes` | `none` | Raw response body from the /metrics endpoint. | **Returns** - Type: `dict` - Direct return expressions: `{'cache_hits': _extract('vllm_prefix_cache_hits_total'), 'cache_misses': _extract('vllm_prefix_cache_misses_total'), 't…` **Exceptions and behavior** Function `parse_metrics_text` calls `_extract`; returns `{'cache_hits': _extract('vllm_prefix_cache_hits_total'), 'cache_misses': _extract('vllm_prefix_cache_misses_total'), 't…`. No direct `raise` statement appears in this definition. [View source #L499-L521](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L499-L521).
vllm_mlx.bench_serve.parse_metrics_text._extract · nested function ```python vllm_mlx.bench_serve.parse_metrics_text._extract(metric_name: str) -> int ``` Nested Function `parse_metrics_text._extract` calls `re.escape`, `re.search`, `int`, `m.group`; returns `int(m.group(1)) if m else 0`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `metric_name` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `int` - Direct return expressions: `int(m.group(1)) if m else 0` **Exceptions and behavior** Nested Function `parse_metrics_text._extract` calls `re.escape`, `re.search`, `int`, `m.group`; returns `int(m.group(1)) if m else 0`. No direct `raise` statement appears in this definition. [View source #L512-L515](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L512-L515).
vllm_mlx.bench_serve.detect_hardware_fingerprint · function ```python vllm_mlx.bench_serve.detect_hardware_fingerprint() -> dict ``` Return a hardware fingerprint dict for the current machine. **Parameters** This callable has no explicit inputs. **Returns** - Type: `dict` - Direct return expressions: `{'chip': hw.chip_name, 'gpu_cores': hw.gpu_cores, 'memory_gb': hw.total_memory_gb, 'bandwidth_gbs': hw.memory_bandwidth…`; `{'chip': '', 'gpu_cores': 0, 'memory_gb': memory_gb, 'bandwidth_gbs': 0.0, 'os_version': os_version}` **Exceptions and behavior** Function `detect_hardware_fingerprint` calls `platform.platform`, `detect_hardware`, `subprocess.run`, `int`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L524-L573](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L524-L573).
vllm_mlx.bench_serve.auto_detect_runtime · function ```python async vllm_mlx.bench_serve.auto_detect_runtime(client: httpx.AsyncClient, base_url: str) -> dict ``` Query the running server and return a runtime descriptor dict. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | An open :class:`httpx.AsyncClient`. | | `base_url` | `str` | `yes` | `none` | Base URL of the server (e.g. ``"http://localhost:8080"``). | **Returns** - Type: `dict` - Direct return expressions: `result` **Exceptions and behavior** Function `auto_detect_runtime` calls `client.get`, `resp.raise_for_status`, `parse_health_response`, `resp.json`; awaits asynchronous work; returns `result`. No direct `raise` statement appears in this definition. [View source #L576-L642](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L576-L642).
vllm_mlx.bench_serve.scrape_metrics · function ```python async vllm_mlx.bench_serve.scrape_metrics(client: httpx.AsyncClient, base_url: str) -> dict ``` Scrape Prometheus metrics from the server. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | An open :class:`httpx.AsyncClient`. | | `base_url` | `str` | `yes` | `none` | Base URL of the server. | **Returns** - Type: `dict` - Direct return expressions: `parse_metrics_text(resp.text)`; `{}` **Exceptions and behavior** Function `scrape_metrics` calls `client.get`, `resp.raise_for_status`, `parse_metrics_text`; awaits asynchronous work; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L645-L661](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L645-L661).
vllm_mlx.bench_serve.clear_runtime_cache · function ```python async vllm_mlx.bench_serve.clear_runtime_cache(client: httpx.AsyncClient, base_url: str) -> dict ``` Clear server-side runtime caches and return a JSON-serializable event. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | Required positional or keyword input. | | `base_url` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `event` **Exceptions and behavior** Function `clear_runtime_cache` calls `client.delete`, `resp.json`, `resp.raise_for_status`, `str`; awaits asynchronous work; returns `event`. No direct `raise` statement appears in this definition. [View source #L664-L684](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L664-L684).
vllm_mlx.bench_serve._normalize_cache_policy · function ```python vllm_mlx.bench_serve._normalize_cache_policy(value: Optional[str]) -> str ``` Normalize cache-policy spelling from CLI or workload JSON. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `value` | `Optional[str]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `policy` **Exceptions and behavior** Function `_normalize_cache_policy` calls `(value or 'preserve').strip().lower().replace`, `(value or 'preserve').strip().lower`, `(value or 'preserve').strip`, `ValueError`; can raise `ValueError`; returns `policy`. Directly raised exceptions: `ValueError`. [View source #L687-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L687-L698).
vllm_mlx.bench_serve.parse_sse_line · function ```python vllm_mlx.bench_serve.parse_sse_line(line: str) -> Optional[dict] ``` Parse one Server-Sent Events line from a streaming chat completion. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `line` | `str` | `yes` | `none` | A single raw line from the SSE stream (may or may not include a trailing newline — it is stripped before processing). | **Returns** - Type: `Optional[dict]` - Direct return expressions: `None`; `{'id': chunk.get('id'), 'content': content, 'finish_reason': finish_reason, 'usage': usage, 'tool_calls_delta': tool_ca…` **Exceptions and behavior** Function `parse_sse_line` calls `line.strip`, `line.startswith`, `len`, `json.loads`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L706-L754](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L706-L754).
vllm_mlx.bench_serve._cancel_server_request · function ```python async vllm_mlx.bench_serve._cancel_server_request(client: httpx.AsyncClient, base_url: str, request_id: Optional[str]) -> None ``` Best-effort server-side cancellation for timed-out workload streams. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | Required positional or keyword input. | | `base_url` | `str` | `yes` | `none` | Required positional or keyword input. | | `request_id` | `Optional[str]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `None` - Direct return expressions: `None` **Exceptions and behavior** Function `_cancel_server_request` calls `client.post`; awaits asynchronous work; returns `None`. No direct `raise` statement appears in this definition. [View source #L757-L770](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L757-L770).
vllm_mlx.bench_serve.accumulate_tool_calls · function ```python vllm_mlx.bench_serve.accumulate_tool_calls(acc: dict[int, dict], delta_list: list[dict]) -> None ``` Merge streamed OpenAI tool-call deltas into *acc* by index. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `acc` | `dict[int, dict]` | `yes` | `none` | Required positional or keyword input. | | `delta_list` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `None` **Exceptions and behavior** Function `accumulate_tool_calls` calls `int`, `tc_delta.get`, `function_delta.get`. No direct `raise` statement appears in this definition. [View source #L773-L792](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L773-L792).
vllm_mlx.bench_serve.finalize_tool_calls · function ```python vllm_mlx.bench_serve.finalize_tool_calls(acc: dict[int, dict]) -> list[dict] ``` Return accumulated tool calls in stream index order. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `acc` | `dict[int, dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[dict]` - Direct return expressions: `[acc[idx] for idx in sorted(acc)]` **Exceptions and behavior** Function `finalize_tool_calls` calls `sorted`; returns `[acc[idx] for idx in sorted(acc)]`. No direct `raise` statement appears in this definition. [View source #L795-L797](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L795-L797).
vllm_mlx.bench_serve.compute_request_metrics · function ```python vllm_mlx.bench_serve.compute_request_metrics(t_start: float, t_first_token: float, token_times: list, t_end: float, prompt_tokens: int, completion_tokens: int) -> dict ``` Compute standard latency and throughput metrics for a single request. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `t_start` | `float` | `yes` | `none` | Timestamp immediately before the request was sent. | | `t_first_token` | `float` | `yes` | `none` | Timestamp when the first content token was received. | | `token_times` | `list` | `yes` | `none` | List of timestamps, one per content token (including the first). When there is only one token ``tpot_ms`` is ``0.0``. | | `t_end` | `float` | `yes` | `none` | Timestamp after the final SSE chunk was consumed. | | `prompt_tokens` | `int` | `yes` | `none` | Number of prompt tokens reported by the server. | | `completion_tokens` | `int` | `yes` | `none` | Number of completion tokens generated. | **Returns** - Type: `dict` - Direct return expressions: `{'ttft_ms': ttft_ms, 'tpot_ms': tpot_ms, 'e2e_latency_ms': e2e_latency_ms, 'gen_tps': gen_tps, 'prompt_tps': prompt_tps}` **Exceptions and behavior** Function `compute_request_metrics` calls `len`, `range`, `statistics.mean`; returns `{'ttft_ms': ttft_ms, 'tpot_ms': tpot_ms, 'e2e_latency_ms': e2e_latency_ms, 'gen_tps': gen_tps, 'prompt_tps': prompt_tps}`. No direct `raise` statement appears in this definition. [View source #L800-L852](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L800-L852).
vllm_mlx.bench_serve.count_prompt_tokens · function ```python async vllm_mlx.bench_serve.count_prompt_tokens(client: httpx.AsyncClient, base_url: str, messages: list[dict], model: str) -> int ``` Count prompt tokens for a message list by sending a 1-token request. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | An open :class:`httpx.AsyncClient`. | | `base_url` | `str` | `yes` | `none` | Base URL of the server. | | `messages` | `list[dict]` | `yes` | `none` | The message list to send. | | `model` | `str` | `yes` | `none` | Model ID to target. | **Returns** - Type: `int` - Direct return expressions: `int((data.get('usage') or {}).get('prompt_tokens', 0))`; `0` **Exceptions and behavior** Function `count_prompt_tokens` calls `client.post`, `resp.raise_for_status`, `resp.json`, `int`; awaits asynchronous work; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L855-L889](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L855-L889).
vllm_mlx.bench_serve.stream_chat_completion · function ```python async vllm_mlx.bench_serve.stream_chat_completion(client: httpx.AsyncClient, base_url: str, messages: list[dict], model: str, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, timeout_s: Optional[float] = None) -> dict ``` Send a streaming chat completion and collect per-token timing data. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | An open :class:`httpx.AsyncClient`. | | `base_url` | `str` | `yes` | `none` | Base URL of the server. | | `messages` | `list[dict]` | `yes` | `none` | The message list to send. | | `model` | `str` | `yes` | `none` | Model ID to target. | | `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate (default ``256``). | | `enable_thinking` | `Optional[bool]` | `no` | `None` | If not ``None``, passed as ``enable_thinking`` in the request body. | | `extra_body` | `Optional[dict]` | `no` | `None` | Optional extra keys merged into the request body. | | `timeout_s` | `Optional[float]` | `no` | `None` | Optional case-level timeout. When set, the stream is closed and best-effort server cancellation is attempted before raising :class:`TimeoutError`. | **Returns** - Type: `dict` - Direct return expressions: `{**metrics, 'completion_tokens': completion_tokens, 'prompt_tokens': prompt_tokens, 'finish_reason': finish_reason, 'co…` **Exceptions and behavior** Function `stream_chat_completion` calls `body.update`, `time.perf_counter`, `asyncio.timeout`, `_consume_stream`; awaits asynchronous work; can raise `TimeoutError`; returns `{**metrics, 'completion_tokens': completion_tokens, 'prompt_tokens': prompt_tokens, 'finish_reason': finish_reason, 'co…`. Directly raised exceptions: `TimeoutError`. [View source #L892-L1012](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L892-L1012).
vllm_mlx.bench_serve.stream_chat_completion._consume_stream · nested function ```python async vllm_mlx.bench_serve.stream_chat_completion._consume_stream() -> None ``` Nested Function `stream_chat_completion._consume_stream` calls `client.stream`, `response.raise_for_status`, `response.aiter_lines`, `parse_sse_line`. **Parameters** This callable has no explicit inputs. **Returns** - Type: `None` **Exceptions and behavior** Nested Function `stream_chat_completion._consume_stream` calls `client.stream`, `response.raise_for_status`, `response.aiter_lines`, `parse_sse_line`. No direct `raise` statement appears in this definition. [View source #L946-L975](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L946-L975).
vllm_mlx.bench_serve.validate_response · function ```python vllm_mlx.bench_serve.validate_response(finish_reason: Optional[str], content: str, status_code: int, *, tool_calls: Optional[list[dict]] = None) -> tuple[bool, str] ``` Validate a single streaming response result. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `finish_reason` | `Optional[str]` | `yes` | `none` | The ``finish_reason`` from the final SSE chunk, or ``None`` if not received. | | `content` | `str` | `yes` | `none` | The accumulated text content of the response. | | `status_code` | `int` | `yes` | `none` | The HTTP status code of the response (use ``200`` for successful streaming requests). | | `tool_calls` | `Optional[list[dict]]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | **Returns** - Type: `tuple[bool, str]` - Direct return expressions: `(False, f'HTTP error {status_code}')`; `(False, 'Missing finish_reason')`; `(False, 'Truncated (finish_reason=length)')`; `(False, 'Empty response content')`; `(True, '')` **Exceptions and behavior** Function `validate_response` has 5 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1020-L1049](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1020-L1049).
vllm_mlx.bench_serve._check_finish_reason · function ```python vllm_mlx.bench_serve._check_finish_reason(allowed: Any, finish_reason: Optional[str]) -> list[str] ``` Verify ``finish_reason`` is in the allowed set, if one is configured. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `allowed` | `Any` | `yes` | `none` | Required positional or keyword input. | | `finish_reason` | `Optional[str]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[str]` - Direct return expressions: `[]`; `[f'finish_reason {finish_reason!r} not in allowed set {allowed_list!r}']` **Exceptions and behavior** Function `_check_finish_reason` calls `isinstance`, `list`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1052-L1059](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1052-L1059).
vllm_mlx.bench_serve._check_length_bounds · function ```python vllm_mlx.bench_serve._check_length_bounds(min_chars: Any, max_chars: Any, content: str) -> list[str] ``` Apply ``min_chars`` / ``max_chars`` content-length bounds. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `min_chars` | `Any` | `yes` | `none` | Required positional or keyword input. | | `max_chars` | `Any` | `yes` | `none` | Required positional or keyword input. | | `content` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[str]` - Direct return expressions: `issues` **Exceptions and behavior** Function `_check_length_bounds` calls `len`, `int`, `issues.append`; returns `issues`. No direct `raise` statement appears in this definition. [View source #L1062-L1069](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1062-L1069).
vllm_mlx.bench_serve._check_regex_patterns · function ```python vllm_mlx.bench_serve._check_regex_patterns(patterns: Any, content: str, *, kind: str, expect_match: bool) -> list[str] ``` Validate that each pattern either matches or does not, per ``expect_match``. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `patterns` | `Any` | `yes` | `none` | Required positional or keyword input. | | `content` | `str` | `yes` | `none` | Required positional or keyword input. | | `kind` | `str` | `yes` | `none` | Required keyword-only input. | | `expect_match` | `bool` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `list[str]` - Direct return expressions: `issues` **Exceptions and behavior** Function `_check_regex_patterns` calls `bool`, `re.search`, `str`, `issues.append`; returns `issues`. No direct `raise` statement appears in this definition. [View source #L1072-L1096](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1072-L1096).
vllm_mlx.bench_serve._check_json_content · function ```python vllm_mlx.bench_serve._check_json_content(should_be_json: Any, content: str) -> list[str] ``` Verify ``content`` parses as JSON when ``checks['json']`` is truthy. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `should_be_json` | `Any` | `yes` | `none` | Required positional or keyword input. | | `content` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[str]` - Direct return expressions: `[]`; `[f'content is not valid JSON: {exc}']` **Exceptions and behavior** Function `_check_json_content` calls `json.loads`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1099-L1107](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1099-L1107).
vllm_mlx.bench_serve._check_tool_call_count_and_names · function ```python vllm_mlx.bench_serve._check_tool_call_count_and_names(checks: dict, tool_calls: list[dict]) -> list[str] ``` Apply ``no_tool_calls`` / ``tool_call_count`` / ``tool_call_names``. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `checks` | `dict` | `yes` | `none` | Required positional or keyword input. | | `tool_calls` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[str]` - Direct return expressions: `issues` **Exceptions and behavior** Function `_check_tool_call_count_and_names` calls `checks.get`, `issues.append`, `len`, `int`; returns `issues`. No direct `raise` statement appears in this definition. [View source #L1110-L1132](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1110-L1132).
vllm_mlx.bench_serve._check_tool_call_args · function ```python vllm_mlx.bench_serve._check_tool_call_args(required_args: Any, tool_calls: list[dict]) -> list[str] ``` Validate parsed JSON arguments include the required keys per function. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `required_args` | `Any` | `yes` | `none` | Required positional or keyword input. | | `tool_calls` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `list[str]` - Direct return expressions: `[]`; `issues` **Exceptions and behavior** Function `_check_tool_call_args` calls `tc.get('function', {}).get`, `tc.get`, `by_name.setdefault(name, []).append`, `by_name.setdefault`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1135-L1174](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1135-L1174).
vllm_mlx.bench_serve.validate_quality_checks · function ```python vllm_mlx.bench_serve.validate_quality_checks(finish_reason: Optional[str], content: str, checks: Optional[dict], *, status_code: int = 200, tool_calls: Optional[list[dict]] = None) -> tuple[bool, list[str]] ``` Validate content against generic workload quality checks. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `finish_reason` | `Optional[str]` | `yes` | `none` | Required positional or keyword input. | | `content` | `str` | `yes` | `none` | Required positional or keyword input. | | `checks` | `Optional[dict]` | `yes` | `none` | Required positional or keyword input. | | `status_code` | `int` | `no` | `200` | Optional keyword-only input; defaults to `200`. | | `tool_calls` | `Optional[list[dict]]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | **Returns** - Type: `tuple[bool, list[str]]` - Direct return expressions: `(not issues, issues)` **Exceptions and behavior** Function `validate_quality_checks` calls `validate_response`, `issues.extend`, `_check_finish_reason`, `checks.get`; returns `(not issues, issues)`. No direct `raise` statement appears in this definition. [View source #L1177-L1231](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1177-L1231).
vllm_mlx.bench_serve.compute_summary_stats · function ```python vllm_mlx.bench_serve.compute_summary_stats(values: list[float]) -> dict ``` Compute summary statistics over a list of floats. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `values` | `list[float]` | `yes` | `none` | Non-empty list of floats to summarise. | **Returns** - Type: `dict` - Direct return expressions: `{'mean': mean, 'stddev': stddev, 'min': sorted_vals[0], 'max': sorted_vals[-1], 'p50': _percentile(50), 'p95': _percent…` **Exceptions and behavior** Function `compute_summary_stats` calls `ValueError`, `len`, `statistics.mean`, `statistics.stdev`; can raise `ValueError`; returns `{'mean': mean, 'stddev': stddev, 'min': sorted_vals[0], 'max': sorted_vals[-1], 'p50': _percentile(50), 'p95': _percent…`. Directly raised exceptions: `ValueError`. [View source #L1234-L1276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1234-L1276).
vllm_mlx.bench_serve.compute_summary_stats._percentile · nested function ```python vllm_mlx.bench_serve.compute_summary_stats._percentile(p: float) -> float ``` Nested Function `compute_summary_stats._percentile` calls `int`; has 3 explicit return paths. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `p` | `float` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `float` - Direct return expressions: `sorted_vals[0]`; `sorted_vals[-1]`; `sorted_vals[lo] + frac * (sorted_vals[hi] - sorted_vals[lo])` **Exceptions and behavior** Nested Function `compute_summary_stats._percentile` calls `int`; has 3 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1256-L1266](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1256-L1266).
vllm_mlx.bench_serve.run_concurrent_requests · function ```python async vllm_mlx.bench_serve.run_concurrent_requests(client: httpx.AsyncClient, base_url: str, prompts: list[list[dict]], model: str, concurrency: int, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, do_validate: bool = True) -> list[dict] ``` Fire ``concurrency`` concurrent streaming requests and collect results. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | An open :class:`httpx.AsyncClient`. | | `base_url` | `str` | `yes` | `none` | Base URL of the server. | | `prompts` | `list[list[dict]]` | `yes` | `none` | List of message dicts to cycle through. | | `model` | `str` | `yes` | `none` | Model ID to target. | | `concurrency` | `int` | `yes` | `none` | Number of simultaneous requests to fire. | | `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate per request (default ``256``). | | `enable_thinking` | `Optional[bool]` | `no` | `None` | Passed through to :func:`stream_chat_completion`. | | `extra_body` | `Optional[dict]` | `no` | `None` | Passed through to :func:`stream_chat_completion`. | | `do_validate` | `bool` | `no` | `True` | When ``True``, call :func:`validate_response` on each result and add a ``"validated"`` key. | **Returns** - Type: `list[dict]` - Direct return expressions: `list(results)` **Exceptions and behavior** Function `run_concurrent_requests` calls `itertools.cycle`, `next`, `range`, `asyncio.gather`; awaits asynchronous work; returns `list(results)`. No direct `raise` statement appears in this definition. [View source #L1279-L1342](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1279-L1342).
vllm_mlx.bench_serve.run_concurrent_requests._single · nested function ```python async vllm_mlx.bench_serve.run_concurrent_requests._single(messages: list[dict]) -> dict ``` Nested Function `run_concurrent_requests._single` calls `stream_chat_completion`, `validate_response`, `result.get`, `str`; awaits asynchronous work; has 2 explicit return paths. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `messages` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `result`; `err` **Exceptions and behavior** Nested Function `run_concurrent_requests._single` calls `stream_chat_completion`, `validate_response`, `result.get`, `str`; awaits asynchronous work; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1315-L1339](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1315-L1339).
vllm_mlx.bench_serve._summary_or_empty · function ```python vllm_mlx.bench_serve._summary_or_empty(values: list[float]) -> dict ``` Function `_summary_or_empty` calls `compute_summary_stats`; returns `compute_summary_stats(values) if values else {}`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `values` | `list[float]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `compute_summary_stats(values) if values else {}` **Exceptions and behavior** Function `_summary_or_empty` calls `compute_summary_stats`; returns `compute_summary_stats(values) if values else {}`. No direct `raise` statement appears in this definition. [View source #L1345-L1346](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1345-L1346).
vllm_mlx.bench_serve._resolve_max_tokens · function ```python vllm_mlx.bench_serve._resolve_max_tokens(case: WorkloadCase, workload: Workload) -> int ``` Return the effective ``max_tokens`` for a case, falling back to workload defaults and finally to 256. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `case` | `WorkloadCase` | `yes` | `none` | Required positional or keyword input. | | `workload` | `Workload` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `int` - Direct return expressions: `int(case.max_tokens or workload.defaults.get('max_tokens', 256))` **Exceptions and behavior** Function `_resolve_max_tokens` calls `int`, `workload.defaults.get`; returns `int(case.max_tokens or workload.defaults.get('max_tokens', 256))`. No direct `raise` statement appears in this definition. [View source #L1349-L1352](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1349-L1352).
vllm_mlx.bench_serve._assemble_case_request_kwargs · function ```python vllm_mlx.bench_serve._assemble_case_request_kwargs(case: WorkloadCase, workload: Workload, model: str) -> dict ``` Build the keyword-arguments dict passed to ``stream_chat_completion`` for one case, applying max_tokens fallback and converting ``policy_timeout_ms`` to seconds. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `case` | `WorkloadCase` | `yes` | `none` | Required positional or keyword input. | | `workload` | `Workload` | `yes` | `none` | Required positional or keyword input. | | `model` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{'messages': case.messages, 'model': model, 'max_tokens': _resolve_max_tokens(case, workload), 'enable_thinking': case.…` **Exceptions and behavior** Function `_assemble_case_request_kwargs` calls `_resolve_max_tokens`; returns `{'messages': case.messages, 'model': model, 'max_tokens': _resolve_max_tokens(case, workload), 'enable_thinking': case.…`. No direct `raise` statement appears in this definition. [View source #L1355-L1372](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1355-L1372).
vllm_mlx.bench_serve._empty_completion_result · function ```python vllm_mlx.bench_serve._empty_completion_result() -> dict ``` Zero-valued completion result used when ``stream_chat_completion`` raises. **Parameters** This callable has no explicit inputs. **Returns** - Type: `dict` - Direct return expressions: `{'ttft_ms': 0.0, 'tpot_ms': 0.0, 'e2e_latency_ms': 0.0, 'gen_tps': 0.0, 'prompt_tps': 0.0, 'prompt_tokens': 0, 'complet…` **Exceptions and behavior** Function `_empty_completion_result` returns `{'ttft_ms': 0.0, 'tpot_ms': 0.0, 'e2e_latency_ms': 0.0, 'gen_tps': 0.0, 'prompt_tps': 0.0, 'prompt_tokens': 0, 'complet…`. No direct `raise` statement appears in this definition. [View source #L1375-L1390](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1375-L1390).
vllm_mlx.bench_serve._fetch_post_run_status · function ```python async vllm_mlx.bench_serve._fetch_post_run_status(client: httpx.AsyncClient, base_url: str) -> dict ``` GET ``/v1/status`` after a case run, swallowing transport errors so a missing or temporarily-unavailable status endpoint does not fail the case record. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | Required positional or keyword input. | | `base_url` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `resp.json()`; `{}` **Exceptions and behavior** Function `_fetch_post_run_status` calls `client.get`, `resp.raise_for_status`, `resp.json`; awaits asynchronous work; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1393-L1402](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1393-L1402).
vllm_mlx.bench_serve._compute_within_policy_timeout · function ```python vllm_mlx.bench_serve._compute_within_policy_timeout(timeout_ms: Optional[int], *, error_present: bool, e2e_latency_ms: float) -> Optional[bool] ``` Resolve the ``policy.within_timeout`` field. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `timeout_ms` | `Optional[int]` | `yes` | `none` | Required positional or keyword input. | | `error_present` | `bool` | `yes` | `none` | Required keyword-only input. | | `e2e_latency_ms` | `float` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `Optional[bool]` - Direct return expressions: `None`; `False`; `e2e_latency_ms <= timeout_ms` **Exceptions and behavior** Function `_compute_within_policy_timeout` has 3 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1405-L1418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1405-L1418).
vllm_mlx.bench_serve._build_tool_calls_summary · function ```python vllm_mlx.bench_serve._build_tool_calls_summary(tool_calls: Any) -> Optional[dict] ``` Compact summary of streamed tool calls for the case record. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `tool_calls` | `Any` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `Optional[dict]` - Direct return expressions: `None`; `{'count': len(tool_calls), 'names': sorted((tc.get('function', {}).get('name', '') for tc in tool_calls)), 'raw': tool_…` **Exceptions and behavior** Function `_build_tool_calls_summary` calls `len`, `sorted`, `tc.get('function', {}).get`, `tc.get`; has 2 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1421-L1434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1421-L1434).
vllm_mlx.bench_serve._build_workload_record · function ```python vllm_mlx.bench_serve._build_workload_record(*, case: WorkloadCase, workload: Workload, model: str, runtime: dict, hardware: dict, run_id: str, timestamp: str, started_wall: str, repetition: int, result: dict, error: str, quality_ok: bool, quality_issues: list[str], content: str, cache_hits_delta: int, cache_misses_delta: int, tokens_saved_delta: int, status_after: dict, cache_reset: Optional[dict], include_content: bool) -> dict ``` Assemble the JSON-serializable workload-case record from the raw inputs and the completion result. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `case` | `WorkloadCase` | `yes` | `none` | Required keyword-only input. | | `workload` | `Workload` | `yes` | `none` | Required keyword-only input. | | `model` | `str` | `yes` | `none` | Required keyword-only input. | | `runtime` | `dict` | `yes` | `none` | Required keyword-only input. | | `hardware` | `dict` | `yes` | `none` | Required keyword-only input. | | `run_id` | `str` | `yes` | `none` | Required keyword-only input. | | `timestamp` | `str` | `yes` | `none` | Required keyword-only input. | | `started_wall` | `str` | `yes` | `none` | Required keyword-only input. | | `repetition` | `int` | `yes` | `none` | Required keyword-only input. | | `result` | `dict` | `yes` | `none` | Required keyword-only input. | | `error` | `str` | `yes` | `none` | Required keyword-only input. | | `quality_ok` | `bool` | `yes` | `none` | Required keyword-only input. | | `quality_issues` | `list[str]` | `yes` | `none` | Required keyword-only input. | | `content` | `str` | `yes` | `none` | Required keyword-only input. | | `cache_hits_delta` | `int` | `yes` | `none` | Required keyword-only input. | | `cache_misses_delta` | `int` | `yes` | `none` | Required keyword-only input. | | `tokens_saved_delta` | `int` | `yes` | `none` | Required keyword-only input. | | `status_after` | `dict` | `yes` | `none` | Required keyword-only input. | | `cache_reset` | `Optional[dict]` | `yes` | `none` | Required keyword-only input. | | `include_content` | `bool` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `dict` - Direct return expressions: `record` **Exceptions and behavior** Function `_build_workload_record` calls `list`, `_resolve_max_tokens`, `len`, `_compute_within_policy_timeout`; returns `record`. No direct `raise` statement appears in this definition. [View source #L1437-L1515](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1437-L1515).
vllm_mlx.bench_serve.run_workload_case · function ```python async vllm_mlx.bench_serve.run_workload_case(client: httpx.AsyncClient, base_url: str, *, workload: Workload, case: WorkloadCase, model: str, runtime: dict, hardware: dict, run_id: str, timestamp: str, repetition: int = 0, scrape: bool = True, include_content: bool = False, cache_reset: Optional[dict] = None) -> dict ``` Run one workload case and return a JSON-serializable result. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `client` | `httpx.AsyncClient` | `yes` | `none` | Required positional or keyword input. | | `base_url` | `str` | `yes` | `none` | Required positional or keyword input. | | `workload` | `Workload` | `yes` | `none` | Required keyword-only input. | | `case` | `WorkloadCase` | `yes` | `none` | Required keyword-only input. | | `model` | `str` | `yes` | `none` | Required keyword-only input. | | `runtime` | `dict` | `yes` | `none` | Required keyword-only input. | | `hardware` | `dict` | `yes` | `none` | Required keyword-only input. | | `run_id` | `str` | `yes` | `none` | Required keyword-only input. | | `timestamp` | `str` | `yes` | `none` | Required keyword-only input. | | `repetition` | `int` | `no` | `0` | Optional keyword-only input; defaults to `0`. | | `scrape` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. | | `include_content` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. | | `cache_reset` | `Optional[dict]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | **Returns** - Type: `dict` - Direct return expressions: `_build_workload_record(case=case, workload=workload, model=model, runtime=runtime, hardware=hardware, run_id=run_id, ti…` **Exceptions and behavior** Function `run_workload_case` calls `scrape_metrics`, `datetime.now(timezone.utc).isoformat`, `datetime.now`, `_assemble_case_request_kwargs`; awaits asynchronous work; returns `_build_workload_record(case=case, workload=workload, model=model, runtime=runtime, hardware=hardware, run_id=run_id, ti…`. No direct `raise` statement appears in this definition. [View source #L1518-L1593](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1518-L1593).
vllm_mlx.bench_serve._group_results_by_case_id · function ```python vllm_mlx.bench_serve._group_results_by_case_id(results: list[dict]) -> dict[str, list[dict]] ``` Bucket workload case records by their ``case_id`` field, defaulting a missing ``case_id`` to the empty string so the grouping is stable. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict[str, list[dict]]` - Direct return expressions: `cases` **Exceptions and behavior** Function `_group_results_by_case_id` calls `cases.setdefault(str(result.get('case_id', '')), []).append`, `cases.setdefault`, `str`, `result.get`; returns `cases`. No direct `raise` statement appears in this definition. [View source #L1596-L1602](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1596-L1602).
vllm_mlx.bench_serve._summarize_case · function ```python vllm_mlx.bench_serve._summarize_case(case_results: list[dict]) -> dict ``` Build the per-case summary block. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `case_results` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{'sample_count': len(case_results), 'repetitions': sorted({int(r.get('repetition', 0)) for r in case_results if r.get('…` **Exceptions and behavior** Function `_summarize_case` calls `r['quality'].get`, `r['policy'].get`, `len`, `sorted`; returns `{'sample_count': len(case_results), 'repetitions': sorted({int(r.get('repetition', 0)) for r in case_results if r.get('…`. No direct `raise` statement appears in this definition. [View source #L1605-L1648](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1605-L1648).
vllm_mlx.bench_serve.summarize_workload_results · function ```python vllm_mlx.bench_serve.summarize_workload_results(results: list[dict]) -> dict ``` Aggregate workload case records into stable qualification summary stats. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[dict]` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{'case_count': len(results), 'unique_case_count': len(cases), 'repetition_count': max((len(summary['repetitions']) for …` **Exceptions and behavior** Function `summarize_workload_results` calls `r['policy'].get`, `_group_results_by_case_id`, `_summarize_case`, `sorted`; returns `{'case_count': len(results), 'unique_case_count': len(cases), 'repetition_count': max((len(summary['repetitions']) for …`. No direct `raise` statement appears in this definition. [View source #L1651-L1689](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1651-L1689).
vllm_mlx.bench_serve.run_bench_serve_workload · function ```python async vllm_mlx.bench_serve.run_bench_serve_workload(*, url: str, workload_path: str, model: Optional[str] = None, output_path: Optional[str] = None, output_format: str = 'json', scrape: bool = True, include_content: bool = False, request_timeout_s: Optional[float] = 300.0, repetitions: int = 1, cache_policy: Optional[str] = None) -> dict ``` Run a declarative workload against a running server. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `url` | `str` | `yes` | `none` | Required keyword-only input. | | `workload_path` | `str` | `yes` | `none` | Required keyword-only input. | | `model` | `Optional[str]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | | `output_path` | `Optional[str]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | | `output_format` | `str` | `no` | `'json'` | Optional keyword-only input; defaults to `'json'`. | | `scrape` | `bool` | `no` | `True` | Optional keyword-only input; defaults to `True`. | | `include_content` | `bool` | `no` | `False` | Optional keyword-only input; defaults to `False`. | | `request_timeout_s` | `Optional[float]` | `no` | `300.0` | Optional keyword-only input; defaults to `300.0`. | | `repetitions` | `int` | `no` | `1` | Optional keyword-only input; defaults to `1`. | | `cache_policy` | `Optional[str]` | `no` | `None` | Optional keyword-only input; defaults to `None`. | **Returns** - Type: `dict` - Direct return expressions: `payload` **Exceptions and behavior** Function `run_bench_serve_workload` calls `ValueError`, `load_workload`, `_normalize_cache_policy`, `workload.defaults.get`; awaits asynchronous work; can raise `ValueError`; returns `payload`. Directly raised exceptions: `ValueError`. [View source #L1692-L1818](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1692-L1818).
vllm_mlx.bench_serve._result_to_dict · function ```python vllm_mlx.bench_serve._result_to_dict(r: BenchServeResult) -> dict ``` Convert a :class:`BenchServeResult` to an ordered dict. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `r` | `BenchServeResult` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{f.name: getattr(r, f.name) for f in _dataclasses.fields(r)}` **Exceptions and behavior** Function `_result_to_dict` calls `getattr`, `_dataclasses.fields`; returns `{f.name: getattr(r, f.name) for f in _dataclasses.fields(r)}`. No direct `raise` statement appears in this definition. [View source #L1840-L1846](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1840-L1846).
vllm_mlx.bench_serve.format_table · function ```python vllm_mlx.bench_serve.format_table(results: list[BenchServeResult]) -> str ``` Render a human-readable terminal table of benchmark results. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[BenchServeResult]` | `yes` | `none` | List of :class:`BenchServeResult` instances. | **Returns** - Type: `str` - Direct return expressions: `_tabulate(rows, headers=_TABLE_COLUMNS, tablefmt='simple')` **Exceptions and behavior** Function `format_table` calls `_result_to_dict`, `d.get`, `isinstance`, `round`; returns `_tabulate(rows, headers=_TABLE_COLUMNS, tablefmt='simple')`. No direct `raise` statement appears in this definition. [View source #L1849-L1871](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1849-L1871).
vllm_mlx.bench_serve.format_json · function ```python vllm_mlx.bench_serve.format_json(results: list[BenchServeResult]) -> str ``` Serialize benchmark results as a JSON array. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[BenchServeResult]` | `yes` | `none` | List of :class:`BenchServeResult` instances. | **Returns** - Type: `str` - Direct return expressions: `json.dumps([_result_to_dict(r) for r in results], indent=2)` **Exceptions and behavior** Function `format_json` calls `json.dumps`, `_result_to_dict`; returns `json.dumps([_result_to_dict(r) for r in results], indent=2)`. No direct `raise` statement appears in this definition. [View source #L1874-L1885](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1874-L1885).
vllm_mlx.bench_serve.format_csv · function ```python vllm_mlx.bench_serve.format_csv(results: list[BenchServeResult]) -> str ``` Serialize benchmark results as CSV with a header row. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[BenchServeResult]` | `yes` | `none` | List of :class:`BenchServeResult` instances. | **Returns** - Type: `str` - Direct return expressions: `buf.getvalue()` **Exceptions and behavior** Function `format_csv` calls `io.StringIO`, `csv_mod.DictWriter`, `writer.writeheader`, `writer.writerow`; returns `buf.getvalue()`. No direct `raise` statement appears in this definition. [View source #L1888-L1904](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1888-L1904).
vllm_mlx.bench_serve._sql_escape · function ```python vllm_mlx.bench_serve._sql_escape(value) -> str ``` Escape a Python value for use as a SQL literal. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `value` | `not annotated` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `'NULL'`; `'1' if value else '0'`; `str(value)`; `f"'{escaped}'"` **Exceptions and behavior** Function `_sql_escape` calls `isinstance`, `math.isnan`, `math.isinf`, `str`; has 4 explicit return paths. No direct `raise` statement appears in this definition. [View source #L1907-L1927](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1907-L1927).
vllm_mlx.bench_serve.format_sql · function ```python vllm_mlx.bench_serve.format_sql(results: list[BenchServeResult]) -> str ``` Emit a SQL ``CREATE TABLE IF NOT EXISTS`` statement and INSERT rows. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[BenchServeResult]` | `yes` | `none` | List of :class:`BenchServeResult` instances. | **Returns** - Type: `str` - Direct return expressions: `'\n'.join(lines)` **Exceptions and behavior** Function `format_sql` calls `_result_to_dict`, `', '.join`, `_sql_escape`, `lines.append`; returns `'\n'.join(lines)`. No direct `raise` statement appears in this definition. [View source #L1945-L1964](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1945-L1964).
vllm_mlx.bench_serve._write_sqlite_rows · function ```python vllm_mlx.bench_serve._write_sqlite_rows(output_path: str, *, table: str, schema: str, columns: list[str], rows: list[dict]) -> None ``` Append benchmark rows to a SQLite database. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `output_path` | `str` | `yes` | `none` | Required positional or keyword input. | | `table` | `str` | `yes` | `none` | Required keyword-only input. | | `schema` | `str` | `yes` | `none` | Required keyword-only input. | | `columns` | `list[str]` | `yes` | `none` | Required keyword-only input. | | `rows` | `list[dict]` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `None` **Exceptions and behavior** Function `_write_sqlite_rows` calls `Path(output_path).expanduser`, `Path`, `_validate_sql_identifier`, `', '.join`. No direct `raise` statement appears in this definition. [View source #L1967-L1990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1967-L1990).
vllm_mlx.bench_serve._validate_sql_identifier · function ```python vllm_mlx.bench_serve._validate_sql_identifier(identifier: str, *, kind: str) -> None ``` Reject unsafe SQL identifiers before string interpolation. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `identifier` | `str` | `yes` | `none` | Required positional or keyword input. | | `kind` | `str` | `yes` | `none` | Required keyword-only input. | **Returns** - Type: `None` **Exceptions and behavior** Function `_validate_sql_identifier` calls `_SQL_IDENTIFIER_RE.fullmatch`, `ValueError`; can raise `ValueError`. Directly raised exceptions: `ValueError`. [View source #L1993-L1996](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1993-L1996).
vllm_mlx.bench_serve.write_sqlite · function ```python vllm_mlx.bench_serve.write_sqlite(results: list[BenchServeResult], output_path: str) -> None ``` Append prompt-sweep benchmark results to a SQLite database. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `results` | `list[BenchServeResult]` | `yes` | `none` | Required positional or keyword input. | | `output_path` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `None` **Exceptions and behavior** Function `write_sqlite` calls `_result_to_dict`, `_write_sqlite_rows`. No direct `raise` statement appears in this definition. [View source #L1999-L2009](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1999-L2009).
vllm_mlx.bench_serve._workload_record_to_row · function ```python vllm_mlx.bench_serve._workload_record_to_row(record: dict) -> dict ``` Function `_workload_record_to_row` calls `record.get`, `metrics.get`, `','.join`, `hardware.get`; returns `{'run_id': record.get('run_id', ''), 'timestamp': record.get('timestamp', ''), 'workload': record.get('workload', ''), …`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `record` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `dict` - Direct return expressions: `{'run_id': record.get('run_id', ''), 'timestamp': record.get('timestamp', ''), 'workload': record.get('workload', ''), …` **Exceptions and behavior** Function `_workload_record_to_row` calls `record.get`, `metrics.get`, `','.join`, `hardware.get`; returns `{'run_id': record.get('run_id', ''), 'timestamp': record.get('timestamp', ''), 'workload': record.get('workload', ''), …`. No direct `raise` statement appears in this definition. [View source #L2069-L2119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2069-L2119).
vllm_mlx.bench_serve.format_workload_table · function ```python vllm_mlx.bench_serve.format_workload_table(payload: dict) -> str ``` Format workload result records as a compact human-readable table. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `_tabulate(rows, headers=_WORKLOAD_TABLE_COLUMNS, tablefmt='simple')` **Exceptions and behavior** Function `format_workload_table` calls `payload.get`, `_workload_record_to_row`, `rows.append`, `isinstance`; returns `_tabulate(rows, headers=_WORKLOAD_TABLE_COLUMNS, tablefmt='simple')`. No direct `raise` statement appears in this definition. [View source #L2122-L2134](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2122-L2134).
vllm_mlx.bench_serve.format_workload_json · function ```python vllm_mlx.bench_serve.format_workload_json(payload: dict) -> str ``` Serialize a workload result payload as indented JSON. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `json.dumps(payload, indent=2)` **Exceptions and behavior** Function `format_workload_json` calls `json.dumps`; returns `json.dumps(payload, indent=2)`. No direct `raise` statement appears in this definition. [View source #L2137-L2140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2137-L2140).
vllm_mlx.bench_serve.format_workload_csv · function ```python vllm_mlx.bench_serve.format_workload_csv(payload: dict) -> str ``` Serialize workload result records with the stable CSV column contract. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `buf.getvalue()` **Exceptions and behavior** Function `format_workload_csv` calls `io.StringIO`, `csv_mod.DictWriter`, `writer.writeheader`, `payload.get`; returns `buf.getvalue()`. No direct `raise` statement appears in this definition. [View source #L2143-L2151](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2143-L2151).
vllm_mlx.bench_serve.format_workload_sql · function ```python vllm_mlx.bench_serve.format_workload_sql(payload: dict) -> str ``` Render SQL statements that create and populate the workload table. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `str` - Direct return expressions: `'\n'.join(lines)` **Exceptions and behavior** Function `format_workload_sql` calls `payload.get`, `_workload_record_to_row`, `', '.join`, `_sql_escape`; returns `'\n'.join(lines)`. No direct `raise` statement appears in this definition. [View source #L2170-L2180](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2170-L2180).
vllm_mlx.bench_serve.write_workload_sqlite · function ```python vllm_mlx.bench_serve.write_workload_sqlite(payload: dict, output_path: str) -> None ``` Append workload result records to a SQLite database. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | | `output_path` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `None` **Exceptions and behavior** Function `write_workload_sqlite` calls `_workload_record_to_row`, `payload.get`, `_write_sqlite_rows`. No direct `raise` statement appears in this definition. [View source #L2183-L2193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2183-L2193).
vllm_mlx.bench_serve.format_workload_payload · function ```python vllm_mlx.bench_serve.format_workload_payload(payload: dict, fmt: str = 'json') -> str ``` Serialize a workload payload in the requested text output format. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `payload` | `dict` | `yes` | `none` | Required positional or keyword input. | | `fmt` | `str` | `no` | `'json'` | Optional positional or keyword input; defaults to `'json'`. | **Returns** - Type: `str` - Direct return expressions: `format_workload_json(payload)`; `format_workload_csv(payload)`; `format_workload_sql(payload)`; `format_workload_table(payload)` **Exceptions and behavior** Function `format_workload_payload` calls `format_workload_json`, `format_workload_csv`, `format_workload_sql`, `format_workload_table`; can raise `ValueError`; has 4 explicit return paths. Directly raised exceptions: `ValueError`. [View source #L2196-L2211](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2196-L2211).
vllm_mlx.bench_serve.run_bench_serve · function ```python async vllm_mlx.bench_serve.run_bench_serve(url: str = 'http://127.0.0.1:8080', model: Optional[str] = None, prompt_sets: list[str] = None, prompt_file: Optional[str] = None, concurrencies: list[int] = None, max_tokens: int = 256, repetitions: int = 3, warmup: int = 1, thinking_values: list[Optional[bool]] = None, extra_bodies: list[str] = None, output_path: Optional[str] = None, fmt: str = 'table', do_validate: bool = True, scrape: bool = True, tag: Optional[str] = None, override_fields: Optional[dict] = None, system_prompt_file: Optional[str] = None, skip_preflight_token_count: bool = False) -> list[BenchServeResult] ``` Run the full bench-serve sweep against a running vllm-mlx server. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `url` | `str` | `no` | `'http://127.0.0.1:8080'` | Base URL of the server. | | `model` | `Optional[str]` | `no` | `None` | Model ID to use. If ``None``, auto-detected from the server. | | `prompt_sets` | `list[str]` | `no` | `None` | List of prompt set names or paths. Defaults to ``["short", "medium", "long"]``. | | `prompt_file` | `Optional[str]` | `no` | `None` | Optional path to an extra prompt file to include. | | `concurrencies` | `list[int]` | `no` | `None` | Concurrency levels to sweep. Defaults to ``[1, 4]``. | | `max_tokens` | `int` | `no` | `256` | Maximum tokens to generate per request. | | `repetitions` | `int` | `no` | `3` | Number of repetitions per sweep config. | | `warmup` | `int` | `no` | `1` | Number of warmup rounds before the first measured repetition. | | `thinking_values` | `list[Optional[bool]]` | `no` | `None` | Values for ``enable_thinking``. Defaults to ``[None]``. | | `extra_bodies` | `list[str]` | `no` | `None` | JSON strings for extra body parameters. Defaults to ``[""]`` (no extra body). | | `output_path` | `Optional[str]` | `no` | `None` | File path to write results to. If ``None``, prints to stdout. | | `fmt` | `str` | `no` | `'table'` | Output format — one of ``"table"``, ``"json"``, ``"csv"``, ``"sql"``, or ``"sqlite"``. | | `do_validate` | `bool` | `no` | `True` | Whether to validate each response. | | `scrape` | `bool` | `no` | `True` | Whether to scrape ``/metrics`` before and after each run. | | `tag` | `Optional[str]` | `no` | `None` | Optional tag string stored in every result row. | | `override_fields` | `Optional[dict]` | `no` | `None` | Dict of field names to override on every result. | | `system_prompt_file` | `Optional[str]` | `no` | `None` | Optional positional or keyword input; defaults to `None`. | | `skip_preflight_token_count` | `bool` | `no` | `False` | Optional positional or keyword input; defaults to `False`. | **Returns** - Type: `list[BenchServeResult]` - Direct return expressions: `[]`; `results` **Exceptions and behavior** Function `run_bench_serve` calls `str`, `uuid.uuid4`, `datetime.now(timezone.utc).isoformat`, `datetime.now`; awaits asynchronous work; can raise `ValueError`; has 2 explicit return paths. Directly raised exceptions: `ValueError`. [View source #L2221-L2638](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2221-L2638).
vllm_mlx.bench_serve.run_bench_serve._mean · nested function ```python vllm_mlx.bench_serve.run_bench_serve._mean(key: str) -> float ``` Nested Function `run_bench_serve._mean` calls `statistics.mean`; returns `statistics.mean(vals) if vals else 0.0`. **Parameters** | Name | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `key` | `str` | `yes` | `none` | Required positional or keyword input. | **Returns** - Type: `float` - Direct return expressions: `statistics.mean(vals) if vals else 0.0` **Exceptions and behavior** Nested Function `run_bench_serve._mean` calls `statistics.mean`; returns `statistics.mean(vals) if vals else 0.0`. No direct `raise` statement appears in this definition. [View source #L2522-L2526](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2522-L2526).
## Complete symbol map This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose. | Symbol | Kind | Signature and inputs | What it does | Source | | --- | --- | --- | --- | --- | | [`WorkloadCase`](#contract-vllm_mlx.bench_serve.WorkloadCase) | class | `WorkloadCase(case_id: str, messages: list[dict], request_path: Optional[str] = None, max_tokens: Optional[int] = None, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, policy_timeout_ms: Optional[int] = None, checks: Optional[dict] = None, tags: tuple[str, ...] = ())` | One declarative benchmark case for contract-style serving tests. | [#L51-L62](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L51-L62) | | [`Workload`](#contract-vllm_mlx.bench_serve.Workload) | class | `Workload(name: str, description: str, defaults: dict, cases: list[WorkloadCase])` | Normalized bench-serve workload manifest. | [#L66-L72](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L66-L72) | | [`load_prompt_set`](#contract-vllm_mlx.bench_serve.load_prompt_set) | function | `load_prompt_set(name_or_path: str) -> list[list[dict]]` | Load a prompt set by builtin name or file path. | [#L75-L137](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L75-L137) | | [`_require_message_list`](#contract-vllm_mlx.bench_serve._require_message_list) | function | `_require_message_list(value: Any, *, label: str) -> list[dict]` | Function `_require_message_list` calls `isinstance`, `ValueError`, `enumerate`; can raise `ValueError`; returns `value`. | [#L140-L148](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L140-L148) | | [`_load_case_request`](#contract-vllm_mlx.bench_serve._load_case_request) | function | `_load_case_request(path: str, *, workload_path: Path, case_id: str) -> dict` | Function `_load_case_request` calls `Path(path).expanduser`, `Path`, `request_path.is_absolute`, `request_path.open`; can raise `ValueError`; returns `request`. | [#L151-L159](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L151-L159) | | [`_request_extra_body`](#contract-vllm_mlx.bench_serve._request_extra_body) | function | `_request_extra_body(request: dict) -> dict` | Function `_request_extra_body` calls `request.items`; returns `{key: value for key, value in request.items() if key not in reserved}`. | [#L162-L171](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L162-L171) | | [`_first_not_none`](#contract-vllm_mlx.bench_serve._first_not_none) | function | `_first_not_none(*values: Any) -> Any` | Function `_first_not_none` has 2 explicit return paths. | [#L174-L178](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L174-L178) | | [`_normalize_tags`](#contract-vllm_mlx.bench_serve._normalize_tags) | function | `_normalize_tags(tags: Any, *, case_id: str) -> tuple[str, ...]` | Coerce a workload case's ``tags`` field to a tuple of strings. | [#L181-L191](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L181-L191) | | [`_merge_case_checks`](#contract-vllm_mlx.bench_serve._merge_case_checks) | function | `_merge_case_checks(default_checks: Any, case_checks: Any, *, case_id: str) -> Optional[dict]` | Merge a case's ``checks`` over the workload defaults. | [#L194-L227](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L194-L227) | | [`_build_workload_case`](#contract-vllm_mlx.bench_serve._build_workload_case) | function | `_build_workload_case(item: Any, idx: int, *, defaults: dict, workload_path: Path) -> WorkloadCase` | Construct one ``WorkloadCase`` from a raw workload entry. | [#L230-L300](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L230-L300) | | [`load_workload`](#contract-vllm_mlx.bench_serve.load_workload) | function | `load_workload(path: str \| Path) -> Workload` | Load a declarative serving benchmark workload. | [#L303-L335](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L303-L335) | | [`BenchServeResult`](#contract-vllm_mlx.bench_serve.BenchServeResult) | class | `BenchServeResult(run_id: str = '', timestamp: str = '', tag: str = '', chip: str = '', gpu_cores: int = 0, memory_gb: float = 0.0, bandwidth_gbs: float = 0.0, os_version: str = '', model_id: str = '', model_type: str = '', engine_type: str = '', mtp_enabled: bool = False, specprefill: bool = False, kv_quant: str = '', cache_type: str = '', prompt_set: str = '', concurrency: int = 1, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: str = '', repetition: int = 0, prompt_tokens: int = 0, ttft_ms: float = 0.0, tpot_ms: float = 0.0, e2e_latency_ms: float = 0.0, gen_tps: float = 0.0, prompt_tps: float = 0.0, throughput_tps: float = 0.0, requests_per_s: float = 0.0, metal_active_gb: float = 0.0, metal_peak_gb: float = 0.0, metal_cache_gb: float = 0.0, cache_hits: int = 0, cache_misses: int = 0, cache_hit_rate: float = 0.0, tokens_saved: int = 0, validated: bool = True)` | Aggregated results from a single bench-serve run configuration. | [#L344-L400](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L344-L400) | | [`expand_sweep`](#contract-vllm_mlx.bench_serve.expand_sweep) | function | `expand_sweep(prompt_sets: list[str], concurrencies: list[int], thinking_values: list[Optional[bool]], extra_bodies: list[str], repetitions: int) -> list[SweepConfig]` | Expand sweep parameters into a flat list of configurations. | [#L411-L444](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L411-L444) | | [`parse_health_response`](#contract-vllm_mlx.bench_serve.parse_health_response) | function | `parse_health_response(data: dict) -> dict` | Extract model identity fields from a GET /health response. | [#L452-L467](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L452-L467) | | [`parse_status_response`](#contract-vllm_mlx.bench_serve.parse_status_response) | function | `parse_status_response(data: dict) -> dict` | Extract metal and cache info from a GET /v1/status response. | [#L470-L496](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L470-L496) | | [`parse_metrics_text`](#contract-vllm_mlx.bench_serve.parse_metrics_text) | function | `parse_metrics_text(text: str) -> dict` | Parse Prometheus text exposition format from GET /metrics. | [#L499-L521](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L499-L521) | | [`parse_metrics_text._extract`](#contract-vllm_mlx.bench_serve.parse_metrics_text._extract) | nested function | `parse_metrics_text._extract(metric_name: str) -> int` | Nested Function `parse_metrics_text._extract` calls `re.escape`, `re.search`, `int`, `m.group`; returns `int(m.group(1)) if m else 0`. | [#L512-L515](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L512-L515) | | [`detect_hardware_fingerprint`](#contract-vllm_mlx.bench_serve.detect_hardware_fingerprint) | function | `detect_hardware_fingerprint() -> dict` | Return a hardware fingerprint dict for the current machine. | [#L524-L573](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L524-L573) | | [`auto_detect_runtime`](#contract-vllm_mlx.bench_serve.auto_detect_runtime) | function | `async auto_detect_runtime(client: httpx.AsyncClient, base_url: str) -> dict` | Query the running server and return a runtime descriptor dict. | [#L576-L642](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L576-L642) | | [`scrape_metrics`](#contract-vllm_mlx.bench_serve.scrape_metrics) | function | `async scrape_metrics(client: httpx.AsyncClient, base_url: str) -> dict` | Scrape Prometheus metrics from the server. | [#L645-L661](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L645-L661) | | [`clear_runtime_cache`](#contract-vllm_mlx.bench_serve.clear_runtime_cache) | function | `async clear_runtime_cache(client: httpx.AsyncClient, base_url: str) -> dict` | Clear server-side runtime caches and return a JSON-serializable event. | [#L664-L684](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L664-L684) | | [`_normalize_cache_policy`](#contract-vllm_mlx.bench_serve._normalize_cache_policy) | function | `_normalize_cache_policy(value: Optional[str]) -> str` | Normalize cache-policy spelling from CLI or workload JSON. | [#L687-L698](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L687-L698) | | [`parse_sse_line`](#contract-vllm_mlx.bench_serve.parse_sse_line) | function | `parse_sse_line(line: str) -> Optional[dict]` | Parse one Server-Sent Events line from a streaming chat completion. | [#L706-L754](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L706-L754) | | [`_cancel_server_request`](#contract-vllm_mlx.bench_serve._cancel_server_request) | function | `async _cancel_server_request(client: httpx.AsyncClient, base_url: str, request_id: Optional[str]) -> None` | Best-effort server-side cancellation for timed-out workload streams. | [#L757-L770](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L757-L770) | | [`accumulate_tool_calls`](#contract-vllm_mlx.bench_serve.accumulate_tool_calls) | function | `accumulate_tool_calls(acc: dict[int, dict], delta_list: list[dict]) -> None` | Merge streamed OpenAI tool-call deltas into *acc* by index. | [#L773-L792](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L773-L792) | | [`finalize_tool_calls`](#contract-vllm_mlx.bench_serve.finalize_tool_calls) | function | `finalize_tool_calls(acc: dict[int, dict]) -> list[dict]` | Return accumulated tool calls in stream index order. | [#L795-L797](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L795-L797) | | [`compute_request_metrics`](#contract-vllm_mlx.bench_serve.compute_request_metrics) | function | `compute_request_metrics(t_start: float, t_first_token: float, token_times: list, t_end: float, prompt_tokens: int, completion_tokens: int) -> dict` | Compute standard latency and throughput metrics for a single request. | [#L800-L852](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L800-L852) | | [`count_prompt_tokens`](#contract-vllm_mlx.bench_serve.count_prompt_tokens) | function | `async count_prompt_tokens(client: httpx.AsyncClient, base_url: str, messages: list[dict], model: str) -> int` | Count prompt tokens for a message list by sending a 1-token request. | [#L855-L889](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L855-L889) | | [`stream_chat_completion`](#contract-vllm_mlx.bench_serve.stream_chat_completion) | function | `async stream_chat_completion(client: httpx.AsyncClient, base_url: str, messages: list[dict], model: str, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, timeout_s: Optional[float] = None) -> dict` | Send a streaming chat completion and collect per-token timing data. | [#L892-L1012](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L892-L1012) | | [`stream_chat_completion._consume_stream`](#contract-vllm_mlx.bench_serve.stream_chat_completion._consume_stream) | nested function | `async stream_chat_completion._consume_stream() -> None` | Nested Function `stream_chat_completion._consume_stream` calls `client.stream`, `response.raise_for_status`, `response.aiter_lines`, `parse_sse_line`. | [#L946-L975](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L946-L975) | | [`validate_response`](#contract-vllm_mlx.bench_serve.validate_response) | function | `validate_response(finish_reason: Optional[str], content: str, status_code: int, *, tool_calls: Optional[list[dict]] = None) -> tuple[bool, str]` | Validate a single streaming response result. | [#L1020-L1049](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1020-L1049) | | [`_check_finish_reason`](#contract-vllm_mlx.bench_serve._check_finish_reason) | function | `_check_finish_reason(allowed: Any, finish_reason: Optional[str]) -> list[str]` | Verify ``finish_reason`` is in the allowed set, if one is configured. | [#L1052-L1059](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1052-L1059) | | [`_check_length_bounds`](#contract-vllm_mlx.bench_serve._check_length_bounds) | function | `_check_length_bounds(min_chars: Any, max_chars: Any, content: str) -> list[str]` | Apply ``min_chars`` / ``max_chars`` content-length bounds. | [#L1062-L1069](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1062-L1069) | | [`_check_regex_patterns`](#contract-vllm_mlx.bench_serve._check_regex_patterns) | function | `_check_regex_patterns(patterns: Any, content: str, *, kind: str, expect_match: bool) -> list[str]` | Validate that each pattern either matches or does not, per ``expect_match``. | [#L1072-L1096](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1072-L1096) | | [`_check_json_content`](#contract-vllm_mlx.bench_serve._check_json_content) | function | `_check_json_content(should_be_json: Any, content: str) -> list[str]` | Verify ``content`` parses as JSON when ``checks['json']`` is truthy. | [#L1099-L1107](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1099-L1107) | | [`_check_tool_call_count_and_names`](#contract-vllm_mlx.bench_serve._check_tool_call_count_and_names) | function | `_check_tool_call_count_and_names(checks: dict, tool_calls: list[dict]) -> list[str]` | Apply ``no_tool_calls`` / ``tool_call_count`` / ``tool_call_names``. | [#L1110-L1132](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1110-L1132) | | [`_check_tool_call_args`](#contract-vllm_mlx.bench_serve._check_tool_call_args) | function | `_check_tool_call_args(required_args: Any, tool_calls: list[dict]) -> list[str]` | Validate parsed JSON arguments include the required keys per function. | [#L1135-L1174](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1135-L1174) | | [`validate_quality_checks`](#contract-vllm_mlx.bench_serve.validate_quality_checks) | function | `validate_quality_checks(finish_reason: Optional[str], content: str, checks: Optional[dict], *, status_code: int = 200, tool_calls: Optional[list[dict]] = None) -> tuple[bool, list[str]]` | Validate content against generic workload quality checks. | [#L1177-L1231](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1177-L1231) | | [`compute_summary_stats`](#contract-vllm_mlx.bench_serve.compute_summary_stats) | function | `compute_summary_stats(values: list[float]) -> dict` | Compute summary statistics over a list of floats. | [#L1234-L1276](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1234-L1276) | | [`compute_summary_stats._percentile`](#contract-vllm_mlx.bench_serve.compute_summary_stats._percentile) | nested function | `compute_summary_stats._percentile(p: float) -> float` | Nested Function `compute_summary_stats._percentile` calls `int`; has 3 explicit return paths. | [#L1256-L1266](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1256-L1266) | | [`run_concurrent_requests`](#contract-vllm_mlx.bench_serve.run_concurrent_requests) | function | `async run_concurrent_requests(client: httpx.AsyncClient, base_url: str, prompts: list[list[dict]], model: str, concurrency: int, max_tokens: int = 256, enable_thinking: Optional[bool] = None, extra_body: Optional[dict] = None, do_validate: bool = True) -> list[dict]` | Fire ``concurrency`` concurrent streaming requests and collect results. | [#L1279-L1342](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1279-L1342) | | [`run_concurrent_requests._single`](#contract-vllm_mlx.bench_serve.run_concurrent_requests._single) | nested function | `async run_concurrent_requests._single(messages: list[dict]) -> dict` | Nested Function `run_concurrent_requests._single` calls `stream_chat_completion`, `validate_response`, `result.get`, `str`; awaits asynchronous work; has 2 explicit return paths. | [#L1315-L1339](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1315-L1339) | | [`_summary_or_empty`](#contract-vllm_mlx.bench_serve._summary_or_empty) | function | `_summary_or_empty(values: list[float]) -> dict` | Function `_summary_or_empty` calls `compute_summary_stats`; returns `compute_summary_stats(values) if values else {}`. | [#L1345-L1346](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1345-L1346) | | [`_resolve_max_tokens`](#contract-vllm_mlx.bench_serve._resolve_max_tokens) | function | `_resolve_max_tokens(case: WorkloadCase, workload: Workload) -> int` | Return the effective ``max_tokens`` for a case, falling back to workload defaults and finally to 256. | [#L1349-L1352](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1349-L1352) | | [`_assemble_case_request_kwargs`](#contract-vllm_mlx.bench_serve._assemble_case_request_kwargs) | function | `_assemble_case_request_kwargs(case: WorkloadCase, workload: Workload, model: str) -> dict` | Build the keyword-arguments dict passed to ``stream_chat_completion`` for one case, applying max_tokens fallback and converting ``policy_timeout_ms`` to seconds. | [#L1355-L1372](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1355-L1372) | | [`_empty_completion_result`](#contract-vllm_mlx.bench_serve._empty_completion_result) | function | `_empty_completion_result() -> dict` | Zero-valued completion result used when ``stream_chat_completion`` raises. | [#L1375-L1390](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1375-L1390) | | [`_fetch_post_run_status`](#contract-vllm_mlx.bench_serve._fetch_post_run_status) | function | `async _fetch_post_run_status(client: httpx.AsyncClient, base_url: str) -> dict` | GET ``/v1/status`` after a case run, swallowing transport errors so a missing or temporarily-unavailable status endpoint does not fail the case record. | [#L1393-L1402](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1393-L1402) | | [`_compute_within_policy_timeout`](#contract-vllm_mlx.bench_serve._compute_within_policy_timeout) | function | `_compute_within_policy_timeout(timeout_ms: Optional[int], *, error_present: bool, e2e_latency_ms: float) -> Optional[bool]` | Resolve the ``policy.within_timeout`` field. | [#L1405-L1418](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1405-L1418) | | [`_build_tool_calls_summary`](#contract-vllm_mlx.bench_serve._build_tool_calls_summary) | function | `_build_tool_calls_summary(tool_calls: Any) -> Optional[dict]` | Compact summary of streamed tool calls for the case record. | [#L1421-L1434](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1421-L1434) | | [`_build_workload_record`](#contract-vllm_mlx.bench_serve._build_workload_record) | function | `_build_workload_record(*, case: WorkloadCase, workload: Workload, model: str, runtime: dict, hardware: dict, run_id: str, timestamp: str, started_wall: str, repetition: int, result: dict, error: str, quality_ok: bool, quality_issues: list[str], content: str, cache_hits_delta: int, cache_misses_delta: int, tokens_saved_delta: int, status_after: dict, cache_reset: Optional[dict], include_content: bool) -> dict` | Assemble the JSON-serializable workload-case record from the raw inputs and the completion result. | [#L1437-L1515](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1437-L1515) | | [`run_workload_case`](#contract-vllm_mlx.bench_serve.run_workload_case) | function | `async run_workload_case(client: httpx.AsyncClient, base_url: str, *, workload: Workload, case: WorkloadCase, model: str, runtime: dict, hardware: dict, run_id: str, timestamp: str, repetition: int = 0, scrape: bool = True, include_content: bool = False, cache_reset: Optional[dict] = None) -> dict` | Run one workload case and return a JSON-serializable result. | [#L1518-L1593](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1518-L1593) | | [`_group_results_by_case_id`](#contract-vllm_mlx.bench_serve._group_results_by_case_id) | function | `_group_results_by_case_id(results: list[dict]) -> dict[str, list[dict]]` | Bucket workload case records by their ``case_id`` field, defaulting a missing ``case_id`` to the empty string so the grouping is stable. | [#L1596-L1602](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1596-L1602) | | [`_summarize_case`](#contract-vllm_mlx.bench_serve._summarize_case) | function | `_summarize_case(case_results: list[dict]) -> dict` | Build the per-case summary block. | [#L1605-L1648](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1605-L1648) | | [`summarize_workload_results`](#contract-vllm_mlx.bench_serve.summarize_workload_results) | function | `summarize_workload_results(results: list[dict]) -> dict` | Aggregate workload case records into stable qualification summary stats. | [#L1651-L1689](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1651-L1689) | | [`run_bench_serve_workload`](#contract-vllm_mlx.bench_serve.run_bench_serve_workload) | function | `async run_bench_serve_workload(*, url: str, workload_path: str, model: Optional[str] = None, output_path: Optional[str] = None, output_format: str = 'json', scrape: bool = True, include_content: bool = False, request_timeout_s: Optional[float] = 300.0, repetitions: int = 1, cache_policy: Optional[str] = None) -> dict` | Run a declarative workload against a running server. | [#L1692-L1818](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1692-L1818) | | [`_result_to_dict`](#contract-vllm_mlx.bench_serve._result_to_dict) | function | `_result_to_dict(r: BenchServeResult) -> dict` | Convert a :class:`BenchServeResult` to an ordered dict. | [#L1840-L1846](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1840-L1846) | | [`format_table`](#contract-vllm_mlx.bench_serve.format_table) | function | `format_table(results: list[BenchServeResult]) -> str` | Render a human-readable terminal table of benchmark results. | [#L1849-L1871](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1849-L1871) | | [`format_json`](#contract-vllm_mlx.bench_serve.format_json) | function | `format_json(results: list[BenchServeResult]) -> str` | Serialize benchmark results as a JSON array. | [#L1874-L1885](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1874-L1885) | | [`format_csv`](#contract-vllm_mlx.bench_serve.format_csv) | function | `format_csv(results: list[BenchServeResult]) -> str` | Serialize benchmark results as CSV with a header row. | [#L1888-L1904](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1888-L1904) | | [`_sql_escape`](#contract-vllm_mlx.bench_serve._sql_escape) | function | `_sql_escape(value) -> str` | Escape a Python value for use as a SQL literal. | [#L1907-L1927](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1907-L1927) | | [`format_sql`](#contract-vllm_mlx.bench_serve.format_sql) | function | `format_sql(results: list[BenchServeResult]) -> str` | Emit a SQL ``CREATE TABLE IF NOT EXISTS`` statement and INSERT rows. | [#L1945-L1964](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1945-L1964) | | [`_write_sqlite_rows`](#contract-vllm_mlx.bench_serve._write_sqlite_rows) | function | `_write_sqlite_rows(output_path: str, *, table: str, schema: str, columns: list[str], rows: list[dict]) -> None` | Append benchmark rows to a SQLite database. | [#L1967-L1990](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1967-L1990) | | [`_validate_sql_identifier`](#contract-vllm_mlx.bench_serve._validate_sql_identifier) | function | `_validate_sql_identifier(identifier: str, *, kind: str) -> None` | Reject unsafe SQL identifiers before string interpolation. | [#L1993-L1996](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1993-L1996) | | [`write_sqlite`](#contract-vllm_mlx.bench_serve.write_sqlite) | function | `write_sqlite(results: list[BenchServeResult], output_path: str) -> None` | Append prompt-sweep benchmark results to a SQLite database. | [#L1999-L2009](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L1999-L2009) | | [`_workload_record_to_row`](#contract-vllm_mlx.bench_serve._workload_record_to_row) | function | `_workload_record_to_row(record: dict) -> dict` | Function `_workload_record_to_row` calls `record.get`, `metrics.get`, `','.join`, `hardware.get`; returns `{'run_id': record.get('run_id', ''), 'timestamp': record.get('timestamp', ''), 'workload': record.get('workload', ''), …`. | [#L2069-L2119](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2069-L2119) | | [`format_workload_table`](#contract-vllm_mlx.bench_serve.format_workload_table) | function | `format_workload_table(payload: dict) -> str` | Format workload result records as a compact human-readable table. | [#L2122-L2134](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2122-L2134) | | [`format_workload_json`](#contract-vllm_mlx.bench_serve.format_workload_json) | function | `format_workload_json(payload: dict) -> str` | Serialize a workload result payload as indented JSON. | [#L2137-L2140](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2137-L2140) | | [`format_workload_csv`](#contract-vllm_mlx.bench_serve.format_workload_csv) | function | `format_workload_csv(payload: dict) -> str` | Serialize workload result records with the stable CSV column contract. | [#L2143-L2151](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2143-L2151) | | [`format_workload_sql`](#contract-vllm_mlx.bench_serve.format_workload_sql) | function | `format_workload_sql(payload: dict) -> str` | Render SQL statements that create and populate the workload table. | [#L2170-L2180](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2170-L2180) | | [`write_workload_sqlite`](#contract-vllm_mlx.bench_serve.write_workload_sqlite) | function | `write_workload_sqlite(payload: dict, output_path: str) -> None` | Append workload result records to a SQLite database. | [#L2183-L2193](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2183-L2193) | | [`format_workload_payload`](#contract-vllm_mlx.bench_serve.format_workload_payload) | function | `format_workload_payload(payload: dict, fmt: str = 'json') -> str` | Serialize a workload payload in the requested text output format. | [#L2196-L2211](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2196-L2211) | | [`run_bench_serve`](#contract-vllm_mlx.bench_serve.run_bench_serve) | function | `async run_bench_serve(url: str = 'http://127.0.0.1:8080', model: Optional[str] = None, prompt_sets: list[str] = None, prompt_file: Optional[str] = None, concurrencies: list[int] = None, max_tokens: int = 256, repetitions: int = 3, warmup: int = 1, thinking_values: list[Optional[bool]] = None, extra_bodies: list[str] = None, output_path: Optional[str] = None, fmt: str = 'table', do_validate: bool = True, scrape: bool = True, tag: Optional[str] = None, override_fields: Optional[dict] = None, system_prompt_file: Optional[str] = None, skip_preflight_token_count: bool = False) -> list[BenchServeResult]` | Run the full bench-serve sweep against a running vllm-mlx server. | [#L2221-L2638](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2221-L2638) | | [`run_bench_serve._mean`](#contract-vllm_mlx.bench_serve.run_bench_serve._mean) | nested function | `run_bench_serve._mean(key: str) -> float` | Nested Function `run_bench_serve._mean` calls `statistics.mean`; returns `statistics.mean(vals) if vals else 0.0`. | [#L2522-L2526](https://github.com/waybarrios/vllm-mlx/blob/a69d47912bcb21d8fe04d48f75fa896b620ffcfa/vllm_mlx/bench_serve.py#L2522-L2526) |