examples.video_benchmark

Video Benchmark Script for vllm-mlx Tests Vision-Language Models with video at different configurations (FPS, frame count, resolution) and measures performance metrics.

View the complete module source at #L1-L563.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

examples.video_benchmark

Video Benchmark Script for vllm-mlx

Tests Vision-Language Models with video at different configurations (FPS, frame count, resolution) and measures performance metrics.

Usage

Direct API benchmark (no server needed):

python examples/video_benchmark.py --model mlx-community/Qwen3-VL-4B-Instruct-3bit

With video URL:

python examples/video_benchmark.py --video-url https://example.com/video.mp4

Quick test:

python examples/video_benchmark.py --quick

examples.video_benchmark.logger module-attribute

logger = logging.getLogger(__name__)

examples.video_benchmark.SAMPLE_VIDEOS module-attribute

SAMPLE_VIDEOS = {'480x270_30s': {'url': 'https://file-examples.com/storage/feb05093c66764aa00cbc58/2017/04/file_example_MP4_480_1_5MG.mp4', 'description': 'Sample video 480x270 (30s)', 'resolution': '480x270', 'duration': 30}, '640x360_30s': {'url': 'https://file-examples.com/storage/feb05093c66764aa00cbc58/2017/04/file_example_MP4_640_3MG.mp4', 'description': 'Sample video 640x360 (30s)', 'resolution': '640x360', 'duration': 30}, '1280x720_30s': {'url': 'https://file-examples.com/storage/feb05093c66764aa00cbc58/2017/04/file_example_MP4_1280_10MG.mp4', 'description': 'Sample video 1280x720 HD (30s)', 'resolution': '1280x720', 'duration': 30}, 'bunny_240p': {'url': 'https://docs.evostream.com/sample_content/assets/bunny.mp4', 'description': 'Big Buck Bunny 240p', 'resolution': '424x240', 'duration': 60}, 'sintel_720p': {'url': 'https://docs.evostream.com/sample_content/assets/sintel1m720p.mp4', 'description': 'Sintel Trailer 720p', 'resolution': '1280x720', 'duration': 60}, 'big_buck_bunny_10s': {'url': 'https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4', 'description': 'Big Buck Bunny (10s, 360p)', 'resolution': '640x360', 'duration': 10}}

examples.video_benchmark.VideoBenchmarkResult dataclass

VideoBenchmarkResult(config_name: str, fps: float, max_frames: int, frames_extracted: int, video_duration: float, time_seconds: float, prompt_tokens: int, completion_tokens: int, tokens_per_second: float, response_preview: str)

Result from a single video benchmark run.

examples.video_benchmark.VideoBenchmarkResult.config_name instance-attribute

config_name: str

examples.video_benchmark.VideoBenchmarkResult.fps instance-attribute

fps: float

examples.video_benchmark.VideoBenchmarkResult.max_frames instance-attribute

max_frames: int

examples.video_benchmark.VideoBenchmarkResult.frames_extracted instance-attribute

frames_extracted: int

examples.video_benchmark.VideoBenchmarkResult.video_duration instance-attribute

video_duration: float

examples.video_benchmark.VideoBenchmarkResult.time_seconds instance-attribute

time_seconds: float

examples.video_benchmark.VideoBenchmarkResult.prompt_tokens instance-attribute

prompt_tokens: int

examples.video_benchmark.VideoBenchmarkResult.completion_tokens instance-attribute

completion_tokens: int

examples.video_benchmark.VideoBenchmarkResult.tokens_per_second instance-attribute

tokens_per_second: float

examples.video_benchmark.VideoBenchmarkResult.response_preview instance-attribute

response_preview: str

examples.video_benchmark.create_test_video

create_test_video(duration: float = 5.0, fps: float = 30.0, width: int = 640, height: int = 480) -> str

Create a synthetic test video with colored frames and text.

Parameters:

• duration (float, default: 5.0 ) –

  Video duration in seconds

• fps (float, default: 30.0 ) –

  Frames per second

• width (int, default: 640 ) –

  Video width

• height (int, default: 480 ) –

  Video height

Returns:

• str –

  Path to created video file

Source code in examples/video_benchmark.py

def create_test_video(
    duration: float = 5.0,
    fps: float = 30.0,
    width: int = 640,
    height: int = 480,
) -> str:
    """
    Create a synthetic test video with colored frames and text.

    Args:
        duration: Video duration in seconds
        fps: Frames per second
        width: Video width
        height: Video height

    Returns:
        Path to created video file
    """
    temp_file = tempfile.NamedTemporaryFile(suffix=".mp4", delete=False)
    temp_file.close()

    fourcc = cv2.VideoWriter_fourcc(*'mp4v')
    out = cv2.VideoWriter(temp_file.name, fourcc, fps, (width, height))

    total_frames = int(duration * fps)

    # Different scenes with colors
    scenes = [
        ((255, 0, 0), "Blue Scene"),      # Blue (BGR)
        ((0, 255, 0), "Green Scene"),     # Green
        ((0, 0, 255), "Red Scene"),       # Red
        ((255, 255, 0), "Cyan Scene"),    # Cyan
        ((255, 0, 255), "Magenta Scene"), # Magenta
        ((0, 255, 255), "Yellow Scene"),  # Yellow
    ]

    frames_per_scene = total_frames // len(scenes)

    for i in range(total_frames):
        frame = np.zeros((height, width, 3), dtype=np.uint8)
        scene_idx = min(i // frames_per_scene, len(scenes) - 1)
        color, scene_name = scenes[scene_idx]
        frame[:] = color

        # Add scene name text
        cv2.putText(
            frame,
            scene_name,
            (width // 4, height // 2 - 30),
            cv2.FONT_HERSHEY_SIMPLEX,
            1.5,
            (255, 255, 255),
            3
        )

        # Add frame counter
        cv2.putText(
            frame,
            f"Frame {i}/{total_frames}",
            (width // 4, height // 2 + 30),
            cv2.FONT_HERSHEY_SIMPLEX,
            1.0,
            (255, 255, 255),
            2
        )

        # Add timestamp
        timestamp = i / fps
        cv2.putText(
            frame,
            f"Time: {timestamp:.1f}s",
            (width // 4, height // 2 + 70),
            cv2.FONT_HERSHEY_SIMPLEX,
            0.8,
            (200, 200, 200),
            2
        )

        out.write(frame)

    out.release()
    return temp_file.name

examples.video_benchmark.download_video

download_video(url: str, timeout: int = 120) -> str

Download video from URL.

Source code in examples/video_benchmark.py

def download_video(url: str, timeout: int = 120) -> str:
    """Download video from URL."""
    import requests

    headers = {
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)"
    }

    logger.info(f"Downloading video from: {url}")
    response = requests.get(url, timeout=timeout, headers=headers, stream=True)
    response.raise_for_status()

    temp_file = tempfile.NamedTemporaryFile(suffix=".mp4", delete=False)
    for chunk in response.iter_content(chunk_size=8192):
        temp_file.write(chunk)
    temp_file.close()

    file_size = Path(temp_file.name).stat().st_size
    logger.info(f"Downloaded: {file_size / 1024 / 1024:.1f} MB")

    return temp_file.name

examples.video_benchmark.get_video_info

get_video_info(video_path: str) -> dict

Get information about a video file.

Source code in examples/video_benchmark.py

def get_video_info(video_path: str) -> dict:
    """Get information about a video file."""
    cap = cv2.VideoCapture(video_path)
    if not cap.isOpened():
        return {"error": "Cannot open video"}

    info = {
        "path": video_path,
        "total_frames": int(cap.get(cv2.CAP_PROP_FRAME_COUNT)),
        "fps": cap.get(cv2.CAP_PROP_FPS) or 30.0,
        "width": int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
        "height": int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)),
    }
    info["duration"] = info["total_frames"] / info["fps"] if info["fps"] > 0 else 0

    cap.release()
    return info

examples.video_benchmark.run_video_benchmark

run_video_benchmark(model, video_path: str, fps: float, max_frames: int, config_name: str, warmup: bool = False) -> VideoBenchmarkResult

Run a single video benchmark configuration.

Source code in examples/video_benchmark.py

def run_video_benchmark(
    model,
    video_path: str,
    fps: float,
    max_frames: int,
    config_name: str,
    warmup: bool = False,
) -> VideoBenchmarkResult:
    """Run a single video benchmark configuration."""

    video_info = get_video_info(video_path)

    if not warmup:
        print(f"  {config_name:>20} | fps={fps:<4} max_frames={max_frames:<3} |", end=" ", flush=True)

    start_time = time.perf_counter()

    output = model.generate(
        prompt="Describe what happens in this video. What do you see?",
        videos=[video_path],
        video_fps=fps,
        video_max_frames=max_frames,
        max_tokens=150,
        temperature=0.7,
    )

    elapsed = time.perf_counter() - start_time

    prompt_tokens = output.prompt_tokens
    completion_tokens = output.completion_tokens
    tps = completion_tokens / elapsed if elapsed > 0 else 0

    # Count actual frames extracted (approximation)
    duration = video_info["duration"]
    frames_from_fps = int(duration * fps)
    frames_extracted = min(frames_from_fps, max_frames, video_info["total_frames"])

    if not warmup:
        print(f"{elapsed:>5.2f}s | {frames_extracted:>2} frames | {completion_tokens:>3} tok | {tps:>5.1f} tok/s")

    return VideoBenchmarkResult(
        config_name=config_name,
        fps=fps,
        max_frames=max_frames,
        frames_extracted=frames_extracted,
        video_duration=duration,
        time_seconds=elapsed,
        prompt_tokens=prompt_tokens,
        completion_tokens=completion_tokens,
        tokens_per_second=tps,
        response_preview=output.text[:100] + "..." if len(output.text) > 100 else output.text,
    )

examples.video_benchmark.run_benchmark

run_benchmark(model_name: str, video_path: str = None, video_url: str = None, video_duration: float = 10.0, warmup_runs: int = 1, quick: bool = False) -> list[VideoBenchmarkResult]

Run full video benchmark across multiple configurations.

Parameters:

• model_name (str) –

  VLM model to use

• video_path (str, default: None ) –

  Local video file path

• video_url (str, default: None ) –

  URL to download video from

• video_duration (float, default: 10.0 ) –

  Duration for synthetic video

• warmup_runs (int, default: 1 ) –

  Number of warmup runs

• quick (bool, default: False ) –

  Run quick benchmark with fewer configs

Returns:

• list[VideoBenchmarkResult] –

  List of VideoBenchmarkResult objects

Source code in examples/video_benchmark.py

def run_benchmark(
    model_name: str,
    video_path: str = None,
    video_url: str = None,
    video_duration: float = 10.0,
    warmup_runs: int = 1,
    quick: bool = False,
) -> list[VideoBenchmarkResult]:
    """
    Run full video benchmark across multiple configurations.

    Args:
        model_name: VLM model to use
        video_path: Local video file path
        video_url: URL to download video from
        video_duration: Duration for synthetic video
        warmup_runs: Number of warmup runs
        quick: Run quick benchmark with fewer configs

    Returns:
        List of VideoBenchmarkResult objects
    """
    from vllm_mlx.models.vlm import MLXVisionLanguageModel

    # Load model
    print(f"\nLoading model: {model_name}")
    start = time.time()
    model = MLXVisionLanguageModel(model_name)
    model.load()
    load_time = time.time() - start
    print(f"Model loaded in {load_time:.2f}s")

    # Get or create video
    if video_path and Path(video_path).exists():
        print(f"\nUsing local video: {video_path}")
    elif video_url:
        video_path = download_video(video_url)
    else:
        print(f"\nCreating synthetic test video ({video_duration}s)...")
        video_path = create_test_video(duration=video_duration)

    video_info = get_video_info(video_path)
    print(f"Video: {video_info['width']}x{video_info['height']}, "
          f"{video_info['duration']:.1f}s, {video_info['fps']:.1f} fps, "
          f"{video_info['total_frames']} frames")

    # Define benchmark configurations
    if quick:
        configs = [
            ("4 frames @ 1fps", 1.0, 4),
            ("8 frames @ 2fps", 2.0, 8),
            ("16 frames @ 2fps", 2.0, 16),
        ]
    else:
        configs = [
            # Varying FPS with same max_frames
            ("2 frames @ 0.5fps", 0.5, 2),
            ("4 frames @ 1fps", 1.0, 4),
            ("8 frames @ 2fps", 2.0, 8),
            ("16 frames @ 4fps", 4.0, 16),
            ("32 frames @ 4fps", 4.0, 32),

            # Varying max_frames with same FPS
            ("4 frames @ 2fps", 2.0, 4),
            ("8 frames @ 2fps", 2.0, 8),
            ("12 frames @ 2fps", 2.0, 12),
            ("16 frames @ 2fps", 2.0, 16),
            ("24 frames @ 2fps", 2.0, 24),

            # High density
            ("32 frames @ 8fps", 8.0, 32),
            ("48 frames @ 8fps", 8.0, 48),
        ]

    # Warmup
    if warmup_runs > 0:
        print(f"\nRunning {warmup_runs} warmup run(s)...")
        for _ in range(warmup_runs):
            run_video_benchmark(model, video_path, 1.0, 4, "warmup", warmup=True)
        print("Warmup complete.")

    # Run benchmarks
    print("\n" + "=" * 80)
    print("VIDEO BENCHMARK - Frame Count & FPS Performance")
    print("=" * 80)
    print(f"Model:          {model_name}")
    print(f"Video Duration: {video_info['duration']:.1f}s")
    print(f"Video Size:     {video_info['width']}x{video_info['height']}")
    print("-" * 80)
    print(f"  {'Configuration':>20} | {'Params':<22} | {'Time':>6} | {'Frames':>6} | {'Tokens':>4} | {'Speed':>9}")
    print("-" * 80)

    results = []
    for config_name, fps, max_frames in configs:
        try:
            result = run_video_benchmark(model, video_path, fps, max_frames, config_name)
            results.append(result)
        except Exception as e:
            print(f"  Error with {config_name}: {e}")

    return results

examples.video_benchmark.print_results

print_results(results: list[VideoBenchmarkResult])

Print benchmark results in a nice table.

Source code in examples/video_benchmark.py

def print_results(results: list[VideoBenchmarkResult]):
    """Print benchmark results in a nice table."""
    from tabulate import tabulate

    if not results:
        print("No results to display.")
        return

    print("\n" + "=" * 80)
    print("BENCHMARK RESULTS SUMMARY")
    print("=" * 80)

    # Table by frame count
    print("\n### By Frame Count ###")
    table_data = []
    for r in sorted(results, key=lambda x: x.frames_extracted):
        table_data.append([
            r.config_name,
            r.frames_extracted,
            f"{r.fps}",
            f"{r.time_seconds:.2f}s",
            r.completion_tokens,
            f"{r.tokens_per_second:.1f}",
        ])

    headers = ["Config", "Frames", "FPS", "Time", "Tokens", "Tok/s"]
    print(tabulate(table_data, headers=headers, tablefmt="simple"))

    # Summary stats
    total_time = sum(r.time_seconds for r in results)
    total_tokens = sum(r.completion_tokens for r in results)
    avg_tps = total_tokens / total_time if total_time > 0 else 0

    print("-" * 80)
    print(f"Total Time:      {total_time:.2f}s")
    print(f"Total Tokens:    {total_tokens}")
    print(f"Average Tok/s:   {avg_tps:.1f}")

    # Find best/worst
    fastest = min(results, key=lambda r: r.time_seconds)
    slowest = max(results, key=lambda r: r.time_seconds)
    most_frames = max(results, key=lambda r: r.frames_extracted)

    print(f"\nFastest:     {fastest.config_name} ({fastest.time_seconds:.2f}s)")
    print(f"Slowest:     {slowest.config_name} ({slowest.time_seconds:.2f}s)")
    print(f"Most Frames: {most_frames.config_name} ({most_frames.frames_extracted} frames)")

    # Frames vs Speed analysis
    print("\n### Frames vs Speed Analysis ###")
    frame_groups = {}
    for r in results:
        key = r.frames_extracted
        if key not in frame_groups:
            frame_groups[key] = []
        frame_groups[key].append(r)

    analysis_data = []
    for frames in sorted(frame_groups.keys()):
        group = frame_groups[frames]
        avg_time = sum(r.time_seconds for r in group) / len(group)
        avg_tps = sum(r.tokens_per_second for r in group) / len(group)
        analysis_data.append([frames, f"{avg_time:.2f}s", f"{avg_tps:.1f}"])

    print(tabulate(analysis_data, headers=["Frames", "Avg Time", "Avg Tok/s"], tablefmt="simple"))

    # Sample response
    print("\n" + "-" * 80)
    print("Sample Response (first config):")
    print(f"  \"{results[0].response_preview}\"")

examples.video_benchmark.save_results

save_results(results: list[VideoBenchmarkResult], output_path: str, model_name: str)

Save benchmark results to JSON file.

Source code in examples/video_benchmark.py

def save_results(results: list[VideoBenchmarkResult], output_path: str, model_name: str):
    """Save benchmark results to JSON file."""
    data = {
        "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
        "model": model_name,
        "benchmark_type": "video",
        "results": [
            {
                "config_name": r.config_name,
                "fps": r.fps,
                "max_frames": r.max_frames,
                "frames_extracted": r.frames_extracted,
                "video_duration": r.video_duration,
                "time_seconds": r.time_seconds,
                "prompt_tokens": r.prompt_tokens,
                "completion_tokens": r.completion_tokens,
                "tokens_per_second": r.tokens_per_second,
                "response_preview": r.response_preview,
            }
            for r in results
        ]
    }

    with open(output_path, "w") as f:
        json.dump(data, f, indent=2)

    print(f"\nResults saved to: {output_path}")

examples.video_benchmark.main

main()

Source code in examples/video_benchmark.py

def main():
    parser = argparse.ArgumentParser(
        description="Video Benchmark for vllm-mlx VLM models",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
    # Basic benchmark with default model
    python examples/video_benchmark.py

    # Specify model
    python examples/video_benchmark.py --model mlx-community/Qwen3-VL-8B-Instruct-4bit

    # Use video from URL
    python examples/video_benchmark.py --video-url https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4

    # Quick test
    python examples/video_benchmark.py --quick

    # Save results
    python examples/video_benchmark.py --output video_benchmark.json
        """,
    )

    parser.add_argument(
        "--model",
        type=str,
        default="mlx-community/Qwen3-VL-4B-Instruct-3bit",
        help="VLM model to use",
    )
    parser.add_argument(
        "--video",
        type=str,
        default=None,
        help="Path to local video file",
    )
    parser.add_argument(
        "--video-url",
        type=str,
        default=None,
        help="URL to download video from",
    )
    parser.add_argument(
        "--duration",
        type=float,
        default=10.0,
        help="Duration of synthetic test video (seconds)",
    )
    parser.add_argument(
        "--warmup",
        type=int,
        default=1,
        help="Number of warmup runs",
    )
    parser.add_argument(
        "--quick",
        action="store_true",
        help="Run quick benchmark with fewer configurations",
    )
    parser.add_argument(
        "--output",
        type=str,
        default=None,
        help="Save results to JSON file",
    )

    args = parser.parse_args()

    # Run benchmark
    results = run_benchmark(
        model_name=args.model,
        video_path=args.video,
        video_url=args.video_url,
        video_duration=args.duration,
        warmup_runs=args.warmup,
        quick=args.quick,
    )

    # Print results
    print_results(results)

    # Save if requested
    if args.output:
        save_results(results, args.output, args.model)

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

examples.video_benchmark.VideoBenchmarkResult · class

examples.video_benchmark.VideoBenchmarkResult(config_name: str, fps: float, max_frames: int, frames_extracted: int, video_duration: float, time_seconds: float, prompt_tokens: int, completion_tokens: int, tokens_per_second: float, response_preview: str)

Result from a single video benchmark run.

Parameters

Name	Type	Required	Default	Description
config_name	str	yes	none	Required constructor field.
fps	float	yes	none	Required constructor field.
max_frames	int	yes	none	Required constructor field.
frames_extracted	int	yes	none	Required constructor field.
video_duration	float	yes	none	Required constructor field.
time_seconds	float	yes	none	Required constructor field.
prompt_tokens	int	yes	none	Required constructor field.
completion_tokens	int	yes	none	Required constructor field.
tokens_per_second	float	yes	none	Required constructor field.
response_preview	str	yes	none	Required constructor field.

Returns

• Constructs: examples.video_benchmark.VideoBenchmarkResult

Exceptions and behavior

Class VideoBenchmarkResult declares 0 direct member(s). No direct raise statement appears in this definition.

View source #L80-L91.

examples.video_benchmark.create_test_video · function

examples.video_benchmark.create_test_video(duration: float = 5.0, fps: float = 30.0, width: int = 640, height: int = 480) -> str

Create a synthetic test video with colored frames and text.

Parameters

Name	Type	Required	Default	Description
duration	float	no	5.0	Video duration in seconds
fps	float	no	30.0	Frames per second
width	int	no	640	Video width
height	int	no	480	Video height

Returns

• Type: str
• Direct return expressions: temp_file.name

Exceptions and behavior

Function create_test_video calls tempfile.NamedTemporaryFile, temp_file.close, cv2.VideoWriter_fourcc, cv2.VideoWriter; returns temp_file.name. No direct raise statement appears in this definition.

View source #L94-L175.

examples.video_benchmark.download_video · function

examples.video_benchmark.download_video(url: str, timeout: int = 120) -> str

Download video from URL.

Parameters

Name	Type	Required	Default	Description
url	str	yes	none	Required positional or keyword input.
timeout	int	no	120	Optional positional or keyword input; defaults to 120.

Returns

• Type: str
• Direct return expressions: temp_file.name

Exceptions and behavior

Function download_video calls logger.info, requests.get, response.raise_for_status, tempfile.NamedTemporaryFile; returns temp_file.name. No direct raise statement appears in this definition.

View source #L178-L198.

examples.video_benchmark.get_video_info · function

examples.video_benchmark.get_video_info(video_path: str) -> dict

Get information about a video file.

Parameters

Name	Type	Required	Default	Description
video_path	str	yes	none	Required positional or keyword input.

Returns

• Type: dict
• Direct return expressions: {'error': 'Cannot open video'}; info

Exceptions and behavior

Function get_video_info calls cv2.VideoCapture, cap.isOpened, int, cap.get; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L201-L217.

examples.video_benchmark.run_video_benchmark · function

examples.video_benchmark.run_video_benchmark(model, video_path: str, fps: float, max_frames: int, config_name: str, warmup: bool = False) -> VideoBenchmarkResult

Run a single video benchmark configuration.

Parameters

Name	Type	Required	Default	Description
model	not annotated	yes	none	Required positional or keyword input.
video_path	str	yes	none	Required positional or keyword input.
fps	float	yes	none	Required positional or keyword input.
max_frames	int	yes	none	Required positional or keyword input.
config_name	str	yes	none	Required positional or keyword input.
warmup	bool	no	False	Optional positional or keyword input; defaults to False.

Returns

• Type: VideoBenchmarkResult
• Direct return expressions: VideoBenchmarkResult(config_name=config_name, fps=fps, max_frames=max_frames, frames_extracted=frames_extracted, video_…

Exceptions and behavior

Function run_video_benchmark calls get_video_info, print, time.perf_counter, model.generate; returns VideoBenchmarkResult(config_name=config_name, fps=fps, max_frames=max_frames, frames_extracted=frames_extracted, video_…. No direct raise statement appears in this definition.

View source #L220-L271.

examples.video_benchmark.run_benchmark · function

examples.video_benchmark.run_benchmark(model_name: str, video_path: str = None, video_url: str = None, video_duration: float = 10.0, warmup_runs: int = 1, quick: bool = False) -> list[VideoBenchmarkResult]

Run full video benchmark across multiple configurations.

Parameters

Name	Type	Required	Default	Description
model_name	str	yes	none	VLM model to use
video_path	str	no	None	Local video file path
video_url	str	no	None	URL to download video from
video_duration	float	no	10.0	Duration for synthetic video
warmup_runs	int	no	1	Number of warmup runs
quick	bool	no	False	Run quick benchmark with fewer configs

Returns

• Type: list[VideoBenchmarkResult]
• Direct return expressions: results

Exceptions and behavior

Function run_benchmark calls print, time.time, MLXVisionLanguageModel, model.load; returns results. No direct raise statement appears in this definition.

View source #L274-L374.

examples.video_benchmark.print_results · function

examples.video_benchmark.print_results(results: list[VideoBenchmarkResult]) -> not annotated

Print benchmark results in a nice table.

Parameters

Name	Type	Required	Default	Description
results	list[VideoBenchmarkResult]	yes	none	Required positional or keyword input.

Returns

• Type: not annotated
• Direct return expressions: None

Exceptions and behavior

Function print_results calls print, sorted, table_data.append, tabulate; returns None. No direct raise statement appears in this definition.

View source #L377-L445.

examples.video_benchmark.save_results · function

examples.video_benchmark.save_results(results: list[VideoBenchmarkResult], output_path: str, model_name: str) -> not annotated

Save benchmark results to JSON file.

Parameters

Name	Type	Required	Default	Description
results	list[VideoBenchmarkResult]	yes	none	Required positional or keyword input.
output_path	str	yes	none	Required positional or keyword input.
model_name	str	yes	none	Required positional or keyword input.

Returns

• Type: not annotated

Exceptions and behavior

Function save_results calls time.strftime, open, json.dump, print. No direct raise statement appears in this definition.

View source #L448-L474.

examples.video_benchmark.main · function

examples.video_benchmark.main() -> not annotated

Function main calls argparse.ArgumentParser, parser.add_argument, parser.parse_args, run_benchmark.

Parameters

This callable has no explicit inputs.

Returns

• Type: not annotated

Exceptions and behavior

Function main calls argparse.ArgumentParser, parser.add_argument, parser.parse_args, run_benchmark. No direct raise statement appears in this definition.

View source #L477-L559.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
VideoBenchmarkResult	class	VideoBenchmarkResult(config_name: str, fps: float, max_frames: int, frames_extracted: int, video_duration: float, time_seconds: float, prompt_tokens: int, completion_tokens: int, tokens_per_second: float, response_preview: str)	Result from a single video benchmark run.	#L80-L91
create_test_video	function	create_test_video(duration: float = 5.0, fps: float = 30.0, width: int = 640, height: int = 480) -> str	Create a synthetic test video with colored frames and text.	#L94-L175
download_video	function	download_video(url: str, timeout: int = 120) -> str	Download video from URL.	#L178-L198
get_video_info	function	get_video_info(video_path: str) -> dict	Get information about a video file.	#L201-L217
run_video_benchmark	function	run_video_benchmark(model, video_path: str, fps: float, max_frames: int, config_name: str, warmup: bool = False) -> VideoBenchmarkResult	Run a single video benchmark configuration.	#L220-L271
run_benchmark	function	run_benchmark(model_name: str, video_path: str = None, video_url: str = None, video_duration: float = 10.0, warmup_runs: int = 1, quick: bool = False) -> list[VideoBenchmarkResult]	Run full video benchmark across multiple configurations.	#L274-L374
print_results	function	print_results(results: list[VideoBenchmarkResult]) -> not annotated	Print benchmark results in a nice table.	#L377-L445
save_results	function	save_results(results: list[VideoBenchmarkResult], output_path: str, model_name: str) -> not annotated	Save benchmark results to JSON file.	#L448-L474
main	function	main() -> not annotated	Function main calls argparse.ArgumentParser, parser.add_argument, parser.parse_args, run_benchmark.	#L477-L559