examples.mllm_example

Multimodal Language Model (MLLM) example using vllm-mlx.

View the complete module source at #L1-L89.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

examples.mllm_example

Multimodal Language Model (MLLM) example using vllm-mlx.

This example demonstrates multimodal inference on Apple Silicon, including image understanding and visual question answering.

examples.mllm_example.main

main()

Source code in examples/mllm_example.py

def main():
    # Use a quantized multimodal language model
    model_name = "mlx-community/Qwen2-VL-2B-Instruct-4bit"

    print(f"Loading MLLM: {model_name}")
    mllm = MLXMultimodalLM(model_name)
    mllm.load()

    print("\n" + "=" * 50)
    print("Multimodal Language Model loaded!")
    print("=" * 50 + "\n")

    # Check for image argument
    if len(sys.argv) < 2:
        print("Usage: python mllm_example.py <image_path>")
        print("\nNo image provided. Demonstrating with text-only mode.\n")

        # Text-only generation (MLLMs can also do this)
        output = mllm.generate(
            prompt="What is the capital of Japan?",
            max_tokens=100,
        )
        print(f"Q: What is the capital of Japan?")
        print(f"A: {output.text}")
        return

    image_path = sys.argv[1]

    if not Path(image_path).exists():
        print(f"Error: Image not found: {image_path}")
        sys.exit(1)

    print(f"Using image: {image_path}\n")

    # Example 1: Describe the image
    print("=" * 50)
    print("Example 1: Image Description")
    print("=" * 50 + "\n")

    description = mllm.describe_image(image_path, max_tokens=300)
    print(f"Description:\n{description}\n")

    # Example 2: Visual Question Answering
    print("=" * 50)
    print("Example 2: Visual Question Answering")
    print("=" * 50 + "\n")

    questions = [
        "What objects can you see in this image?",
        "What colors are dominant in this image?",
        "Is there any text visible in the image?",
    ]

    for question in questions:
        answer = mllm.answer_about_image(image_path, question, max_tokens=150)
        print(f"Q: {question}")
        print(f"A: {answer}\n")

    # Example 3: Custom prompt with image
    print("=" * 50)
    print("Example 3: Custom Analysis")
    print("=" * 50 + "\n")

    output = mllm.generate(
        prompt="Analyze this image and provide a creative story inspired by what you see.",
        images=[image_path],
        max_tokens=400,
        temperature=0.9,
    )
    print(f"Creative Story:\n{output.text}")

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

examples.mllm_example.main · function

examples.mllm_example.main() -> not annotated

Function main calls print, MLXMultimodalLM, mllm.load, len; returns None.

Parameters

This callable has no explicit inputs.

Returns

• Type: not annotated
• Direct return expressions: None

Exceptions and behavior

Function main calls print, MLXMultimodalLM, mllm.load, len; returns None. No direct raise statement appears in this definition.

View source #L16-L85.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
main	function	main() -> not annotated	Function main calls print, MLXMultimodalLM, mllm.load, len; returns None.	#L16-L85