图像基准测试

运行图像基准测试

# 完整基准测试（10 种分辨率）
vllm-mlx-bench --model mlx-community/Qwen3-VL-8B-Instruct-4bit

# 快速基准测试（4 种分辨率）
vllm-mlx-bench --model mlx-community/Qwen3-VL-8B-Instruct-4bit --quick

测试结果 - Qwen3-VL-8B-Instruct-4bit（M4 Max，128GB）

Resolution	Pixels	Time	Tokens	Speed
224x224	50K	1.04s	78	74.8 tok/s
336x336	113K	0.94s	64	68.3 tok/s
448x448	201K	1.45s	70	48.1 tok/s
512x512	262K	1.58s	99	62.8 tok/s
672x672	452K	1.83s	83	45.3 tok/s
768x768	590K	2.05s	91	44.3 tok/s
896x896	803K	2.61s	90	34.5 tok/s
1024x1024	1.0M	2.79s	76	27.2 tok/s
1280x720	922K	2.97s	96	32.4 tok/s
1920x1080	2.1M	6.30s	89	14.1 tok/s

摘要： 所有分辨率的平均速度为 45.2 tok/s。最快为 224x224（74.8 tok/s），最慢为 1920x1080（14.1 tok/s）。

测试结果 - Qwen3-VL-8B-Instruct-4bit（M1 Max，64GB）

本地 MLLM 基准测试：

Resolution	Pixels	Time	Tokens	Speed
224x224	50K	1.84s	78	42.5 tok/s
448x448	201K	2.28s	70	30.7 tok/s
768x768	590K	4.39s	91	20.7 tok/s
1024x1024	1.0M	6.41s	76	11.9 tok/s

Configs	Total Time (s)	Total Tokens	Aggregate Tok/s
4	14.92	315	21.1

测试结果 - Qwen3-VL-4B-Instruct-3bit 服务端（M1 Max，64GB）

Resolution	Pixels	Time	Tokens	Speed
224x224	50K	1.65s	113	68.4 tok/s
448x448	201K	2.09s	120	57.5 tok/s
768x768	590K	2.93s	106	36.2 tok/s
1024x1024	1.0M	4.12s	100	24.3 tok/s

Configs	Total Time (s)	Total Tokens	Aggregate Tok/s
4	10.79	439	40.7

MLLM 前缀缓存测试结果

======================================================================
  MLLM PREFIX CACHE TEST
======================================================================
  Model: mlx-community/Qwen3-VL-4B-Instruct-3bit
  Test: Verify KV cache reuse for repeated image/video + prompt combinations
  Expected behavior:
    - Same image + same prompt → cache HIT
    - Same image + different prompt → cache MISS
    - Different image + same prompt → cache MISS
----------------------------------------------------------------------
  SETUP: Loading Model
----------------------------------------------------------------------
    Model loaded in 0.11s

----------------------------------------------------------------------
  SETUP: Creating Test Images
----------------------------------------------------------------------
    Resized: 224x224, 336x336, 512x512, 768x768

----------------------------------------------------------------------
  TEST 1: Image Cache - Basic Hit/Miss
----------------------------------------------------------------------
    Results:
    Step   | Description               | Expected | Actual | Time   | Status
    -------+---------------------------+----------+--------+--------+-------
    1a     | First image+prompt        | MISS     | MISS   | 0.10ms | ✓
    1b     | Same image+prompt         | HIT      | HIT    | 0.18ms | ✓
    1c     | Different prompt          | MISS     | MISS   | 0.01ms | ✓
    1d     | Return to original        | HIT      | HIT    | 0.18ms | ✓

----------------------------------------------------------------------
  TEST 2: Different Images
----------------------------------------------------------------------
    Results:
    Step   | Description               | Expected | Actual | Time   | Status
    -------+---------------------------+----------+--------+--------+-------
    2a     | Image A first request     | MISS     | MISS   | 0.01ms | ✓
    2b     | Image B first request     | MISS     | MISS   | 0.01ms | ✓
    2c     | Image A cached            | HIT      | HIT    | 0.13ms | ✓

----------------------------------------------------------------------
  TEST 3: Image Resolutions
----------------------------------------------------------------------
    Results:
    Step   | Description           | Expected | Actual | Time   | Status
    -------+-----------------------+----------+--------+--------+-------
    3.1a | 224x224 first         | MISS     | MISS   | 0.01ms | ✓
    3.1b | 224x224 cached        | HIT      | HIT    | 0.20ms | ✓
    3.2a | 336x336 first         | MISS     | MISS   | 0.01ms | ✓
    3.2b | 336x336 cached        | HIT      | HIT    | 0.21ms | ✓
    3.3a | 512x512 first         | MISS     | MISS   | 0.12ms | ✓
    3.3b | 512x512 cached        | HIT      | HIT    | 0.20ms | ✓
    3.4a | 768x768 first         | MISS     | MISS   | 0.12ms | ✓
    3.4b | 768x768 cached        | HIT      | HIT    | 0.24ms | ✓
======================================================================

缓存键策略

• 图像： hash(image_content) + hash(prompt)

相同图像与相同提示词始终命中缓存。不同图像或不同提示词将不命中缓存。

性能提示

• 分辨率越小，处理速度越快（如 224x224 对比 1920x1080）
• 请根据任务需求选择合适的分辨率
• 批量处理尺寸相近的图像，以获得稳定的吞吐量

指标说明

Metric	Description
Resolution	图像尺寸（宽 x 高）
Pixels	总像素数
Time	生成时间
Tokens	生成的输出 token 数量
Speed	每秒 token 数（tok/s）