Skip to content

vllm_mlx.patches.qwen3_5_mllm

Runtime patch for mlx-vlm's Qwen3.5 attention to support BatchKVCache.

View the complete module source at #L1-L266.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.patches.qwen3_5_mllm

Runtime patch for mlx-vlm's Qwen3.5 attention to support BatchKVCache.

Qwen 3.6 artifacts use the mlx-vlm Qwen3.5 language module in this stack. The attention patch therefore lives in the Qwen3.5 compatibility module while serving Qwen 3.6 27B/35B/122B artifacts.

mlx-vlm's Qwen3_5Attention uses cache.offset directly for kv_seq_len computation and mask slicing. BatchKVCache stores offset as mx.array (per-batch-item), not int, causing:

mask = mask[..., :kv_seq_len]
ValueError: Slice indices must be integers or None.

This patch replaces Qwen3_5Attention.call with a version that converts cache.offset to int before using it for arithmetic/slicing, while leaving the actual cache.offset untouched so update_and_fetch still works correctly with per-batch offsets.

vllm_mlx.patches.qwen3_5_mllm.logger module-attribute 

logger = logging.getLogger(__name__)

vllm_mlx.patches.qwen3_5_mllm._cache_offset_to_int 

_cache_offset_to_int(cache) -> int

Extract cache offset as int, handling BatchKVCache mx.array offset.

Source code in vllm_mlx/patches/qwen3_5_mllm.py 
33
34
35
36
37
38
39
40
41
42
def _cache_offset_to_int(cache) -> int:
    """Extract cache offset as int, handling BatchKVCache mx.array offset."""
    if cache is None:
        return 0
    off = cache.offset
    if isinstance(off, int):
        return off
    if isinstance(off, mx.array):
        return int(off.max().item()) if off.ndim > 0 else int(off.item())
    return int(off)

vllm_mlx.patches.qwen3_5_mllm._default_target_verify_linears 

_default_target_verify_linears(linears, x, target_verify: bool)
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
45
46
def _default_target_verify_linears(linears, x, target_verify: bool):
    return tuple(linear(x) for linear in linears)

vllm_mlx.patches.qwen3_5_mllm._default_target_verify_left_padded_attention 

_default_target_verify_left_padded_attention(*args, **kwargs)
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
49
50
def _default_target_verify_left_padded_attention(*args, **kwargs):
    return None

vllm_mlx.patches.qwen3_5_mllm._normalize_position_inputs 

_normalize_position_inputs(position_ids: Optional[array], position_embeddings: Optional[tuple[array, array]], length: int) -> tuple[Optional[array], Optional[tuple[array, array]]]
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
def _normalize_position_inputs(
    position_ids: Optional[mx.array],
    position_embeddings: Optional[tuple[mx.array, mx.array]],
    length: int,
) -> tuple[Optional[mx.array], Optional[tuple[mx.array, mx.array]]]:
    if position_ids is None or position_ids.shape[-1] == length:
        return position_ids, position_embeddings
    logger.debug(
        "[Qwen3.5 patch] Recomputing stale position_ids: got %s, expected %s",
        position_ids.shape[-1],
        length,
    )
    return None, None

vllm_mlx.patches.qwen3_5_mllm._position_ids_for_offset 

_position_ids_for_offset(offset: int, length: int) -> array
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
68
69
70
71
def _position_ids_for_offset(offset: int, length: int) -> mx.array:
    position_ids = mx.arange(offset, offset + length)
    position_ids = mx.expand_dims(position_ids, axis=0)
    return mx.tile(position_ids, (3, 1, 1))

vllm_mlx.patches.qwen3_5_mllm._kv_seq_len 

_kv_seq_len(keys: array, cache: Optional[Any], offset: int) -> int
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
74
75
76
def _kv_seq_len(keys: mx.array, cache: Optional[Any], offset: int) -> int:
    length = keys.shape[-2]
    return length + offset + 1 if cache is not None else length

vllm_mlx.patches.qwen3_5_mllm._apply_rotary 

_apply_rotary(attention, queries: array, keys: array, values: array, position_ids: array, position_embeddings: Optional[tuple[array, array]], apply_multimodal_rotary_pos_emb) -> tuple[array, array]
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def _apply_rotary(
    attention,
    queries: mx.array,
    keys: mx.array,
    values: mx.array,
    position_ids: mx.array,
    position_embeddings: Optional[tuple[mx.array, mx.array]],
    apply_multimodal_rotary_pos_emb,
) -> tuple[mx.array, mx.array]:
    if position_embeddings is not None:
        cos, sin = position_embeddings
        return apply_multimodal_rotary_pos_emb(queries, keys, cos, sin)

    if hasattr(attention.rotary_emb, "apply_rotary"):
        return attention.rotary_emb.apply_rotary(
            queries,
            keys,
            position_ids,
            unsqueeze_dim=1,
        )

    cos, sin = attention.rotary_emb(values, position_ids)
    return apply_multimodal_rotary_pos_emb(queries, keys, cos, sin)

vllm_mlx.patches.qwen3_5_mllm._slice_attention_mask 

_slice_attention_mask(mask: Optional[array], cache: Optional[Any], kv_seq_len: int, length: int) -> Optional[array]
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
104
105
106
107
108
109
110
111
112
113
114
115
116
def _slice_attention_mask(
    mask: Optional[mx.array],
    cache: Optional[Any],
    kv_seq_len: int,
    length: int,
) -> Optional[mx.array]:
    if mask is None or not isinstance(mask, mx.array):
        return mask
    if cache is not None and hasattr(cache, "_idx") and hasattr(cache, "left_padding"):
        kv_seq_len = int(cache._idx) + length
    elif isinstance(kv_seq_len, mx.array):
        kv_seq_len = int(kv_seq_len.max().item())
    return mask[..., : int(kv_seq_len)]

vllm_mlx.patches.qwen3_5_mllm._maybe_target_verify_attention 

_maybe_target_verify_attention(queries: array, keys: array, values: array, *, cache: Optional[Any], mask: Optional[array], scale: float, target_verify: bool, length: int, left_padded_decode: bool, target_verify_left_padded_attention) -> Optional[array]
Source code in vllm_mlx/patches/qwen3_5_mllm.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
def _maybe_target_verify_attention(
    queries: mx.array,
    keys: mx.array,
    values: mx.array,
    *,
    cache: Optional[Any],
    mask: Optional[mx.array],
    scale: float,
    target_verify: bool,
    length: int,
    left_padded_decode: bool,
    target_verify_left_padded_attention,
) -> Optional[mx.array]:
    if not ((target_verify and length > 1) or left_padded_decode):
        return None
    return target_verify_left_padded_attention(
        queries,
        keys,
        values,
        cache=cache,
        scale=scale,
        mask=mask,
    )

vllm_mlx.patches.qwen3_5_mllm.patch_qwen35_attention_for_batching 

patch_qwen35_attention_for_batching() -> bool

Monkey-patch Qwen3_5Attention.call to handle BatchKVCache.

Returns True if patch was applied, False if mlx-vlm is not installed or Qwen3.5 module not available.

Source code in vllm_mlx/patches/qwen3_5_mllm.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
def patch_qwen35_attention_for_batching() -> bool:
    """Monkey-patch Qwen3_5Attention.__call__ to handle BatchKVCache.

    Returns True if patch was applied, False if mlx-vlm is not installed
    or Qwen3.5 module not available.
    """
    try:
        qwen35_language = importlib.import_module("mlx_vlm.models.qwen3_5.language")
        from mlx_lm.models.base import scaled_dot_product_attention
    except ImportError:
        logger.debug("[Qwen3.5 patch] mlx-vlm Qwen3.5 module not available")
        return False

    Qwen3_5Attention = qwen35_language.Qwen3_5Attention
    apply_multimodal_rotary_pos_emb = qwen35_language.apply_multimodal_rotary_pos_emb
    target_verify_linears = getattr(
        qwen35_language,
        "_target_verify_linears",
        _default_target_verify_linears,
    )
    target_verify_left_padded_attention = getattr(
        qwen35_language,
        "_target_verify_left_padded_attention",
        _default_target_verify_left_padded_attention,
    )

    if getattr(Qwen3_5Attention, "_batch_patched", False):
        logger.debug("[Qwen3.5 patch] Already patched")
        return True

    def _patched_call(
        self,
        x: mx.array,
        mask: Optional[mx.array] = None,
        cache: Optional[Any] = None,
        position_ids: Optional[mx.array] = None,
        position_embeddings: Optional[tuple[mx.array, mx.array]] = None,
        target_verify: bool = False,
    ) -> mx.array:
        B, L, _D = x.shape

        q_proj_output, keys, values = target_verify_linears(
            (self.q_proj, self.k_proj, self.v_proj),
            x,
            target_verify,
        )
        queries, gate = mx.split(
            q_proj_output.reshape(B, L, self.num_attention_heads, -1),
            2,
            axis=-1,
        )
        gate = gate.reshape(B, L, -1)

        queries = self.q_norm(queries).transpose(0, 2, 1, 3)
        keys = self.k_norm(keys.reshape(B, L, self.num_key_value_heads, -1)).transpose(
            0, 2, 1, 3
        )
        values = values.reshape(B, L, self.num_key_value_heads, -1).transpose(
            0, 2, 1, 3
        )

        offset = _cache_offset_to_int(cache)
        position_ids, position_embeddings = _normalize_position_inputs(
            position_ids,
            position_embeddings,
            L,
        )

        if position_ids is None:
            position_ids = _position_ids_for_offset(offset, L)
        kv_seq_len = _kv_seq_len(keys, cache, offset)

        queries, keys = _apply_rotary(
            self,
            queries,
            keys,
            values,
            position_ids,
            position_embeddings,
            apply_multimodal_rotary_pos_emb,
        )

        mask = _slice_attention_mask(mask, cache, kv_seq_len, L)

        if cache is not None:
            keys, values = cache.update_and_fetch(keys, values)

        left_padded_decode = (
            mask == "left_padded_decode" if isinstance(mask, str) else False
        )
        if left_padded_decode:
            mask = None

        output = _maybe_target_verify_attention(
            queries,
            keys,
            values,
            cache=cache,
            mask=mask,
            scale=self.scale,
            target_verify=target_verify,
            length=L,
            left_padded_decode=left_padded_decode,
            target_verify_left_padded_attention=target_verify_left_padded_attention,
        )

        if output is None:
            output = scaled_dot_product_attention(
                queries,
                keys,
                values,
                cache=cache,
                scale=self.scale,
                mask=mask,
            )
        output = output.transpose(0, 2, 1, 3).reshape(B, L, -1)

        return self.o_proj(output * mx.sigmoid(gate))

    Qwen3_5Attention.__call__ = _patched_call
    setattr(Qwen3_5Attention, "_batch_patched", True)
    logger.info("[Qwen3.5 patch] Attention patched for BatchKVCache support")
    return True

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.patches.qwen3_5_mllm._cache_offset_to_int · function 
vllm_mlx.patches.qwen3_5_mllm._cache_offset_to_int(cache) -> int

Extract cache offset as int, handling BatchKVCache mx.array offset.

Parameters

Name Type Required Default Description
cache not annotated yes none Required positional or keyword input.

Returns

  • Type: int
  • Direct return expressions: 0; off; int(off.max().item()) if off.ndim > 0 else int(off.item()); int(off)

Exceptions and behavior

Function _cache_offset_to_int calls isinstance, int, off.max().item, off.max; has 4 explicit return paths. No direct raise statement appears in this definition.

View source #L33-L42.

vllm_mlx.patches.qwen3_5_mllm._default_target_verify_linears · function 
vllm_mlx.patches.qwen3_5_mllm._default_target_verify_linears(linears, x, target_verify: bool) -> not annotated

Function _default_target_verify_linears calls tuple, linear; returns tuple((linear(x) for linear in linears)).

Parameters

Name Type Required Default Description
linears not annotated yes none Required positional or keyword input.
x not annotated yes none Required positional or keyword input.
target_verify bool yes none Required positional or keyword input.

Returns

  • Type: not annotated
  • Direct return expressions: tuple((linear(x) for linear in linears))

Exceptions and behavior

Function _default_target_verify_linears calls tuple, linear; returns tuple((linear(x) for linear in linears)). No direct raise statement appears in this definition.

View source #L45-L46.

vllm_mlx.patches.qwen3_5_mllm._default_target_verify_left_padded_attention · function 
vllm_mlx.patches.qwen3_5_mllm._default_target_verify_left_padded_attention(*args, **kwargs) -> not annotated

Function _default_target_verify_left_padded_attention returns None.

Parameters

Name Type Required Default Description
*args not annotated no none Additional variadic positional inputs accepted by this callable.
**kwargs not annotated no none Additional variadic keyword inputs accepted by this callable.

Returns

  • Type: not annotated
  • Direct return expressions: None

Exceptions and behavior

Function _default_target_verify_left_padded_attention returns None. No direct raise statement appears in this definition.

View source #L49-L50.

vllm_mlx.patches.qwen3_5_mllm._normalize_position_inputs · function 
vllm_mlx.patches.qwen3_5_mllm._normalize_position_inputs(position_ids: Optional[mx.array], position_embeddings: Optional[tuple[mx.array, mx.array]], length: int) -> tuple[Optional[mx.array], Optional[tuple[mx.array, mx.array]]]

Function _normalize_position_inputs calls logger.debug; has 2 explicit return paths.

Parameters

Name Type Required Default Description
position_ids Optional[mx.array] yes none Required positional or keyword input.
position_embeddings Optional[tuple[mx.array, mx.array]] yes none Required positional or keyword input.
length int yes none Required positional or keyword input.

Returns

  • Type: tuple[Optional[mx.array], Optional[tuple[mx.array, mx.array]]]
  • Direct return expressions: (position_ids, position_embeddings); (None, None)

Exceptions and behavior

Function _normalize_position_inputs calls logger.debug; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L53-L65.

vllm_mlx.patches.qwen3_5_mllm._position_ids_for_offset · function 
vllm_mlx.patches.qwen3_5_mllm._position_ids_for_offset(offset: int, length: int) -> mx.array

Function _position_ids_for_offset calls mx.arange, mx.expand_dims, mx.tile; returns mx.tile(position_ids, (3, 1, 1)).

Parameters

Name Type Required Default Description
offset int yes none Required positional or keyword input.
length int yes none Required positional or keyword input.

Returns

  • Type: mx.array
  • Direct return expressions: mx.tile(position_ids, (3, 1, 1))

Exceptions and behavior

Function _position_ids_for_offset calls mx.arange, mx.expand_dims, mx.tile; returns mx.tile(position_ids, (3, 1, 1)). No direct raise statement appears in this definition.

View source #L68-L71.

vllm_mlx.patches.qwen3_5_mllm._kv_seq_len · function 
vllm_mlx.patches.qwen3_5_mllm._kv_seq_len(keys: mx.array, cache: Optional[Any], offset: int) -> int

Function _kv_seq_len returns length + offset + 1 if cache is not None else length.

Parameters

Name Type Required Default Description
keys mx.array yes none Required positional or keyword input.
cache Optional[Any] yes none Required positional or keyword input.
offset int yes none Required positional or keyword input.

Returns

  • Type: int
  • Direct return expressions: length + offset + 1 if cache is not None else length

Exceptions and behavior

Function _kv_seq_len returns length + offset + 1 if cache is not None else length. No direct raise statement appears in this definition.

View source #L74-L76.

vllm_mlx.patches.qwen3_5_mllm._apply_rotary · function 
vllm_mlx.patches.qwen3_5_mllm._apply_rotary(attention, queries: mx.array, keys: mx.array, values: mx.array, position_ids: mx.array, position_embeddings: Optional[tuple[mx.array, mx.array]], apply_multimodal_rotary_pos_emb) -> tuple[mx.array, mx.array]

Function _apply_rotary calls apply_multimodal_rotary_pos_emb, hasattr, attention.rotary_emb.apply_rotary, attention.rotary_emb; has 2 explicit return paths.

Parameters

Name Type Required Default Description
attention not annotated yes none Required positional or keyword input.
queries mx.array yes none Required positional or keyword input.
keys mx.array yes none Required positional or keyword input.
values mx.array yes none Required positional or keyword input.
position_ids mx.array yes none Required positional or keyword input.
position_embeddings Optional[tuple[mx.array, mx.array]] yes none Required positional or keyword input.
apply_multimodal_rotary_pos_emb not annotated yes none Required positional or keyword input.

Returns

  • Type: tuple[mx.array, mx.array]
  • Direct return expressions: apply_multimodal_rotary_pos_emb(queries, keys, cos, sin); attention.rotary_emb.apply_rotary(queries, keys, position_ids, unsqueeze_dim=1)

Exceptions and behavior

Function _apply_rotary calls apply_multimodal_rotary_pos_emb, hasattr, attention.rotary_emb.apply_rotary, attention.rotary_emb; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L79-L101.

vllm_mlx.patches.qwen3_5_mllm._slice_attention_mask · function 
vllm_mlx.patches.qwen3_5_mllm._slice_attention_mask(mask: Optional[mx.array], cache: Optional[Any], kv_seq_len: int, length: int) -> Optional[mx.array]

Function _slice_attention_mask calls isinstance, hasattr, int, kv_seq_len.max().item; has 2 explicit return paths.

Parameters

Name Type Required Default Description
mask Optional[mx.array] yes none Required positional or keyword input.
cache Optional[Any] yes none Required positional or keyword input.
kv_seq_len int yes none Required positional or keyword input.
length int yes none Required positional or keyword input.

Returns

  • Type: Optional[mx.array]
  • Direct return expressions: mask; mask[..., :int(kv_seq_len)]

Exceptions and behavior

Function _slice_attention_mask calls isinstance, hasattr, int, kv_seq_len.max().item; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L104-L116.

vllm_mlx.patches.qwen3_5_mllm._maybe_target_verify_attention · function 
vllm_mlx.patches.qwen3_5_mllm._maybe_target_verify_attention(queries: mx.array, keys: mx.array, values: mx.array, *, cache: Optional[Any], mask: Optional[mx.array], scale: float, target_verify: bool, length: int, left_padded_decode: bool, target_verify_left_padded_attention) -> Optional[mx.array]

Function _maybe_target_verify_attention calls target_verify_left_padded_attention; has 2 explicit return paths.

Parameters

Name Type Required Default Description
queries mx.array yes none Required positional or keyword input.
keys mx.array yes none Required positional or keyword input.
values mx.array yes none Required positional or keyword input.
cache Optional[Any] yes none Required keyword-only input.
mask Optional[mx.array] yes none Required keyword-only input.
scale float yes none Required keyword-only input.
target_verify bool yes none Required keyword-only input.
length int yes none Required keyword-only input.
left_padded_decode bool yes none Required keyword-only input.
target_verify_left_padded_attention not annotated yes none Required keyword-only input.

Returns

  • Type: Optional[mx.array]
  • Direct return expressions: None; target_verify_left_padded_attention(queries, keys, values, cache=cache, scale=scale, mask=mask)

Exceptions and behavior

Function _maybe_target_verify_attention calls target_verify_left_padded_attention; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L119-L141.

vllm_mlx.patches.qwen3_5_mllm.patch_qwen35_attention_for_batching · function 
vllm_mlx.patches.qwen3_5_mllm.patch_qwen35_attention_for_batching() -> bool

Monkey-patch Qwen3_5Attention.call to handle BatchKVCache.

Parameters

This callable has no explicit inputs.

Returns

  • Type: bool
  • Direct return expressions: False; True

Exceptions and behavior

Function patch_qwen35_attention_for_batching calls importlib.import_module, logger.debug, getattr, setattr; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L144-L266.

vllm_mlx.patches.qwen3_5_mllm.patch_qwen35_attention_for_batching._patched_call · nested function 
vllm_mlx.patches.qwen3_5_mllm.patch_qwen35_attention_for_batching._patched_call(x: mx.array, mask: Optional[mx.array] = None, cache: Optional[Any] = None, position_ids: Optional[mx.array] = None, position_embeddings: Optional[tuple[mx.array, mx.array]] = None, target_verify: bool = False) -> mx.array

Nested Function patch_qwen35_attention_for_batching._patched_call calls target_verify_linears, mx.split, q_proj_output.reshape, gate.reshape; returns self.o_proj(output * mx.sigmoid(gate)).

Parameters

Name Type Required Default Description
x mx.array yes none Required positional or keyword input.
mask Optional[mx.array] no None Optional positional or keyword input; defaults to None.
cache Optional[Any] no None Optional positional or keyword input; defaults to None.
position_ids Optional[mx.array] no None Optional positional or keyword input; defaults to None.
position_embeddings Optional[tuple[mx.array, mx.array]] no None Optional positional or keyword input; defaults to None.
target_verify bool no False Optional positional or keyword input; defaults to False.

Returns

  • Type: mx.array
  • Direct return expressions: self.o_proj(output * mx.sigmoid(gate))

Exceptions and behavior

Nested Function patch_qwen35_attention_for_batching._patched_call calls target_verify_linears, mx.split, q_proj_output.reshape, gate.reshape; returns self.o_proj(output * mx.sigmoid(gate)). No direct raise statement appears in this definition.

View source #L174-L261.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
_cache_offset_to_int function _cache_offset_to_int(cache) -> int Extract cache offset as int, handling BatchKVCache mx.array offset. #L33-L42
_default_target_verify_linears function _default_target_verify_linears(linears, x, target_verify: bool) -> not annotated Function _default_target_verify_linears calls tuple, linear; returns tuple((linear(x) for linear in linears)). #L45-L46
_default_target_verify_left_padded_attention function _default_target_verify_left_padded_attention(*args, **kwargs) -> not annotated Function _default_target_verify_left_padded_attention returns None. #L49-L50
_normalize_position_inputs function _normalize_position_inputs(position_ids: Optional[mx.array], position_embeddings: Optional[tuple[mx.array, mx.array]], length: int) -> tuple[Optional[mx.array], Optional[tuple[mx.array, mx.array]]] Function _normalize_position_inputs calls logger.debug; has 2 explicit return paths. #L53-L65
_position_ids_for_offset function _position_ids_for_offset(offset: int, length: int) -> mx.array Function _position_ids_for_offset calls mx.arange, mx.expand_dims, mx.tile; returns mx.tile(position_ids, (3, 1, 1)). #L68-L71
_kv_seq_len function _kv_seq_len(keys: mx.array, cache: Optional[Any], offset: int) -> int Function _kv_seq_len returns length + offset + 1 if cache is not None else length. #L74-L76
_apply_rotary function _apply_rotary(attention, queries: mx.array, keys: mx.array, values: mx.array, position_ids: mx.array, position_embeddings: Optional[tuple[mx.array, mx.array]], apply_multimodal_rotary_pos_emb) -> tuple[mx.array, mx.array] Function _apply_rotary calls apply_multimodal_rotary_pos_emb, hasattr, attention.rotary_emb.apply_rotary, attention.rotary_emb; has 2 explicit return paths. #L79-L101
_slice_attention_mask function _slice_attention_mask(mask: Optional[mx.array], cache: Optional[Any], kv_seq_len: int, length: int) -> Optional[mx.array] Function _slice_attention_mask calls isinstance, hasattr, int, kv_seq_len.max().item; has 2 explicit return paths. #L104-L116
_maybe_target_verify_attention function _maybe_target_verify_attention(queries: mx.array, keys: mx.array, values: mx.array, *, cache: Optional[Any], mask: Optional[mx.array], scale: float, target_verify: bool, length: int, left_padded_decode: bool, target_verify_left_padded_attention) -> Optional[mx.array] Function _maybe_target_verify_attention calls target_verify_left_padded_attention; has 2 explicit return paths. #L119-L141
patch_qwen35_attention_for_batching function patch_qwen35_attention_for_batching() -> bool Monkey-patch Qwen3_5Attention.call to handle BatchKVCache. #L144-L266
patch_qwen35_attention_for_batching._patched_call nested function patch_qwen35_attention_for_batching._patched_call(x: mx.array, mask: Optional[mx.array] = None, cache: Optional[Any] = None, position_ids: Optional[mx.array] = None, position_embeddings: Optional[tuple[mx.array, mx.array]] = None, target_verify: bool = False) -> mx.array Nested Function patch_qwen35_attention_for_batching._patched_call calls target_verify_linears, mx.split, q_proj_output.reshape, gate.reshape; returns self.o_proj(output * mx.sigmoid(gate)). #L174-L261