vllm_mlx.tool_parsers.qwen_tool_parser

Qwen tool call parser for vllm-mlx.

View the complete module source at #L1-L351.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.tool_parsers.qwen_tool_parser

Qwen tool call parser for vllm-mlx.

Handles Qwen's tool calling formats: - XML style: {"name": "func", "arguments": {...}} - Bracket style: [Calling tool: func_name({"arg": "value"})] - Function style: value

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser

QwenToolParser(tokenizer: PreTrainedTokenizerBase | None = None)

Bases: ToolParser

Tool call parser for Qwen models.

Supports multiple Qwen tool call formats: - XML: {"name": "func", "arguments": {...}} - Bracket: [Calling tool: func_name({"arg": "value"})] - Function: value

Used when --enable-auto-tool-choice --tool-call-parser qwen are set.

Source code in vllm_mlx/tool_parsers/abstract_tool_parser.py

def __init__(self, tokenizer: PreTrainedTokenizerBase | None = None):
    """
    Initialize the tool parser.

    Args:
        tokenizer: The tokenizer for the model (optional, some parsers need it)
    """
    self.model_tokenizer = tokenizer
    # State for streaming parsing
    self.current_tool_id: int = -1
    self.prev_tool_call_arr: list[dict] = []

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.SUPPORTS_NATIVE_TOOL_FORMAT class-attribute instance-attribute

SUPPORTS_NATIVE_TOOL_FORMAT = True

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.XML_PATTERN class-attribute instance-attribute

XML_PATTERN = re.compile('<tool_call>\\s*(\\{.*?\\})\\s*</tool_call>', re.DOTALL)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.BRACKET_PATTERN class-attribute instance-attribute

BRACKET_PATTERN = re.compile('\\[Calling tool:\\s*(\\w+)\\((\\{.*?\\})\\)\\]', re.DOTALL)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.FUNCTION_PATTERN class-attribute instance-attribute

FUNCTION_PATTERN = re.compile('<function=([^>]+)>(.*?)</function>', re.DOTALL)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.PARAM_PATTERN class-attribute instance-attribute

PARAM_PATTERN = re.compile('<parameter=([^>]+)>\\s*(.*?)\\s*</parameter>', re.DOTALL)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.EMPTY_TOOL_CALL class-attribute instance-attribute

EMPTY_TOOL_CALL = re.compile('<tool_call>\\s*</tool_call>', re.DOTALL)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._PARTIAL_MARKERS class-attribute instance-attribute

_PARTIAL_MARKERS = ('<function', '[Calling tool', '<tool_call')

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls

extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Qwen model response.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def extract_tool_calls(
    self, model_output: str, request: dict[str, Any] | None = None
) -> ExtractedToolCallInformation:
    """
    Extract tool calls from a complete Qwen model response.
    """
    tool_calls = []

    # Strip <think> tags first (fallback when no reasoning parser)
    cleaned_text = self.strip_think_tags(model_output)

    # Try bracket pattern first (Qwen3 style)
    bracket_matches = self.BRACKET_PATTERN.findall(cleaned_text)
    for name, args_str in bracket_matches:
        try:
            arguments = json.loads(args_str)
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": name.strip(),
                    "arguments": (
                        json.dumps(arguments, ensure_ascii=False)
                        if isinstance(arguments, dict)
                        else str(arguments)
                    ),
                }
            )
        except json.JSONDecodeError:
            continue

    if bracket_matches:
        cleaned_text = self.BRACKET_PATTERN.sub("", cleaned_text).strip()

    # Try XML pattern (traditional Qwen style)
    xml_matches = self.XML_PATTERN.findall(cleaned_text)
    for match in xml_matches:
        try:
            data = json.loads(match)
            name = data.get("name", "")
            arguments = data.get("arguments", {})
            if name:
                tool_calls.append(
                    {
                        "id": generate_tool_id(),
                        "name": name,
                        "arguments": (
                            json.dumps(arguments, ensure_ascii=False)
                            if isinstance(arguments, dict)
                            else str(arguments)
                        ),
                    }
                )
        except json.JSONDecodeError:
            continue

    if xml_matches:
        cleaned_text = self.XML_PATTERN.sub("", cleaned_text).strip()

    # Try function-style: <function=name><parameter=key>value</parameter></function>
    # Qwen3.5/3.6 emit this format natively, sometimes wrapped in <tool_call>...
    # </tool_call>. Always run this pass — earlier guards skipped it when an
    # XML/bracket call had already been extracted, leaking the function-style
    # markup of subsequent tool calls into `content` (multi-format mixed
    # responses).
    func_matches = self.FUNCTION_PATTERN.findall(cleaned_text)
    for name, params_block in func_matches:
        # Try JSON arguments first (e.g. <function=name>{"key": "val"}</function>)
        params_block_stripped = params_block.strip()
        if params_block_stripped.startswith("{"):
            try:
                arguments = json.loads(params_block_stripped)
                tool_calls.append(
                    {
                        "id": generate_tool_id(),
                        "name": name.strip(),
                        "arguments": json.dumps(arguments, ensure_ascii=False),
                    }
                )
                continue
            except json.JSONDecodeError:
                pass
        # Parse <parameter=key>value</parameter> tags
        params = self.PARAM_PATTERN.findall(params_block)
        arguments = {}
        for p_name, p_value in params:
            arguments[p_name.strip()] = _parse_param_value(p_value.strip())
        tool_calls.append(
            {
                "id": generate_tool_id(),
                "name": name.strip(),
                "arguments": json.dumps(arguments, ensure_ascii=False),
            }
        )
    if func_matches:
        cleaned_text = self.FUNCTION_PATTERN.sub("", cleaned_text).strip()

    if tool_calls:
        # Clean up empty <tool_call> wrappers left after function extraction
        cleaned_text = self.EMPTY_TOOL_CALL.sub("", cleaned_text).strip()
        # Strip any trailing unclosed tool-call markup left when generation
        # was truncated mid-call (finish_reason="length" hitting max_tokens).
        cleaned_text = self._strip_unclosed_markup(cleaned_text)
        return ExtractedToolCallInformation(
            tools_called=True,
            tool_calls=tool_calls,
            content=cleaned_text if cleaned_text else None,
        )
    else:
        # No complete tool calls were extracted. If the output ends with a
        # truncated tool-call marker (e.g. max_tokens hit mid <tool_call>...),
        # strip it so callers don't see raw markup in `content`. We only
        # strip the trailing partial — earlier complete sentences stay.
        # Always return a string (empty if fully stripped) so the caller
        # can distinguish "parser processed this" from "parser declined";
        # `or None` would conflate stripped-to-empty with no-result and let
        # a fallback path resurface the raw markup.
        stripped = self._strip_unclosed_markup(model_output)
        return ExtractedToolCallInformation(
            tools_called=False, tool_calls=[], content=stripped
        )

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._strip_unclosed_markup staticmethod

_strip_unclosed_markup(text: str) -> str

Strip a trailing unclosed tool-call marker (truncated output).

When generation hits max_tokens mid-tool-call, a partial <tool_call>/<function=/[Calling tool: marker remains in the cleaned text. We locate the earliest such unclosed marker and drop everything from there to the end so the API response never carries raw markup.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

@staticmethod
def _strip_unclosed_markup(text: str) -> str:
    """Strip a trailing unclosed tool-call marker (truncated output).

    When generation hits max_tokens mid-tool-call, a partial
    ``<tool_call>``/``<function=``/``[Calling tool:`` marker remains
    in the cleaned text. We locate the earliest such *unclosed* marker
    and drop everything from there to the end so the API response
    never carries raw markup.
    """
    if not text:
        return text
    earliest = len(text)

    # <tool_call> without matching </tool_call>
    idx = text.rfind("<tool_call>")
    if idx >= 0 and "</tool_call>" not in text[idx:]:
        earliest = min(earliest, idx)

    # <function=...> without matching </function>
    idx = text.rfind("<function=")
    if idx >= 0 and "</function>" not in text[idx:]:
        earliest = min(earliest, idx)

    # [Calling tool: ... without matching )]
    idx = text.rfind("[Calling tool:")
    if idx >= 0 and ")]" not in text[idx:]:
        earliest = min(earliest, idx)

    return text[:earliest].rstrip()

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._has_partial_marker

_has_partial_marker(text: str) -> bool

Check if text ends with an incomplete tool call marker prefix.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def _has_partial_marker(self, text: str) -> bool:
    """Check if text ends with an incomplete tool call marker prefix."""
    return self._get_partial_marker_len(text) > 0

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._get_partial_marker_len

_get_partial_marker_len(text: str) -> int

Return the length of a partial tool call marker suffix at end of text.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def _get_partial_marker_len(self, text: str) -> int:
    """Return the length of a partial tool call marker suffix at end of text."""
    tail = text[-20:]
    best = 0
    for marker in self._PARTIAL_MARKERS:
        for length in range(len(marker), 0, -1):
            if tail.endswith(marker[:length]) and length > best:
                best = length
                break
    return best

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._was_buffering

_was_buffering(previous_text: str) -> bool

Check if the previous call was buffering a partial marker.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def _was_buffering(self, previous_text: str) -> bool:
    """Check if the previous call was buffering a partial marker."""
    return self._has_partial_marker(previous_text)

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls_streaming

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Qwen model output.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def extract_tool_calls_streaming(
    self,
    previous_text: str,
    current_text: str,
    delta_text: str,
    previous_token_ids: Sequence[int] | None = None,
    current_token_ids: Sequence[int] | None = None,
    delta_token_ids: Sequence[int] | None = None,
    request: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
    """
    Extract tool calls from streaming Qwen model output.
    """
    # Check for complete tool call markers
    has_tool_marker = (
        "<tool_call>" in current_text
        or "[Calling tool:" in current_text
        or "<function=" in current_text
    )

    if not has_tool_marker:
        # Buffer partial markers (e.g. "<function" before "=" arrives).
        # Only the marker suffix is buffered; content before it in the
        # same delta is emitted immediately so no text is lost.
        if self._has_partial_marker(current_text):
            marker_len = self._get_partial_marker_len(current_text)
            marker_start = len(current_text) - marker_len
            safe_chars = marker_start - len(previous_text)
            if safe_chars > 0:
                return {"content": delta_text[:safe_chars]}
            return None
        # If we were buffering before but the marker didn't complete,
        # emit the buffered marker prefix together with the new delta.
        if self._was_buffering(previous_text):
            for marker in self._PARTIAL_MARKERS:
                for length in range(len(marker), 0, -1):
                    prefix = marker[:length]
                    if previous_text.endswith(prefix):
                        return {"content": prefix + delta_text}
            return {"content": delta_text}
        return {"content": delta_text}

    # Handle <function=name>...</function> (Qwen3.5 native format)
    if "<function=" in current_text:
        func_close_count = current_text.count("</function>")
        prev_func_close = previous_text.count("</function>")

        if current_text.count("<function=") > func_close_count:
            # Inside an incomplete function block, suppress output
            return None

        if func_close_count > prev_func_close:
            # New function block(s) completed
            result = self.extract_tool_calls(current_text)
            if result.tools_called:
                new_calls = result.tool_calls[prev_func_close:]
                if new_calls:
                    return {
                        "tool_calls": [
                            {
                                "index": prev_func_close + i,
                                "id": tc["id"],
                                "type": "function",
                                "function": {
                                    "name": tc["name"],
                                    "arguments": tc["arguments"],
                                },
                            }
                            for i, tc in enumerate(new_calls)
                        ]
                    }

        return None

    # If we're in a tool call, accumulate and parse at the end.
    # Check current_text (accumulated), not delta_text — closing markers
    # like ")]" or "</tool_call>" often span token boundaries and may
    # never appear within a single delta chunk.
    if "</tool_call>" in current_text or ")]" in current_text:
        # Tool call complete, parse the whole thing
        result = self.extract_tool_calls(current_text)
        if result.tools_called:
            return {
                "tool_calls": [
                    {
                        "index": i,
                        "id": tc["id"],
                        "type": "function",
                        "function": {
                            "name": tc["name"],
                            "arguments": tc["arguments"],
                        },
                    }
                    for i, tc in enumerate(result.tool_calls)
                ]
            }

    return None

vllm_mlx.tool_parsers.qwen_tool_parser._parse_param_value

_parse_param_value(val: str) -> Any

Parse a parameter value, handling JSON literals and plain strings.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def _parse_param_value(val: str) -> Any:
    """Parse a parameter value, handling JSON literals and plain strings."""
    try:
        return json.loads(val)
    except (json.JSONDecodeError, ValueError):
        pass
    try:
        python_val = ast.literal_eval(val)
        if isinstance(python_val, set):
            python_val = sorted(python_val, key=str)
        if isinstance(python_val, (complex, bytes)):
            return val
        json.dumps(python_val)
        return python_val
    except (ValueError, SyntaxError, TypeError):
        return val

vllm_mlx.tool_parsers.qwen_tool_parser.generate_tool_id

generate_tool_id() -> str

Generate a unique tool call ID.

Source code in vllm_mlx/tool_parsers/qwen_tool_parser.py

def generate_tool_id() -> str:
    """Generate a unique tool call ID."""
    return f"call_{uuid.uuid4().hex[:8]}"

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.tool_parsers.qwen_tool_parser._parse_param_value · function

vllm_mlx.tool_parsers.qwen_tool_parser._parse_param_value(val: str) -> Any

Parse a parameter value, handling JSON literals and plain strings.

Parameters

Name	Type	Required	Default	Description
val	str	yes	none	Required positional or keyword input.

Returns

• Type: Any
• Direct return expressions: json.loads(val); val; python_val

Exceptions and behavior

Function _parse_param_value calls json.loads, ast.literal_eval, isinstance, sorted; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L25-L40.

vllm_mlx.tool_parsers.qwen_tool_parser.generate_tool_id · function

vllm_mlx.tool_parsers.qwen_tool_parser.generate_tool_id() -> str

Generate a unique tool call ID.

Parameters

This callable has no explicit inputs.

Returns

• Type: str
• Direct return expressions: f'call_{uuid.uuid4().hex[:8]}'

Exceptions and behavior

Function generate_tool_id calls uuid.uuid4; returns f'call_{uuid.uuid4().hex[:8]}'. No direct raise statement appears in this definition.

View source #L43-L45.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser · class

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser()

Tool call parser for Qwen models.

Parameters

This callable has no explicit inputs.

Returns

• Constructs: vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser

Exceptions and behavior

Class QwenToolParser derives from ToolParser and declares 6 direct member(s). No direct raise statement appears in this definition.

View source #L49-L351.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Qwen model response.

Parameters

Name	Type	Required	Default	Description
model_output	str	yes	none	Required positional or keyword input.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: ExtractedToolCallInformation
• Direct return expressions: ExtractedToolCallInformation(tools_called=True, tool_calls=tool_calls, content=cleaned_text if cleaned_text else None); ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=stripped)

Exceptions and behavior

Method QwenToolParser.extract_tool_calls calls self.strip_think_tags, self.BRACKET_PATTERN.findall, json.loads, tool_calls.append; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L78-L197.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._strip_unclosed_markup · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._strip_unclosed_markup(text: str) -> str

Strip a trailing unclosed tool-call marker (truncated output).

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Required positional or keyword input.

Returns

• Type: str
• Direct return expressions: text; text[:earliest].rstrip()

Exceptions and behavior

Method QwenToolParser._strip_unclosed_markup calls len, text.rfind, min, text[:earliest].rstrip; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L200-L228.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._has_partial_marker · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._has_partial_marker(text: str) -> bool

Check if text ends with an incomplete tool call marker prefix.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: self._get_partial_marker_len(text) > 0

Exceptions and behavior

Method QwenToolParser._has_partial_marker calls self._get_partial_marker_len; returns self._get_partial_marker_len(text) > 0. No direct raise statement appears in this definition.

View source #L235-L237.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._get_partial_marker_len · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._get_partial_marker_len(text: str) -> int

Return the length of a partial tool call marker suffix at end of text.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Required positional or keyword input.

Returns

• Type: int
• Direct return expressions: best

Exceptions and behavior

Method QwenToolParser._get_partial_marker_len calls range, len, tail.endswith; returns best. No direct raise statement appears in this definition.

View source #L239-L248.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._was_buffering · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser._was_buffering(previous_text: str) -> bool

Check if the previous call was buffering a partial marker.

Parameters

Name	Type	Required	Default	Description
previous_text	str	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: self._has_partial_marker(previous_text)

Exceptions and behavior

Method QwenToolParser._was_buffering calls self._has_partial_marker; returns self._has_partial_marker(previous_text). No direct raise statement appears in this definition.

View source #L250-L252.

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls_streaming · method

vllm_mlx.tool_parsers.qwen_tool_parser.QwenToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Qwen model output.

Parameters

Name	Type	Required	Default	Description
previous_text	str	yes	none	Required positional or keyword input.
current_text	str	yes	none	Required positional or keyword input.
delta_text	str	yes	none	Required positional or keyword input.
previous_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
current_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
delta_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: dict[str, Any] | None
• Direct return expressions: {'content': delta_text[:safe_chars]}; None; {'content': prefix + delta_text}; {'content': delta_text}; {'tool_calls': [{'index': prev_func_close + i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'ar…; {'tool_calls': [{'index': i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'arguments': tc['argu…

Exceptions and behavior

Method QwenToolParser.extract_tool_calls_streaming calls self._has_partial_marker, self._get_partial_marker_len, len, self._was_buffering; has 6 explicit return paths. No direct raise statement appears in this definition.

View source #L254-L351.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
_parse_param_value	function	_parse_param_value(val: str) -> Any	Parse a parameter value, handling JSON literals and plain strings.	#L25-L40
generate_tool_id	function	generate_tool_id() -> str	Generate a unique tool call ID.	#L43-L45
QwenToolParser	class	QwenToolParser()	Tool call parser for Qwen models.	#L49-L351
QwenToolParser.extract_tool_calls	method	QwenToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] \| None = None) -> ExtractedToolCallInformation	Extract tool calls from a complete Qwen model response.	#L78-L197
QwenToolParser._strip_unclosed_markup	method	QwenToolParser._strip_unclosed_markup(text: str) -> str	Strip a trailing unclosed tool-call marker (truncated output).	#L200-L228
QwenToolParser._has_partial_marker	method	QwenToolParser._has_partial_marker(text: str) -> bool	Check if text ends with an incomplete tool call marker prefix.	#L235-L237
QwenToolParser._get_partial_marker_len	method	QwenToolParser._get_partial_marker_len(text: str) -> int	Return the length of a partial tool call marker suffix at end of text.	#L239-L248
QwenToolParser._was_buffering	method	QwenToolParser._was_buffering(previous_text: str) -> bool	Check if the previous call was buffering a partial marker.	#L250-L252
QwenToolParser.extract_tool_calls_streaming	method	QwenToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] \| None = None, current_token_ids: Sequence[int] \| None = None, delta_token_ids: Sequence[int] \| None = None, request: dict[str, Any] \| None = None) -> dict[str, Any] \| None	Extract tool calls from streaming Qwen model output.	#L254-L351