vllm_mlx.tool_parsers.harmony_tool_parser

Harmony tool call parser for GPT-OSS models.

View the complete module source at #L1-L253.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.tool_parsers.harmony_tool_parser

Harmony tool call parser for GPT-OSS models.

Harmony uses control tokens and channels for tool calling:

<|channel|>commentary to=functions.get_weather
<|constrain|>json
<|message|>{"location": "San Francisco"}
<|call|>

The final response is in the 'final' channel:

<|channel|>final
<|message|>The weather is 72F.
<|return|>

vllm_mlx.tool_parsers.harmony_tool_parser._COMMENTARY_BLOCK_PATTERN module-attribute

_COMMENTARY_BLOCK_PATTERN = re.compile('<\\|channel\\|>commentary\\s+to=functions\\.(\\w+)(?:\\s*<\\|constrain\\|>\\w+)?\\s*<\\|message\\|>((?:(?!<\\|channel\\|>).)*?)(?P<terminator><\\|call\\|>|<\\|end\\|>|<\\|return\\|>|<\\|start\\|>|<\\|channel\\|>|\\Z)', re.DOTALL)

vllm_mlx.tool_parsers.harmony_tool_parser._FINAL_BLOCK_PATTERN module-attribute

_FINAL_BLOCK_PATTERN = re.compile('<\\|channel\\|>final\\s*<\\|message\\|>(.*?)<\\|return\\|>', re.DOTALL)

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser

HarmonyToolParser(tokenizer: PreTrainedTokenizerBase | None = None)

Bases: ToolParser

Tool call parser for GPT-OSS models using Harmony format.

Harmony uses control tokens and 3 channels: - analysis: internal reasoning (handled by reasoning parser) - commentary: tool calls addressed with to=functions.{name} - final: user-facing response

Used when --enable-auto-tool-choice --tool-call-parser harmony are set.

Source code in vllm_mlx/tool_parsers/abstract_tool_parser.py

def __init__(self, tokenizer: PreTrainedTokenizerBase | None = None):
    """
    Initialize the tool parser.

    Args:
        tokenizer: The tokenizer for the model (optional, some parsers need it)
    """
    self.model_tokenizer = tokenizer
    # State for streaming parsing
    self.current_tool_id: int = -1
    self.prev_tool_call_arr: list[dict] = []

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.SUPPORTS_NATIVE_TOOL_FORMAT class-attribute instance-attribute

SUPPORTS_NATIVE_TOOL_FORMAT = False

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls

extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Harmony model response.

Parses commentary channel blocks for tool calls and the final channel for the user-facing content.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def extract_tool_calls(
    self, model_output: str, request: dict[str, Any] | None = None
) -> ExtractedToolCallInformation:
    """
    Extract tool calls from a complete Harmony model response.

    Parses commentary channel blocks for tool calls and the final
    channel for the user-facing content.
    """
    tool_calls = []

    # Extract tool calls from commentary channel blocks
    for match in _COMMENTARY_BLOCK_PATTERN.finditer(model_output):
        tool_name = match.group(1)
        args_str = match.group(2).strip()
        terminator = match.group("terminator")
        # End-of-string and a following channel boundary mean the args
        # did not stop at an explicit call terminator: truncated or moved
        # on to another channel, never a call.
        strict = terminator in ("", "<|channel|>")

        try:
            arguments = json.loads(args_str)
            tool_calls.append(
                {
                    "id": _generate_tool_id(),
                    "name": tool_name,
                    "arguments": (
                        json.dumps(arguments, ensure_ascii=False)
                        if isinstance(arguments, dict)
                        else str(arguments)
                    ),
                }
            )
        except json.JSONDecodeError:
            if strict:
                # No explicit terminator + invalid JSON: treat as
                # truncated, not a tool call.
                continue
            # Explicit terminator + invalid JSON: keep raw-args fallback.
            tool_calls.append(
                {
                    "id": _generate_tool_id(),
                    "name": tool_name,
                    "arguments": args_str,
                }
            )

    # Extract final channel content
    final_match = _FINAL_BLOCK_PATTERN.search(model_output)
    content = final_match.group(1).strip() if final_match else None

    if tool_calls:
        return ExtractedToolCallInformation(
            tools_called=True,
            tool_calls=tool_calls,
            content=content,
        )

    # No tool calls: return all text as content
    # If there's a final channel, use that; otherwise return the raw output
    # stripped of control tokens
    if content is None:
        content = _strip_control_tokens(model_output)

    return ExtractedToolCallInformation(
        tools_called=False,
        tool_calls=[],
        content=content,
    )

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls_streaming

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Harmony model output.

A commentary block completes when an explicit terminator arrives (<|call|>, <|end|>, <|return|>, <|start|>) or when the model moves on to the <|channel|>final block; the completed call is emitted once (deduplicated by name + arguments). Final-channel content is emitted as regular content deltas and plain text passes through unchanged.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def extract_tool_calls_streaming(
    self,
    previous_text: str,
    current_text: str,
    delta_text: str,
    previous_token_ids: Sequence[int] | None = None,
    current_token_ids: Sequence[int] | None = None,
    delta_token_ids: Sequence[int] | None = None,
    request: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
    """
    Extract tool calls from streaming Harmony model output.

    A commentary block completes when an explicit terminator arrives
    (<|call|>, <|end|>, <|return|>, <|start|>) or when the model moves on
    to the <|channel|>final block; the completed call is emitted once
    (deduplicated by name + arguments). Final-channel content is emitted
    as regular content deltas and plain text passes through unchanged.
    """
    if not hasattr(self, "_emitted_streaming_signatures"):
        self._emitted_streaming_signatures = set()

    # No harmony channel at all: plain text passes through
    if "<|channel|>" not in current_text:
        return {"content": delta_text}

    # A commentary block completed: an explicit terminator arrived in this
    # delta, or the model moved on to the final channel.
    block_completed = any(
        tok in delta_text
        for tok in ("<|call|>", "<|end|>", "<|return|>", "<|start|>")
    ) or (
        "<|channel|>final" in current_text
        and "<|channel|>final" not in previous_text
    )

    if block_completed:
        result = self.extract_tool_calls(current_text)
        if result.tools_called:
            emitted = []
            for i, tc in enumerate(result.tool_calls):
                signature = (tc["name"], tc["arguments"])
                if signature not in self._emitted_streaming_signatures:
                    self._emitted_streaming_signatures.add(signature)
                    emitted.append(
                        {
                            "index": i,
                            "id": tc["id"],
                            "type": "function",
                            "function": {
                                "name": tc["name"],
                                "arguments": tc["arguments"],
                            },
                        }
                    )
            if emitted:
                return {"tool_calls": emitted}

    # In the final channel, emit content
    if "<|channel|>final" in current_text and "<|call|>" not in current_text:
        # Only emit content after <|message|> in the final channel
        if "<|message|>" in current_text:
            final_start = current_text.rfind("<|channel|>final")
            msg_start = current_text.find("<|message|>", final_start)
            if msg_start >= 0:
                msg_content = current_text[msg_start + len("<|message|>") :]
                # Strip trailing control tokens
                msg_content = msg_content.replace("<|return|>", "").strip()
                if msg_content and not _is_control_token(delta_text):
                    return {"content": delta_text}

    # Building tool call or in analysis channel, suppress output
    return None

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.reset

reset() -> None

Reset parser state for a new request.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def reset(self) -> None:
    """Reset parser state for a new request."""
    super().reset()
    self._emitted_streaming_signatures = set()

vllm_mlx.tool_parsers.harmony_tool_parser._generate_tool_id

_generate_tool_id() -> str

Generate a unique tool call ID.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def _generate_tool_id() -> str:
    """Generate a unique tool call ID."""
    return f"call_{uuid.uuid4().hex[:8]}"

vllm_mlx.tool_parsers.harmony_tool_parser._strip_control_tokens

_strip_control_tokens(text: str) -> str

Remove Harmony control tokens from text.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def _strip_control_tokens(text: str) -> str:
    """Remove Harmony control tokens from text."""
    tokens = [
        "<|start|>",
        "<|end|>",
        "<|message|>",
        "<|channel|>",
        "<|constrain|>",
        "<|return|>",
        "<|call|>",
    ]
    result = text
    for token in tokens:
        result = result.replace(token, "")
    # Clean up channel names and constrain values
    result = re.sub(r"(?:analysis|commentary|final)\s*", "", result)
    result = re.sub(r"to=functions\.\w+\s*", "", result)
    result = re.sub(r"json\s*", "", result)
    return result.strip()

vllm_mlx.tool_parsers.harmony_tool_parser._is_control_token

_is_control_token(text: str) -> bool

Check if text is a Harmony control token.

Source code in vllm_mlx/tool_parsers/harmony_tool_parser.py

def _is_control_token(text: str) -> bool:
    """Check if text is a Harmony control token."""
    return text.strip() in {
        "<|start|>",
        "<|end|>",
        "<|message|>",
        "<|channel|>",
        "<|constrain|>",
        "<|return|>",
        "<|call|>",
    }

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.tool_parsers.harmony_tool_parser._generate_tool_id · function

vllm_mlx.tool_parsers.harmony_tool_parser._generate_tool_id() -> str

Generate a unique tool call ID.

Parameters

This callable has no explicit inputs.

Returns

• Type: str
• Direct return expressions: f'call_{uuid.uuid4().hex[:8]}'

Exceptions and behavior

Function _generate_tool_id calls uuid.uuid4; returns f'call_{uuid.uuid4().hex[:8]}'. No direct raise statement appears in this definition.

View source #L32-L34.

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser · class

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser()

Tool call parser for GPT-OSS models using Harmony format.

Parameters

This callable has no explicit inputs.

Returns

• Constructs: vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser

Exceptions and behavior

Class HarmonyToolParser derives from ToolParser and declares 3 direct member(s). No direct raise statement appears in this definition.

View source #L57-L219.

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls · method

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Harmony model response.

Parameters

Name	Type	Required	Default	Description
model_output	str	yes	none	Required positional or keyword input.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: ExtractedToolCallInformation
• Direct return expressions: ExtractedToolCallInformation(tools_called=True, tool_calls=tool_calls, content=content); ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=content)

Exceptions and behavior

Method HarmonyToolParser.extract_tool_calls calls _COMMENTARY_BLOCK_PATTERN.finditer, match.group, match.group(2).strip, json.loads; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L71-L140.

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls_streaming · method

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Harmony model output.

Parameters

Name	Type	Required	Default	Description
previous_text	str	yes	none	Required positional or keyword input.
current_text	str	yes	none	Required positional or keyword input.
delta_text	str	yes	none	Required positional or keyword input.
previous_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
current_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
delta_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: dict[str, Any] | None
• Direct return expressions: {'content': delta_text}; {'tool_calls': emitted}; None

Exceptions and behavior

Method HarmonyToolParser.extract_tool_calls_streaming updates self._emitted_streaming_signatures; calls hasattr, set, any, self.extract_tool_calls; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L142-L214.

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.reset · method

vllm_mlx.tool_parsers.harmony_tool_parser.HarmonyToolParser.reset() -> None

Reset parser state for a new request.

Parameters

This callable has no explicit inputs.

Returns

• Type: None

Exceptions and behavior

Method HarmonyToolParser.reset updates self._emitted_streaming_signatures; calls super().reset, super, set. No direct raise statement appears in this definition.

View source #L216-L219.

vllm_mlx.tool_parsers.harmony_tool_parser._strip_control_tokens · function

vllm_mlx.tool_parsers.harmony_tool_parser._strip_control_tokens(text: str) -> str

Remove Harmony control tokens from text.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Required positional or keyword input.

Returns

• Type: str
• Direct return expressions: result.strip()

Exceptions and behavior

Function _strip_control_tokens calls result.replace, re.sub, result.strip; returns result.strip(). No direct raise statement appears in this definition.

View source #L222-L240.

vllm_mlx.tool_parsers.harmony_tool_parser._is_control_token · function

vllm_mlx.tool_parsers.harmony_tool_parser._is_control_token(text: str) -> bool

Check if text is a Harmony control token.

Parameters

Name	Type	Required	Default	Description
text	str	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: text.strip() in {'<|start|>', '<|end|>', '<|message|>', '<|channel|>', '<|constrain|>', '<|return|>', '<|call|>'}

Exceptions and behavior

Function _is_control_token calls text.strip; returns text.strip() in {'<|start|>', '<|end|>', '<|message|>', '<|channel|>', '<|constrain|>', '<|return|>', '<|call|>'}. No direct raise statement appears in this definition.

View source #L243-L253.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
_generate_tool_id	function	_generate_tool_id() -> str	Generate a unique tool call ID.	#L32-L34
HarmonyToolParser	class	HarmonyToolParser()	Tool call parser for GPT-OSS models using Harmony format.	#L57-L219
HarmonyToolParser.extract_tool_calls	method	HarmonyToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] \| None = None) -> ExtractedToolCallInformation	Extract tool calls from a complete Harmony model response.	#L71-L140
HarmonyToolParser.extract_tool_calls_streaming	method	HarmonyToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] \| None = None, current_token_ids: Sequence[int] \| None = None, delta_token_ids: Sequence[int] \| None = None, request: dict[str, Any] \| None = None) -> dict[str, Any] \| None	Extract tool calls from streaming Harmony model output.	#L142-L214
HarmonyToolParser.reset	method	HarmonyToolParser.reset() -> None	Reset parser state for a new request.	#L216-L219
_strip_control_tokens	function	_strip_control_tokens(text: str) -> str	Remove Harmony control tokens from text.	#L222-L240
_is_control_token	function	_is_control_token(text: str) -> bool	Check if text is a Harmony control token.	#L243-L253