vllm_mlx.tool_parsers.hermes_tool_parser

Hermes/Nous tool call parser for vllm-mlx.

View the complete module source at #L1-L336.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.tool_parsers.hermes_tool_parser

Hermes/Nous tool call parser for vllm-mlx.

Handles Hermes-style tool calling format used by NousResearch models.

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser

HermesToolParser(tokenizer: PreTrainedTokenizerBase | None = None)

Bases: ToolParser

Tool call parser for Hermes/Nous models.

Supports Hermes tool call format: - {"name": "func", "arguments": {...}} - Sometimes with additional reasoning in - Fallback: raw JSON {"name": "func", "arguments": {...}} (for models that omit tags)

Used when --enable-auto-tool-choice --tool-call-parser hermes are set.

Source code in vllm_mlx/tool_parsers/abstract_tool_parser.py

def __init__(self, tokenizer: PreTrainedTokenizerBase | None = None):
    """
    Initialize the tool parser.

    Args:
        tokenizer: The tokenizer for the model (optional, some parsers need it)
    """
    self.model_tokenizer = tokenizer
    # State for streaming parsing
    self.current_tool_id: int = -1
    self.prev_tool_call_arr: list[dict] = []

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.SUPPORTS_NATIVE_TOOL_FORMAT class-attribute instance-attribute

SUPPORTS_NATIVE_TOOL_FORMAT = True

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.TOOL_CALL_PATTERN class-attribute instance-attribute

TOOL_CALL_PATTERN = re.compile('<tool_call>\\s*(\\{.*?\\})\\s*</tool_call>', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.TOOL_CALL_LENIENT_PATTERN class-attribute instance-attribute

TOOL_CALL_LENIENT_PATTERN = re.compile('<tool_call[^{]*(\\{"name":\\s*"[^"]+",\\s*"arguments":\\s*\\{[^}]*\\}\\})', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.NEMOTRON_PATTERN class-attribute instance-attribute

NEMOTRON_PATTERN = re.compile('<tool_call>\\s*<function=([^>]+)>(.*?)</function>\\s*</tool_call>', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.PARAM_PATTERN class-attribute instance-attribute

PARAM_PATTERN = re.compile('<parameter=([^>]+)>\\s*(.*?)\\s*</parameter>', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.REASONING_PATTERN class-attribute instance-attribute

REASONING_PATTERN = re.compile('<tool_call_reasoning>(.*?)</tool_call_reasoning>', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.RAW_JSON_TOOL_PATTERN class-attribute instance-attribute

RAW_JSON_TOOL_PATTERN = re.compile('\\{"name":\\s*"([^"]+)",\\s*"arguments":\\s*(\\{[^}]*\\})\\}', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.BARE_FUNCTION_PATTERN class-attribute instance-attribute

BARE_FUNCTION_PATTERN = re.compile('<function=([^>]+)>(.*?)</function>', re.DOTALL)

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls

extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Hermes model response.

Source code in vllm_mlx/tool_parsers/hermes_tool_parser.py

def extract_tool_calls(
    self, model_output: str, request: dict[str, Any] | None = None
) -> ExtractedToolCallInformation:
    """
    Extract tool calls from a complete Hermes model response.
    """
    tool_calls = []
    cleaned_text = model_output

    # Strip <think> tags first (fallback when no reasoning parser)
    cleaned_text = self.strip_think_tags(cleaned_text)

    # Remove reasoning tags first (keep for content)
    reasoning_matches = self.REASONING_PATTERN.findall(cleaned_text)
    cleaned_text = self.REASONING_PATTERN.sub("", cleaned_text)

    # Parse tool calls with <tool_call> tags (primary format)
    matches = self.TOOL_CALL_PATTERN.findall(cleaned_text)
    for match in matches:
        try:
            data = json.loads(match)
            name = data.get("name", "")
            arguments = data.get("arguments", {})
            if name:
                tool_calls.append(
                    {
                        "id": generate_tool_id(),
                        "name": name,
                        "arguments": (
                            json.dumps(arguments, ensure_ascii=False)
                            if isinstance(arguments, dict)
                            else str(arguments)
                        ),
                    }
                )
        except json.JSONDecodeError:
            continue

    if matches:
        cleaned_text = self.TOOL_CALL_PATTERN.sub("", cleaned_text).strip()

    # Try Nemotron XML format if no JSON tool calls found
    if not tool_calls:
        nemotron_matches = self.NEMOTRON_PATTERN.findall(cleaned_text)
        for name, params_block in nemotron_matches:
            params = self.PARAM_PATTERN.findall(params_block)
            arguments = {}
            for p_name, p_value in params:
                arguments[p_name.strip()] = _parse_param_value(p_value.strip())
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": name.strip(),
                    "arguments": json.dumps(arguments, ensure_ascii=False),
                }
            )
        if nemotron_matches:
            cleaned_text = self.NEMOTRON_PATTERN.sub("", cleaned_text).strip()

    # Try bare Nemotron XML: <function=name>...</function> without <tool_call> wrapper
    # This happens when the chat template provides <tool_call> as generation prompt
    # and the model generates <function=...> directly.
    if not tool_calls:
        bare_matches = self.BARE_FUNCTION_PATTERN.findall(cleaned_text)
        for name, params_block in bare_matches:
            params = self.PARAM_PATTERN.findall(params_block)
            arguments = {}
            for p_name, p_value in params:
                arguments[p_name.strip()] = _parse_param_value(p_value.strip())
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": name.strip(),
                    "arguments": json.dumps(arguments, ensure_ascii=False),
                }
            )
        if bare_matches:
            cleaned_text = self.BARE_FUNCTION_PATTERN.sub("", cleaned_text).strip()

    # Fallback: try lenient pattern for malformed tags like <tool_call without >
    if not tool_calls:
        lenient_matches = self.TOOL_CALL_LENIENT_PATTERN.findall(cleaned_text)
        for match in lenient_matches[:1]:  # Only first to avoid hallucinations
            try:
                data = json.loads(match)
                name = data.get("name", "")
                arguments = data.get("arguments", {})
                if name:
                    tool_calls.append(
                        {
                            "id": generate_tool_id(),
                            "name": name,
                            "arguments": (
                                json.dumps(arguments, ensure_ascii=False)
                                if isinstance(arguments, dict)
                                else str(arguments)
                            ),
                        }
                    )
                    cleaned_text = self.TOOL_CALL_LENIENT_PATTERN.sub(
                        "", cleaned_text, count=1
                    ).strip()
            except json.JSONDecodeError:
                continue

    # Fallback: try raw JSON format if no tagged tool calls found
    # Only parse the FIRST valid tool call to avoid hallucinated multiple calls
    if not tool_calls:
        raw_matches = self.RAW_JSON_TOOL_PATTERN.findall(cleaned_text)
        if raw_matches:
            name, args_str = raw_matches[0]
            try:
                arguments = json.loads(args_str)
                valid_tool = True
                if request and "tools" in request:
                    tool_names = [
                        t.get("function", {}).get("name", "")
                        for t in request.get("tools", [])
                        if isinstance(t, dict)
                    ]
                    valid_tool = name in tool_names

                if valid_tool and name:
                    tool_calls.append(
                        {
                            "id": generate_tool_id(),
                            "name": name,
                            "arguments": json.dumps(arguments, ensure_ascii=False),
                        }
                    )
                    cleaned_text = self.RAW_JSON_TOOL_PATTERN.sub(
                        "", cleaned_text, count=1
                    ).strip()
            except json.JSONDecodeError:
                pass

    # Include reasoning in content if present
    if reasoning_matches:
        reasoning_text = " ".join(reasoning_matches)
        if cleaned_text:
            cleaned_text = f"{cleaned_text}\n\n(Reasoning: {reasoning_text})"
        else:
            cleaned_text = f"(Reasoning: {reasoning_text})"

    if tool_calls:
        return ExtractedToolCallInformation(
            tools_called=True,
            tool_calls=tool_calls,
            content=cleaned_text if cleaned_text else None,
        )
    else:
        return ExtractedToolCallInformation(
            tools_called=False, tool_calls=[], content=cleaned_text
        )

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser._format_streaming_tool_calls staticmethod

_format_streaming_tool_calls(tool_calls: list[dict], start_index: int = 0) -> dict[str, Any]

Format tool calls for streaming response.

Source code in vllm_mlx/tool_parsers/hermes_tool_parser.py

@staticmethod
def _format_streaming_tool_calls(
    tool_calls: list[dict], start_index: int = 0
) -> dict[str, Any]:
    """Format tool calls for streaming response."""
    return {
        "tool_calls": [
            {
                "index": start_index + i,
                "id": tc["id"],
                "type": "function",
                "function": {
                    "name": tc["name"],
                    "arguments": tc["arguments"],
                },
            }
            for i, tc in enumerate(tool_calls)
        ]
    }

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls_streaming

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Hermes model output.

Uses tag counting to correctly handle multiple sequential tool calls.

Source code in vllm_mlx/tool_parsers/hermes_tool_parser.py

def extract_tool_calls_streaming(
    self,
    previous_text: str,
    current_text: str,
    delta_text: str,
    previous_token_ids: Sequence[int] | None = None,
    current_token_ids: Sequence[int] | None = None,
    delta_token_ids: Sequence[int] | None = None,
    request: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
    """
    Extract tool calls from streaming Hermes model output.

    Uses tag counting to correctly handle multiple sequential tool calls.
    """
    # Count <tool_call> / </tool_call> tags for multi-tool support
    open_count = current_text.count("<tool_call>")
    close_count = current_text.count("</tool_call>")
    prev_close_count = previous_text.count("</tool_call>")

    if open_count > 0:
        if open_count > close_count:
            # Inside an incomplete tool call block, suppress output
            return None

        if close_count > prev_close_count:
            # New tool call(s) completed in this delta
            result = self.extract_tool_calls(current_text, request)
            if result.tools_called:
                # Only emit newly completed tool calls (skip already emitted)
                new_calls = result.tool_calls[prev_close_count:]
                if new_calls:
                    return self._format_streaming_tool_calls(
                        new_calls, start_index=prev_close_count
                    )

        # All current tool calls already emitted, pass content through
        return {"content": delta_text}

    # Bare Nemotron XML: <function=name>...</function> without <tool_call> wrapper
    # This happens when the chat template provides <tool_call> as generation prompt.
    if "<function=" in current_text:
        func_close_count = current_text.count("</function>")
        prev_func_close = previous_text.count("</function>")

        if current_text.count("<function=") > func_close_count:
            # Inside an incomplete function block, suppress output
            return None

        if func_close_count > prev_func_close:
            # New function block(s) completed
            result = self.extract_tool_calls(current_text, request)
            if result.tools_called:
                new_calls = result.tool_calls[prev_func_close:]
                if new_calls:
                    return self._format_streaming_tool_calls(
                        new_calls, start_index=prev_func_close
                    )

        return {"content": delta_text}

    # Fallback: check for raw JSON tool calls (detect closing brace pattern)
    if '{"name":' in current_text and '"arguments":' in current_text:
        if delta_text.rstrip().endswith("}"):
            result = self.extract_tool_calls(current_text, request)
            if result.tools_called:
                return self._format_streaming_tool_calls(result.tool_calls)
        return None

    return {"content": delta_text}

vllm_mlx.tool_parsers.hermes_tool_parser.generate_tool_id

generate_tool_id() -> str

Generate a unique tool call ID.

Source code in vllm_mlx/tool_parsers/hermes_tool_parser.py

def generate_tool_id() -> str:
    """Generate a unique tool call ID."""
    return f"call_{uuid.uuid4().hex[:8]}"

vllm_mlx.tool_parsers.hermes_tool_parser._parse_param_value

_parse_param_value(val: str) -> Any

Parse a tool call parameter value, handling both JSON and Python literals.

Tries json.loads first. If that fails, falls back to ast.literal_eval for Python literal syntax (single quotes, True/False, None). Converts sets to lists and rejects types that are not JSON-serializable (complex, bytes) to avoid crashes during json.dumps later.

Source code in vllm_mlx/tool_parsers/hermes_tool_parser.py

def _parse_param_value(val: str) -> Any:
    """Parse a tool call parameter value, handling both JSON and Python literals.

    Tries json.loads first. If that fails, falls back to ast.literal_eval
    for Python literal syntax (single quotes, True/False, None). Converts
    sets to lists and rejects types that are not JSON-serializable (complex,
    bytes) to avoid crashes during json.dumps later.
    """
    try:
        return json.loads(val)
    except (json.JSONDecodeError, ValueError):
        pass

    try:
        python_val = ast.literal_eval(val)
        if isinstance(python_val, set):
            python_val = sorted(python_val, key=str)
        if isinstance(python_val, (complex, bytes)):
            return val
        json.dumps(python_val)
        return python_val
    except (ValueError, SyntaxError, TypeError):
        return val

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.tool_parsers.hermes_tool_parser.generate_tool_id · function

vllm_mlx.tool_parsers.hermes_tool_parser.generate_tool_id() -> str

Generate a unique tool call ID.

Parameters

This callable has no explicit inputs.

Returns

• Type: str
• Direct return expressions: f'call_{uuid.uuid4().hex[:8]}'

Exceptions and behavior

Function generate_tool_id calls uuid.uuid4; returns f'call_{uuid.uuid4().hex[:8]}'. No direct raise statement appears in this definition.

View source #L22-L24.

vllm_mlx.tool_parsers.hermes_tool_parser._parse_param_value · function

vllm_mlx.tool_parsers.hermes_tool_parser._parse_param_value(val: str) -> Any

Parse a tool call parameter value, handling both JSON and Python literals.

Parameters

Name	Type	Required	Default	Description
val	str	yes	none	Required positional or keyword input.

Returns

• Type: Any
• Direct return expressions: json.loads(val); val; python_val

Exceptions and behavior

Function _parse_param_value calls json.loads, ast.literal_eval, isinstance, sorted; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L27-L49.

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser · class

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser()

Tool call parser for Hermes/Nous models.

Parameters

This callable has no explicit inputs.

Returns

• Constructs: vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser

Exceptions and behavior

Class HermesToolParser derives from ToolParser and declares 3 direct member(s). No direct raise statement appears in this definition.

View source #L53-L336.

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls · method

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from a complete Hermes model response.

Parameters

Name	Type	Required	Default	Description
model_output	str	yes	none	Required positional or keyword input.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: ExtractedToolCallInformation
• Direct return expressions: ExtractedToolCallInformation(tools_called=True, tool_calls=tool_calls, content=cleaned_text if cleaned_text else None); ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=cleaned_text)

Exceptions and behavior

Method HermesToolParser.extract_tool_calls calls self.strip_think_tags, self.REASONING_PATTERN.findall, self.REASONING_PATTERN.sub, self.TOOL_CALL_PATTERN.findall; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L92-L245.

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser._format_streaming_tool_calls · method

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser._format_streaming_tool_calls(tool_calls: list[dict], start_index: int = 0) -> dict[str, Any]

Format tool calls for streaming response.

Parameters

Name	Type	Required	Default	Description
tool_calls	list[dict]	yes	none	Required positional or keyword input.
start_index	int	no	0	Optional positional or keyword input; defaults to 0.

Returns

• Type: dict[str, Any]
• Direct return expressions: {'tool_calls': [{'index': start_index + i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'argume…

Exceptions and behavior

Method HermesToolParser._format_streaming_tool_calls calls enumerate; returns {'tool_calls': [{'index': start_index + i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'argume…. No direct raise statement appears in this definition.

View source #L248-L265.

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls_streaming · method

vllm_mlx.tool_parsers.hermes_tool_parser.HermesToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Hermes model output.

Parameters

Name	Type	Required	Default	Description
previous_text	str	yes	none	Required positional or keyword input.
current_text	str	yes	none	Required positional or keyword input.
delta_text	str	yes	none	Required positional or keyword input.
previous_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
current_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
delta_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: dict[str, Any] | None
• Direct return expressions: None; self._format_streaming_tool_calls(new_calls, start_index=prev_close_count); {'content': delta_text}; self._format_streaming_tool_calls(new_calls, start_index=prev_func_close); self._format_streaming_tool_calls(result.tool_calls)

Exceptions and behavior

Method HermesToolParser.extract_tool_calls_streaming calls current_text.count, previous_text.count, self.extract_tool_calls, self._format_streaming_tool_calls; has 5 explicit return paths. No direct raise statement appears in this definition.

View source #L267-L336.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
generate_tool_id	function	generate_tool_id() -> str	Generate a unique tool call ID.	#L22-L24
_parse_param_value	function	_parse_param_value(val: str) -> Any	Parse a tool call parameter value, handling both JSON and Python literals.	#L27-L49
HermesToolParser	class	HermesToolParser()	Tool call parser for Hermes/Nous models.	#L53-L336
HermesToolParser.extract_tool_calls	method	HermesToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] \| None = None) -> ExtractedToolCallInformation	Extract tool calls from a complete Hermes model response.	#L92-L245
HermesToolParser._format_streaming_tool_calls	method	HermesToolParser._format_streaming_tool_calls(tool_calls: list[dict], start_index: int = 0) -> dict[str, Any]	Format tool calls for streaming response.	#L248-L265
HermesToolParser.extract_tool_calls_streaming	method	HermesToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] \| None = None, current_token_ids: Sequence[int] \| None = None, delta_token_ids: Sequence[int] \| None = None, request: dict[str, Any] \| None = None) -> dict[str, Any] \| None	Extract tool calls from streaming Hermes model output.	#L267-L336