vllm_mlx.tool_parsers.functionary_tool_parser

Functionary tool call parser for vllm-mlx.

View the complete module source at #L1-L193.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.tool_parsers.functionary_tool_parser

Functionary tool call parser for vllm-mlx.

Handles MeetKai Functionary models' tool calling format. Similar to OpenAI function calling with JSON arguments.

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser

FunctionaryToolParser(tokenizer: PreTrainedTokenizerBase | None = None)

Bases: ToolParser

Tool call parser for MeetKai Functionary models.

Supports Functionary's tool call format similar to OpenAI:
- Uses special tokens to mark tool calls
- Arguments are JSON strings

Formats supported:
- <|from|>assistant

<|recipient|>func_name <|content|>{"args": ...} - {"args": ...}

Used when --enable-auto-tool-choice --tool-call-parser functionary are set.

Source code in vllm_mlx/tool_parsers/abstract_tool_parser.py

def __init__(self, tokenizer: PreTrainedTokenizerBase | None = None):
    """
    Initialize the tool parser.

    Args:
        tokenizer: The tokenizer for the model (optional, some parsers need it)
    """
    self.model_tokenizer = tokenizer
    # State for streaming parsing
    self.current_tool_id: int = -1
    self.prev_tool_call_arr: list[dict] = []

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.SUPPORTS_NATIVE_TOOL_FORMAT class-attribute instance-attribute

SUPPORTS_NATIVE_TOOL_FORMAT = True

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.RECIPIENT_PATTERN class-attribute instance-attribute

RECIPIENT_PATTERN = re.compile('<\\|recipient\\|>\\s*(\\w+)\\s*\\n<\\|content\\|>\\s*(\\{.*?\\})(?=<\\||$)', re.DOTALL)

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.FUNCTION_PATTERN class-attribute instance-attribute

FUNCTION_PATTERN = re.compile('<function=([^>]+)>(\\{.*?\\})</function>', re.DOTALL)

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.JSON_ARRAY_PATTERN class-attribute instance-attribute

JSON_ARRAY_PATTERN = re.compile('^\\s*\\[.*\\]\\s*$', re.DOTALL)

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls

extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from Functionary model output.

Source code in vllm_mlx/tool_parsers/functionary_tool_parser.py

def extract_tool_calls(
    self, model_output: str, request: dict[str, Any] | None = None
) -> ExtractedToolCallInformation:
    """
    Extract tool calls from Functionary model output.
    """
    tool_calls = []
    cleaned_text = model_output

    # Try recipient pattern (Functionary v3)
    recipient_matches = self.RECIPIENT_PATTERN.findall(model_output)
    for func_name, args_str in recipient_matches:
        if func_name.lower() in ["all", "user"]:
            continue  # Skip non-function recipients
        try:
            json.loads(args_str)
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name,
                    "arguments": args_str,
                }
            )
        except json.JSONDecodeError:
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name,
                    "arguments": args_str,
                }
            )

    if recipient_matches:
        cleaned_text = self.RECIPIENT_PATTERN.sub("", cleaned_text)
        cleaned_text = re.sub(r"<\|from\|>assistant\s*", "", cleaned_text).strip()

    # Try function pattern
    function_matches = self.FUNCTION_PATTERN.findall(cleaned_text)
    for func_name, args_str in function_matches:
        try:
            json.loads(args_str)
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name.strip(),
                    "arguments": args_str,
                }
            )
        except json.JSONDecodeError:
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name.strip(),
                    "arguments": args_str,
                }
            )

    if function_matches:
        cleaned_text = self.FUNCTION_PATTERN.sub("", cleaned_text).strip()

    # Try JSON array format
    if not tool_calls and self.JSON_ARRAY_PATTERN.match(model_output.strip()):
        try:
            parsed = json.loads(model_output.strip())
            if isinstance(parsed, list):
                for call in parsed:
                    if isinstance(call, dict) and "name" in call:
                        args = call.get("arguments", {})
                        tool_calls.append(
                            {
                                "id": generate_tool_id(),
                                "name": call["name"],
                                "arguments": (
                                    json.dumps(args, ensure_ascii=False)
                                    if isinstance(args, dict)
                                    else str(args)
                                ),
                            }
                        )
                cleaned_text = None
        except json.JSONDecodeError:
            pass

    if tool_calls:
        return ExtractedToolCallInformation(
            tools_called=True,
            tool_calls=tool_calls,
            content=cleaned_text if cleaned_text else None,
        )

    return ExtractedToolCallInformation(
        tools_called=False, tool_calls=[], content=model_output
    )

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls_streaming

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Functionary model output.

Source code in vllm_mlx/tool_parsers/functionary_tool_parser.py

def extract_tool_calls_streaming(
    self,
    previous_text: str,
    current_text: str,
    delta_text: str,
    previous_token_ids: Sequence[int] | None = None,
    current_token_ids: Sequence[int] | None = None,
    delta_token_ids: Sequence[int] | None = None,
    request: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
    """
    Extract tool calls from streaming Functionary model output.
    """
    markers = ["<|recipient|>", "<function=", "["]
    has_marker = any(m in current_text for m in markers)

    if not has_marker:
        return {"content": delta_text}

    end_markers = ["<|content|>", "</function>", "]"]
    if any(m in delta_text for m in end_markers):
        result = self.extract_tool_calls(current_text)
        if result.tools_called:
            return {
                "tool_calls": [
                    {
                        "index": i,
                        "id": tc["id"],
                        "type": "function",
                        "function": {
                            "name": tc["name"],
                            "arguments": tc["arguments"],
                        },
                    }
                    for i, tc in enumerate(result.tool_calls)
                ]
            }

    return None

vllm_mlx.tool_parsers.functionary_tool_parser.generate_tool_id

generate_tool_id() -> str

Generate a unique tool call ID.

Source code in vllm_mlx/tool_parsers/functionary_tool_parser.py

def generate_tool_id() -> str:
    """Generate a unique tool call ID."""
    return f"call_{uuid.uuid4().hex[:8]}"

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.tool_parsers.functionary_tool_parser.generate_tool_id · function

vllm_mlx.tool_parsers.functionary_tool_parser.generate_tool_id() -> str

Generate a unique tool call ID.

Parameters

This callable has no explicit inputs.

Returns

• Type: str
• Direct return expressions: f'call_{uuid.uuid4().hex[:8]}'

Exceptions and behavior

Function generate_tool_id calls uuid.uuid4; returns f'call_{uuid.uuid4().hex[:8]}'. No direct raise statement appears in this definition.

View source #L22-L24.

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser · class

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser()

Tool call parser for MeetKai Functionary models.

Parameters

This callable has no explicit inputs.

Returns

• Constructs: vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser

Exceptions and behavior

Class FunctionaryToolParser derives from ToolParser and declares 2 direct member(s). No direct raise statement appears in this definition.

View source #L28-L193.

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls · method

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from Functionary model output.

Parameters

Name	Type	Required	Default	Description
model_output	str	yes	none	Required positional or keyword input.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: ExtractedToolCallInformation
• Direct return expressions: ExtractedToolCallInformation(tools_called=True, tool_calls=tool_calls, content=cleaned_text if cleaned_text else None); ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=model_output)

Exceptions and behavior

Method FunctionaryToolParser.extract_tool_calls calls self.RECIPIENT_PATTERN.findall, func_name.lower, json.loads, tool_calls.append; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L61-L153.

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls_streaming · method

vllm_mlx.tool_parsers.functionary_tool_parser.FunctionaryToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Functionary model output.

Parameters

Name	Type	Required	Default	Description
previous_text	str	yes	none	Required positional or keyword input.
current_text	str	yes	none	Required positional or keyword input.
delta_text	str	yes	none	Required positional or keyword input.
previous_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
current_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
delta_token_ids	Sequence[int] \| None	no	None	Optional positional or keyword input; defaults to None.
request	dict[str, Any] \| None	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: dict[str, Any] | None
• Direct return expressions: {'content': delta_text}; {'tool_calls': [{'index': i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'arguments': tc['argu…; None

Exceptions and behavior

Method FunctionaryToolParser.extract_tool_calls_streaming calls any, self.extract_tool_calls, enumerate; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L155-L193.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
generate_tool_id	function	generate_tool_id() -> str	Generate a unique tool call ID.	#L22-L24
FunctionaryToolParser	class	FunctionaryToolParser()	Tool call parser for MeetKai Functionary models.	#L28-L193
FunctionaryToolParser.extract_tool_calls	method	FunctionaryToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] \| None = None) -> ExtractedToolCallInformation	Extract tool calls from Functionary model output.	#L61-L153
FunctionaryToolParser.extract_tool_calls_streaming	method	FunctionaryToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] \| None = None, current_token_ids: Sequence[int] \| None = None, delta_token_ids: Sequence[int] \| None = None, request: dict[str, Any] \| None = None) -> dict[str, Any] \| None	Extract tool calls from streaming Functionary model output.	#L155-L193