Skip to content

vllm_mlx.tool_parsers.kimi_tool_parser

Kimi/Moonshot tool call parser for vllm-mlx.

View the complete module source at #L1-L160.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.tool_parsers.kimi_tool_parser

Kimi/Moonshot tool call parser for vllm-mlx.

Handles Kimi K2 and related models' tool calling format: - <|tool_calls_section_begin|>...<|tool_calls_section_end|> - <|tool_call_begin|>func_name:0<|tool_call_argument_begin|>{...}<|tool_call_end|>

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser 

KimiToolParser(tokenizer: PreTrainedTokenizerBase | None = None)

Bases: ToolParser

Tool call parser for Kimi K2 and Moonshot models.

Supports Kimi's tool call format: <|tool_calls_section_begin|> <|tool_call_begin|>func:0<|tool_call_argument_begin|>{...}<|tool_call_end|> <|tool_calls_section_end|>

Used when --enable-auto-tool-choice --tool-call-parser kimi are set.

Source code in vllm_mlx/tool_parsers/abstract_tool_parser.py 
103
104
105
106
107
108
109
110
111
112
113
def __init__(self, tokenizer: PreTrainedTokenizerBase | None = None):
    """
    Initialize the tool parser.

    Args:
        tokenizer: The tokenizer for the model (optional, some parsers need it)
    """
    self.model_tokenizer = tokenizer
    # State for streaming parsing
    self.current_tool_id: int = -1
    self.prev_tool_call_arr: list[dict] = []

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.SUPPORTS_NATIVE_TOOL_FORMAT class-attribute instance-attribute 

SUPPORTS_NATIVE_TOOL_FORMAT = True

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALLS_START class-attribute instance-attribute 

TOOL_CALLS_START = '<|tool_calls_section_begin|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALLS_START_ALT class-attribute instance-attribute 

TOOL_CALLS_START_ALT = '<|tool_call_section_begin|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALLS_END class-attribute instance-attribute 

TOOL_CALLS_END = '<|tool_calls_section_end|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALLS_END_ALT class-attribute instance-attribute 

TOOL_CALLS_END_ALT = '<|tool_call_section_end|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALL_START class-attribute instance-attribute 

TOOL_CALL_START = '<|tool_call_begin|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALL_END class-attribute instance-attribute 

TOOL_CALL_END = '<|tool_call_end|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_ARG_START class-attribute instance-attribute 

TOOL_ARG_START = '<|tool_call_argument_begin|>'

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.TOOL_CALL_PATTERN class-attribute instance-attribute 

TOOL_CALL_PATTERN = re.compile('<\\|tool_call_begin\\|>\\s*(?P<func_id>[^<]+?)(?::\\d+)?\\s*<\\|tool_call_argument_begin\\|>\\s*(?P<args>.*?)\\s*<\\|tool_call_end\\|>', re.DOTALL)

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser._has_tool_section 

_has_tool_section(text: str) -> bool

Check if text contains tool section markers.

Source code in vllm_mlx/tool_parsers/kimi_tool_parser.py 
59
60
61
62
63
64
65
def _has_tool_section(self, text: str) -> bool:
    """Check if text contains tool section markers."""
    return (
        self.TOOL_CALLS_START in text
        or self.TOOL_CALLS_START_ALT in text
        or self.TOOL_CALL_START in text
    )

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls 

extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from Kimi model output.

Source code in vllm_mlx/tool_parsers/kimi_tool_parser.py 
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
def extract_tool_calls(
    self, model_output: str, request: dict[str, Any] | None = None
) -> ExtractedToolCallInformation:
    """
    Extract tool calls from Kimi model output.
    """
    if not self._has_tool_section(model_output):
        return ExtractedToolCallInformation(
            tools_called=False, tool_calls=[], content=model_output
        )

    tool_calls = []

    # Extract content before tool calls
    content = None
    for marker in [self.TOOL_CALLS_START, self.TOOL_CALLS_START_ALT]:
        if marker in model_output:
            idx = model_output.find(marker)
            content = model_output[:idx].strip() if idx > 0 else None
            break

    # Find all tool calls
    matches = self.TOOL_CALL_PATTERN.findall(model_output)
    for match in matches:
        func_id, func_args = match
        # func_id format: functions.get_weather:0 or get_weather:0
        func_name = func_id.split(":")[-2] if ":" in func_id else func_id
        func_name = func_name.split(".")[-1]  # Remove 'functions.' prefix

        try:
            # Validate JSON
            json.loads(func_args)
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name.strip(),
                    "arguments": func_args.strip(),
                }
            )
        except json.JSONDecodeError:
            tool_calls.append(
                {
                    "id": generate_tool_id(),
                    "name": func_name.strip(),
                    "arguments": func_args.strip(),
                }
            )

    if tool_calls:
        return ExtractedToolCallInformation(
            tools_called=True,
            tool_calls=tool_calls,
            content=content,
        )
    else:
        return ExtractedToolCallInformation(
            tools_called=False, tool_calls=[], content=model_output
        )

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls_streaming 

extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Kimi model output.

Source code in vllm_mlx/tool_parsers/kimi_tool_parser.py 
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
def extract_tool_calls_streaming(
    self,
    previous_text: str,
    current_text: str,
    delta_text: str,
    previous_token_ids: Sequence[int] | None = None,
    current_token_ids: Sequence[int] | None = None,
    delta_token_ids: Sequence[int] | None = None,
    request: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
    """
    Extract tool calls from streaming Kimi model output.
    """
    if not self._has_tool_section(current_text):
        return {"content": delta_text}

    if self.TOOL_CALL_END in delta_text:
        result = self.extract_tool_calls(current_text)
        if result.tools_called:
            return {
                "tool_calls": [
                    {
                        "index": i,
                        "id": tc["id"],
                        "type": "function",
                        "function": {
                            "name": tc["name"],
                            "arguments": tc["arguments"],
                        },
                    }
                    for i, tc in enumerate(result.tool_calls)
                ]
            }

    return None

vllm_mlx.tool_parsers.kimi_tool_parser.generate_tool_id 

generate_tool_id() -> str

Generate a unique tool call ID.

Source code in vllm_mlx/tool_parsers/kimi_tool_parser.py 
23
24
25
def generate_tool_id() -> str:
    """Generate a unique tool call ID."""
    return f"call_{uuid.uuid4().hex[:8]}"

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.tool_parsers.kimi_tool_parser.generate_tool_id · function 
vllm_mlx.tool_parsers.kimi_tool_parser.generate_tool_id() -> str

Generate a unique tool call ID.

Parameters

This callable has no explicit inputs.

Returns

  • Type: str
  • Direct return expressions: f'call_{uuid.uuid4().hex[:8]}'

Exceptions and behavior

Function generate_tool_id calls uuid.uuid4; returns f'call_{uuid.uuid4().hex[:8]}'. No direct raise statement appears in this definition.

View source #L23-L25.

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser · class 
vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser()

Tool call parser for Kimi K2 and Moonshot models.

Parameters

This callable has no explicit inputs.

Returns

  • Constructs: vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser

Exceptions and behavior

Class KimiToolParser derives from ToolParser and declares 3 direct member(s). No direct raise statement appears in this definition.

View source #L29-L160.

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser._has_tool_section · method 
vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser._has_tool_section(text: str) -> bool

Check if text contains tool section markers.

Parameters

Name Type Required Default Description
text str yes none Required positional or keyword input.

Returns

  • Type: bool
  • Direct return expressions: self.TOOL_CALLS_START in text or self.TOOL_CALLS_START_ALT in text or self.TOOL_CALL_START in text

Exceptions and behavior

Method KimiToolParser._has_tool_section returns self.TOOL_CALLS_START in text or self.TOOL_CALLS_START_ALT in text or self.TOOL_CALL_START in text. No direct raise statement appears in this definition.

View source #L59-L65.

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls · method 
vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] | None = None) -> ExtractedToolCallInformation

Extract tool calls from Kimi model output.

Parameters

Name Type Required Default Description
model_output str yes none Required positional or keyword input.
request dict[str, Any] \| None no None Optional positional or keyword input; defaults to None.

Returns

  • Type: ExtractedToolCallInformation
  • Direct return expressions: ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=model_output); ExtractedToolCallInformation(tools_called=True, tool_calls=tool_calls, content=content)

Exceptions and behavior

Method KimiToolParser.extract_tool_calls calls self._has_tool_section, ExtractedToolCallInformation, model_output.find, model_output[:idx].strip; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L67-L124.

vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls_streaming · method 
vllm_mlx.tool_parsers.kimi_tool_parser.KimiToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] | None = None, current_token_ids: Sequence[int] | None = None, delta_token_ids: Sequence[int] | None = None, request: dict[str, Any] | None = None) -> dict[str, Any] | None

Extract tool calls from streaming Kimi model output.

Parameters

Name Type Required Default Description
previous_text str yes none Required positional or keyword input.
current_text str yes none Required positional or keyword input.
delta_text str yes none Required positional or keyword input.
previous_token_ids Sequence[int] \| None no None Optional positional or keyword input; defaults to None.
current_token_ids Sequence[int] \| None no None Optional positional or keyword input; defaults to None.
delta_token_ids Sequence[int] \| None no None Optional positional or keyword input; defaults to None.
request dict[str, Any] \| None no None Optional positional or keyword input; defaults to None.

Returns

  • Type: dict[str, Any] | None
  • Direct return expressions: {'content': delta_text}; {'tool_calls': [{'index': i, 'id': tc['id'], 'type': 'function', 'function': {'name': tc['name'], 'arguments': tc['argu…; None

Exceptions and behavior

Method KimiToolParser.extract_tool_calls_streaming calls self._has_tool_section, self.extract_tool_calls, enumerate; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L126-L160.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
generate_tool_id function generate_tool_id() -> str Generate a unique tool call ID. #L23-L25
KimiToolParser class KimiToolParser() Tool call parser for Kimi K2 and Moonshot models. #L29-L160
KimiToolParser._has_tool_section method KimiToolParser._has_tool_section(text: str) -> bool Check if text contains tool section markers. #L59-L65
KimiToolParser.extract_tool_calls method KimiToolParser.extract_tool_calls(model_output: str, request: dict[str, Any] \| None = None) -> ExtractedToolCallInformation Extract tool calls from Kimi model output. #L67-L124
KimiToolParser.extract_tool_calls_streaming method KimiToolParser.extract_tool_calls_streaming(previous_text: str, current_text: str, delta_text: str, previous_token_ids: Sequence[int] \| None = None, current_token_ids: Sequence[int] \| None = None, delta_token_ids: Sequence[int] \| None = None, request: dict[str, Any] \| None = None) -> dict[str, Any] \| None Extract tool calls from streaming Kimi model output. #L126-L160