Skip to content

vllm_mlx.engine.chat_template_safety

Safety normalization for messages before Jinja chat-template rendering.

View the complete module source at #L1-L90.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.engine.chat_template_safety

Safety normalization for messages before Jinja chat-template rendering.

vllm_mlx.engine.chat_template_safety._close_dangling_think_before_tool_call 

_close_dangling_think_before_tool_call(content: str) -> str

Keep raw tool XML out of an unterminated <think> section.

Qwen 3.6 can produce assistant history where <think> is opened and a raw <tool_call> follows before </think>. Rendering that history as-is conditions the next turn as though the tool call is still reasoning. Close the dangling thinking span immediately before the first tool call.

This mirrors the template-side repair described by Cheuk-Yiu Chan: https://allanchan339.github.io/bug-fixes/2026/05/02/Qwen36-27B-updated-jinja.html

Source code in vllm_mlx/engine/chat_template_safety.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
def _close_dangling_think_before_tool_call(content: str) -> str:
    """Keep raw tool XML out of an unterminated ``<think>`` section.

    Qwen 3.6 can produce assistant history where ``<think>`` is opened and a
    raw ``<tool_call>`` follows before ``</think>``. Rendering that history as-is
    conditions the next turn as though the tool call is still reasoning. Close
    the dangling thinking span immediately before the first tool call.

    This mirrors the template-side repair described by Cheuk-Yiu Chan:
    https://allanchan339.github.io/bug-fixes/2026/05/02/Qwen36-27B-updated-jinja.html
    """
    if "<tool_call>" not in content or "<think>" not in content:
        return content

    last_think = content.rfind("<think>")
    last_close = content.rfind("</think>")
    tool_pos = content.find("<tool_call>")
    if last_close >= last_think and last_close != -1:
        return content
    if tool_pos > last_think:
        return content[:tool_pos] + "</think>" + content[tool_pos:]
    return content + "</think>"

vllm_mlx.engine.chat_template_safety._message_to_dict 

_message_to_dict(message: Any) -> dict[str, Any] | Any

Convert OpenAI message model objects without stringifying them.

Source code in vllm_mlx/engine/chat_template_safety.py 
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
def _message_to_dict(message: Any) -> dict[str, Any] | Any:
    """Convert OpenAI message model objects without stringifying them."""
    if isinstance(message, dict):
        return dict(message)
    model_dump = getattr(message, "model_dump", None)
    if callable(model_dump):
        return {
            key: value
            for key, value in model_dump(exclude_none=True).items()
            if value is not None
        }
    legacy_dict = getattr(message, "dict", None)
    if callable(legacy_dict):
        return {k: v for k, v in legacy_dict().items() if v is not None}
    return message

vllm_mlx.engine.chat_template_safety.normalize_messages_for_chat_template 

normalize_messages_for_chat_template(messages: list[Any]) -> list[dict]

Return a JSON-safe copy of messages for chat-template rendering.

Normalizations: - close dangling <think> spans before raw <tool_call> XML in assistant content - convert OpenAI tool-call argument JSON strings to mappings for templates that iterate argument keys

Source code in vllm_mlx/engine/chat_template_safety.py 
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def normalize_messages_for_chat_template(messages: list[Any]) -> list[dict]:
    """Return a JSON-safe copy of messages for chat-template rendering.

    Normalizations:
    - close dangling ``<think>`` spans before raw ``<tool_call>`` XML in
      assistant content
    - convert OpenAI tool-call argument JSON strings to mappings for templates
      that iterate argument keys
    """
    normalized = json.loads(
        json.dumps([_message_to_dict(message) for message in messages], default=str)
    )
    for message in normalized:
        if not isinstance(message, dict):
            continue
        if message.get("role") != "assistant":
            continue

        content = message.get("content")
        if isinstance(content, str):
            message["content"] = _close_dangling_think_before_tool_call(content)

        tool_calls = message.get("tool_calls")
        if not isinstance(tool_calls, list):
            continue
        for tool_call in tool_calls:
            if not isinstance(tool_call, dict):
                continue
            function = tool_call.get("function")
            if not isinstance(function, dict):
                continue
            arguments: Any = function.get("arguments")
            if not isinstance(arguments, str):
                continue
            try:
                parsed = json.loads(arguments)
            except (json.JSONDecodeError, ValueError, TypeError):
                parsed = {"value": arguments}
            if not isinstance(parsed, dict):
                parsed = {"value": parsed}
            function["arguments"] = parsed
    return normalized

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.engine.chat_template_safety._close_dangling_think_before_tool_call · function 
vllm_mlx.engine.chat_template_safety._close_dangling_think_before_tool_call(content: str) -> str

Keep raw tool XML out of an unterminated <think> section.

Parameters

Name Type Required Default Description
content str yes none Required positional or keyword input.

Returns

  • Type: str
  • Direct return expressions: content; content[:tool_pos] + '</think>' + content[tool_pos:]; content + '</think>'

Exceptions and behavior

Function _close_dangling_think_before_tool_call calls content.rfind, content.find; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L8-L29.

vllm_mlx.engine.chat_template_safety._message_to_dict · function 
vllm_mlx.engine.chat_template_safety._message_to_dict(message: Any) -> dict[str, Any] | Any

Convert OpenAI message model objects without stringifying them.

Parameters

Name Type Required Default Description
message Any yes none Required positional or keyword input.

Returns

  • Type: dict[str, Any] | Any
  • Direct return expressions: dict(message); {key: value for key, value in model_dump(exclude_none=True).items() if value is not None}; {k: v for k, v in legacy_dict().items() if v is not None}; message

Exceptions and behavior

Function _message_to_dict calls isinstance, dict, getattr, callable; has 4 explicit return paths. No direct raise statement appears in this definition.

View source #L32-L46.

vllm_mlx.engine.chat_template_safety.normalize_messages_for_chat_template · function 
vllm_mlx.engine.chat_template_safety.normalize_messages_for_chat_template(messages: list[Any]) -> list[dict]

Return a JSON-safe copy of messages for chat-template rendering.

Parameters

Name Type Required Default Description
messages list[Any] yes none Required positional or keyword input.

Returns

  • Type: list[dict]
  • Direct return expressions: normalized

Exceptions and behavior

Function normalize_messages_for_chat_template calls json.loads, json.dumps, _message_to_dict, isinstance; returns normalized. No direct raise statement appears in this definition.

View source #L49-L90.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
_close_dangling_think_before_tool_call function _close_dangling_think_before_tool_call(content: str) -> str Keep raw tool XML out of an unterminated <think> section. #L8-L29
_message_to_dict function _message_to_dict(message: Any) -> dict[str, Any] \| Any Convert OpenAI message model objects without stringifying them. #L32-L46
normalize_messages_for_chat_template function normalize_messages_for_chat_template(messages: list[Any]) -> list[dict] Return a JSON-safe copy of messages for chat-template rendering. #L49-L90