Skip to content

vllm_mlx.utils.truncation

Shared resolution of the tokenizer truncation length for embedding and reranker models.

View the complete module source at #L1-L83.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.utils.truncation

Shared resolution of the tokenizer truncation length for embedding and reranker models.

The input token limit follows each model's own context window (max_position_embeddings) instead of a hard-coded 512. A finite tokenizer limit further constrains that architecture value, while HuggingFace's huge unset placeholder is ignored. This keeps position-table offsets safe for RoBERTa-family models without losing model-derived limits for sentinel values.

vllm_mlx.utils.truncation.MAX_LENGTH_DEFAULT module-attribute 

MAX_LENGTH_DEFAULT = 512

vllm_mlx.utils.truncation.TOKENIZER_SENTINEL_THRESHOLD module-attribute 

TOKENIZER_SENTINEL_THRESHOLD = 1000000

vllm_mlx.utils.truncation._config_get 

_config_get(config: Any, key: str) -> Any

Read key from a model config that may be a dict or an object.

Source code in vllm_mlx/utils/truncation.py 
25
26
27
28
29
30
31
def _config_get(config: Any, key: str) -> Any:
    """Read ``key`` from a model config that may be a dict or an object."""
    if config is None:
        return None
    if isinstance(config, dict):
        return config.get(key)
    return getattr(config, key, None)

vllm_mlx.utils.truncation.inner_tokenizer 

inner_tokenizer(tokenizer: Any) -> Any

Unwrap a wrapping tokenizer to its inner _tokenizer when present.

Source code in vllm_mlx/utils/truncation.py 
34
35
36
def inner_tokenizer(tokenizer: Any) -> Any:
    """Unwrap a wrapping tokenizer to its inner ``_tokenizer`` when present."""
    return getattr(tokenizer, "_tokenizer", tokenizer)

vllm_mlx.utils.truncation._positive_int 

_positive_int(value: Any) -> int | None
Source code in vllm_mlx/utils/truncation.py 
39
40
41
42
def _positive_int(value: Any) -> int | None:
    if isinstance(value, int) and not isinstance(value, bool) and value > 0:
        return value
    return None

vllm_mlx.utils.truncation.resolve_max_length 

resolve_max_length(config: Any, tokenizer: Any, *, default: int = MAX_LENGTH_DEFAULT, sentinel_threshold: int = TOKENIZER_SENTINEL_THRESHOLD) -> int

Resolve the tokenizer truncation length for a model.

Source order
  1. config.max_position_embeddings — an explicit value supplied by the model's own architecture.
  2. A finite tokenizer.model_max_length further constrains the architecture value. This matters for RoBERTa-family models, whose position table includes reserved padding positions.
  3. default, when neither source yields a usable value.

Parameters:

  • config (Any) –

    Model config as a dict (reranker) or object (embeddings).

  • tokenizer (Any) –

    The tokenizer (possibly wrapping an inner _tokenizer).

  • default (int, default: MAX_LENGTH_DEFAULT ) –

    Fallback when no usable value is found.

  • sentinel_threshold (int, default: TOKENIZER_SENTINEL_THRESHOLD ) –

    Tokenizer-derived values at or above this are treated as an unset HuggingFace sentinel, not a real length.

Returns:

  • int –

    The truncation length to pass as max_length.

Source code in vllm_mlx/utils/truncation.py 
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def resolve_max_length(
    config: Any,
    tokenizer: Any,
    *,
    default: int = MAX_LENGTH_DEFAULT,
    sentinel_threshold: int = TOKENIZER_SENTINEL_THRESHOLD,
) -> int:
    """
    Resolve the tokenizer truncation length for a model.

    Source order:
      1. ``config.max_position_embeddings`` — an explicit value supplied by
         the model's own architecture.
      2. A finite ``tokenizer.model_max_length`` further constrains the
         architecture value. This matters for RoBERTa-family models, whose
         position table includes reserved padding positions.
      3. ``default``, when neither source yields a usable value.

    Args:
        config: Model config as a dict (reranker) or object (embeddings).
        tokenizer: The tokenizer (possibly wrapping an inner ``_tokenizer``).
        default: Fallback when no usable value is found.
        sentinel_threshold: Tokenizer-derived values at or above this are
            treated as an unset HuggingFace sentinel, not a real length.

    Returns:
        The truncation length to pass as ``max_length``.
    """
    config_val = _positive_int(_config_get(config, "max_position_embeddings"))
    tok_val = _positive_int(
        getattr(inner_tokenizer(tokenizer), "model_max_length", None)
    )
    if tok_val is not None and tok_val < sentinel_threshold:
        resolved = min(config_val, tok_val) if config_val is not None else tok_val
    else:
        resolved = config_val
    if resolved is None:
        resolved = default
    return resolved

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.utils.truncation._config_get · function 
vllm_mlx.utils.truncation._config_get(config: Any, key: str) -> Any

Read key from a model config that may be a dict or an object.

Parameters

Name Type Required Default Description
config Any yes none Required positional or keyword input.
key str yes none Required positional or keyword input.

Returns

  • Type: Any
  • Direct return expressions: None; config.get(key); getattr(config, key, None)

Exceptions and behavior

Function _config_get calls isinstance, config.get, getattr; has 3 explicit return paths. No direct raise statement appears in this definition.

View source #L25-L31.

vllm_mlx.utils.truncation.inner_tokenizer · function 
vllm_mlx.utils.truncation.inner_tokenizer(tokenizer: Any) -> Any

Unwrap a wrapping tokenizer to its inner _tokenizer when present.

Parameters

Name Type Required Default Description
tokenizer Any yes none Required positional or keyword input.

Returns

  • Type: Any
  • Direct return expressions: getattr(tokenizer, '_tokenizer', tokenizer)

Exceptions and behavior

Function inner_tokenizer calls getattr; returns getattr(tokenizer, '_tokenizer', tokenizer). No direct raise statement appears in this definition.

View source #L34-L36.

vllm_mlx.utils.truncation._positive_int · function 
vllm_mlx.utils.truncation._positive_int(value: Any) -> int | None

Function _positive_int calls isinstance; has 2 explicit return paths.

Parameters

Name Type Required Default Description
value Any yes none Required positional or keyword input.

Returns

  • Type: int | None
  • Direct return expressions: value; None

Exceptions and behavior

Function _positive_int calls isinstance; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L39-L42.

vllm_mlx.utils.truncation.resolve_max_length · function 
vllm_mlx.utils.truncation.resolve_max_length(config: Any, tokenizer: Any, *, default: int = MAX_LENGTH_DEFAULT, sentinel_threshold: int = TOKENIZER_SENTINEL_THRESHOLD) -> int

Resolve the tokenizer truncation length for a model.

Parameters

Name Type Required Default Description
config Any yes none Model config as a dict (reranker) or object (embeddings).
tokenizer Any yes none The tokenizer (possibly wrapping an inner _tokenizer).
default int no MAX_LENGTH_DEFAULT Fallback when no usable value is found.
sentinel_threshold int no TOKENIZER_SENTINEL_THRESHOLD Tokenizer-derived values at or above this are treated as an unset HuggingFace sentinel, not a real length.

Returns

  • Type: int
  • Direct return expressions: resolved

Exceptions and behavior

Function resolve_max_length calls _positive_int, _config_get, getattr, inner_tokenizer; returns resolved. No direct raise statement appears in this definition.

View source #L45-L83.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol Kind Signature and inputs What it does Source
_config_get function _config_get(config: Any, key: str) -> Any Read key from a model config that may be a dict or an object. #L25-L31
inner_tokenizer function inner_tokenizer(tokenizer: Any) -> Any Unwrap a wrapping tokenizer to its inner _tokenizer when present. #L34-L36
_positive_int function _positive_int(value: Any) -> int \| None Function _positive_int calls isinstance; has 2 explicit return paths. #L39-L42
resolve_max_length function resolve_max_length(config: Any, tokenizer: Any, *, default: int = MAX_LENGTH_DEFAULT, sentinel_threshold: int = TOKENIZER_SENTINEL_THRESHOLD) -> int Resolve the tokenizer truncation length for a model. #L45-L83