vllm_mlx.request

Request management for vllm-mlx continuous batching.

View the complete module source at #L1-L227.

API details

Each callable below includes its exact signature, type annotations, inputs, defaults, return contract, documented exceptions, implementation source, and parsed docstring sections when the source provides them.

vllm_mlx.request

Request management for vllm-mlx continuous batching.

This module provides Request and RequestStatus classes adapted from vLLM's request management system, simplified for MLX backend.

vllm_mlx.request.RequestStatus

Bases: IntEnum

Status of a request in the scheduling system.

vllm_mlx.request.RequestStatus.WAITING class-attribute instance-attribute

WAITING = enum.auto()

vllm_mlx.request.RequestStatus.RUNNING class-attribute instance-attribute

RUNNING = enum.auto()

vllm_mlx.request.RequestStatus.PREEMPTED class-attribute instance-attribute

PREEMPTED = enum.auto()

vllm_mlx.request.RequestStatus.FINISHED_STOPPED class-attribute instance-attribute

FINISHED_STOPPED = enum.auto()

vllm_mlx.request.RequestStatus.FINISHED_LENGTH_CAPPED class-attribute instance-attribute

FINISHED_LENGTH_CAPPED = enum.auto()

vllm_mlx.request.RequestStatus.FINISHED_ABORTED class-attribute instance-attribute

FINISHED_ABORTED = enum.auto()

vllm_mlx.request.RequestStatus.is_finished staticmethod

is_finished(status: RequestStatus) -> bool

Check if the status indicates a finished request.

Source code in vllm_mlx/request.py

@staticmethod
def is_finished(status: "RequestStatus") -> bool:
    """Check if the status indicates a finished request."""
    return status > RequestStatus.PREEMPTED

vllm_mlx.request.RequestStatus.get_finish_reason staticmethod

get_finish_reason(status: RequestStatus) -> Optional[str]

Get the finish reason string for a finished status.

Source code in vllm_mlx/request.py

@staticmethod
def get_finish_reason(status: "RequestStatus") -> Optional[str]:
    """Get the finish reason string for a finished status."""
    if status == RequestStatus.FINISHED_STOPPED:
        return "stop"
    elif status == RequestStatus.FINISHED_LENGTH_CAPPED:
        return "length"
    elif status == RequestStatus.FINISHED_ABORTED:
        return "abort"
    return None

vllm_mlx.request.SamplingParams dataclass

SamplingParams(max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, top_k: int = 0, min_p: float = 0.0, presence_penalty: float = 0.0, repetition_penalty: float = 1.0, stop: Optional[List[str]] = None, stop_token_ids: Optional[List[int]] = None, logits_processors: Optional[List[Callable]] = None)

Sampling parameters for text generation.

vllm_mlx.request.SamplingParams.max_tokens class-attribute instance-attribute

max_tokens: int = 256

vllm_mlx.request.SamplingParams.temperature class-attribute instance-attribute

temperature: float = 0.7

vllm_mlx.request.SamplingParams.top_p class-attribute instance-attribute

top_p: float = 0.9

vllm_mlx.request.SamplingParams.top_k class-attribute instance-attribute

top_k: int = 0

vllm_mlx.request.SamplingParams.min_p class-attribute instance-attribute

min_p: float = 0.0

vllm_mlx.request.SamplingParams.presence_penalty class-attribute instance-attribute

presence_penalty: float = 0.0

vllm_mlx.request.SamplingParams.repetition_penalty class-attribute instance-attribute

repetition_penalty: float = 1.0

vllm_mlx.request.SamplingParams.stop class-attribute instance-attribute

stop: Optional[List[str]] = None

vllm_mlx.request.SamplingParams.stop_token_ids class-attribute instance-attribute

stop_token_ids: Optional[List[int]] = None

vllm_mlx.request.SamplingParams.logits_processors class-attribute instance-attribute

logits_processors: Optional[List[Callable]] = None

vllm_mlx.request.SamplingParams.__post_init__

__post_init__()

Source code in vllm_mlx/request.py

def __post_init__(self):
    if self.stop is None:
        self.stop = []
    if self.stop_token_ids is None:
        self.stop_token_ids = []

vllm_mlx.request.Request dataclass

Request(request_id: str, prompt: Union[str, List[int]], sampling_params: SamplingParams, arrival_time: float = time(), priority: int = 0, prompt_token_ids: Optional[List[int]] = None, num_prompt_tokens: int = 0, status: RequestStatus = WAITING, num_computed_tokens: int = 0, output_token_ids: List[int] = list(), output_text: str = '', batch_uid: Optional[int] = None, prompt_cache: Optional[List[Any]] = None, cached_tokens: int = 0, remaining_tokens: Optional[List[int]] = None, prefix_boundary: int = 0, block_table: Optional[BlockTable] = None, shared_prefix_blocks: int = 0, images: Optional[List[Any]] = None, videos: Optional[List[Any]] = None, pixel_values: Optional[Any] = None, image_grid_thw: Optional[Any] = None, attention_mask: Optional[Any] = None, multimodal_kwargs: Optional[Dict[str, Any]] = None, is_multimodal: bool = False, finish_reason: Optional[str] = None, first_token_time: Optional[float] = None, cache_hit_type: Optional[str] = None)

Represents a single inference request in the scheduling system.

Adapted from vLLM's Request class with simplifications for MLX backend.

Attributes:

• request_id (str) –

  Unique identifier for this request

• prompt (Union[str, List[int]]) –

  The input prompt (string or token ids)

• prompt_token_ids (Optional[List[int]]) –

  Tokenized prompt

• sampling_params (SamplingParams) –

  Parameters for generation

• arrival_time (float) –

  When the request was received

• status (RequestStatus) –

  Current status of the request

• num_prompt_tokens (int) –

  Number of tokens in the prompt

• num_computed_tokens (int) –

  Number of tokens processed so far

• output_token_ids (List[int]) –

  Generated token ids

• output_text (str) –

  Generated text (decoded)

vllm_mlx.request.Request.request_id instance-attribute

request_id: str

vllm_mlx.request.Request.prompt instance-attribute

prompt: Union[str, List[int]]

vllm_mlx.request.Request.sampling_params instance-attribute

sampling_params: SamplingParams

vllm_mlx.request.Request.arrival_time class-attribute instance-attribute

arrival_time: float = field(default_factory=time.time)

vllm_mlx.request.Request.priority class-attribute instance-attribute

priority: int = 0

vllm_mlx.request.Request.prompt_token_ids class-attribute instance-attribute

prompt_token_ids: Optional[List[int]] = None

vllm_mlx.request.Request.num_prompt_tokens class-attribute instance-attribute

num_prompt_tokens: int = 0

vllm_mlx.request.Request.status class-attribute instance-attribute

status: RequestStatus = RequestStatus.WAITING

vllm_mlx.request.Request.num_computed_tokens class-attribute instance-attribute

num_computed_tokens: int = 0

vllm_mlx.request.Request.output_token_ids class-attribute instance-attribute

output_token_ids: List[int] = field(default_factory=list)

vllm_mlx.request.Request.output_text class-attribute instance-attribute

output_text: str = ''

vllm_mlx.request.Request.batch_uid class-attribute instance-attribute

batch_uid: Optional[int] = None

vllm_mlx.request.Request.prompt_cache class-attribute instance-attribute

prompt_cache: Optional[List[Any]] = None

vllm_mlx.request.Request.cached_tokens class-attribute instance-attribute

cached_tokens: int = 0

vllm_mlx.request.Request.remaining_tokens class-attribute instance-attribute

remaining_tokens: Optional[List[int]] = None

vllm_mlx.request.Request.prefix_boundary class-attribute instance-attribute

prefix_boundary: int = 0

vllm_mlx.request.Request.block_table class-attribute instance-attribute

block_table: Optional[BlockTable] = None

vllm_mlx.request.Request.shared_prefix_blocks class-attribute instance-attribute

shared_prefix_blocks: int = 0

vllm_mlx.request.Request.images class-attribute instance-attribute

images: Optional[List[Any]] = None

vllm_mlx.request.Request.videos class-attribute instance-attribute

videos: Optional[List[Any]] = None

vllm_mlx.request.Request.pixel_values class-attribute instance-attribute

pixel_values: Optional[Any] = None

vllm_mlx.request.Request.image_grid_thw class-attribute instance-attribute

image_grid_thw: Optional[Any] = None

vllm_mlx.request.Request.attention_mask class-attribute instance-attribute

attention_mask: Optional[Any] = None

vllm_mlx.request.Request.multimodal_kwargs class-attribute instance-attribute

multimodal_kwargs: Optional[Dict[str, Any]] = None

vllm_mlx.request.Request.is_multimodal class-attribute instance-attribute

is_multimodal: bool = False

vllm_mlx.request.Request.finish_reason class-attribute instance-attribute

finish_reason: Optional[str] = None

vllm_mlx.request.Request.first_token_time class-attribute instance-attribute

first_token_time: Optional[float] = None

vllm_mlx.request.Request.cache_hit_type class-attribute instance-attribute

cache_hit_type: Optional[str] = None

vllm_mlx.request.Request.num_output_tokens property

num_output_tokens: int

Number of output tokens generated so far.

vllm_mlx.request.Request.num_tokens property

num_tokens: int

Total number of tokens (prompt + output).

vllm_mlx.request.Request.max_tokens property

max_tokens: int

Maximum output tokens for this request.

vllm_mlx.request.Request.is_finished

is_finished() -> bool

Check if request has finished.

Source code in vllm_mlx/request.py

def is_finished(self) -> bool:
    """Check if request has finished."""
    return RequestStatus.is_finished(self.status)

vllm_mlx.request.Request.get_finish_reason

get_finish_reason() -> Optional[str]

Get the finish reason if finished.

Source code in vllm_mlx/request.py

def get_finish_reason(self) -> Optional[str]:
    """Get the finish reason if finished."""
    if self.finish_reason:
        return self.finish_reason
    return RequestStatus.get_finish_reason(self.status)

vllm_mlx.request.Request.append_output_token

append_output_token(token_id: int) -> None

Append a generated token to the output.

Source code in vllm_mlx/request.py

def append_output_token(self, token_id: int) -> None:
    """Append a generated token to the output."""
    self.output_token_ids.append(token_id)
    self.num_computed_tokens += 1

vllm_mlx.request.Request.set_finished

set_finished(status: RequestStatus, reason: Optional[str] = None) -> None

Mark the request as finished.

Source code in vllm_mlx/request.py

def set_finished(self, status: RequestStatus, reason: Optional[str] = None) -> None:
    """Mark the request as finished."""
    self.status = status
    self.finish_reason = reason or RequestStatus.get_finish_reason(status)

vllm_mlx.request.Request.__lt__

__lt__(other: Request) -> bool

Compare requests for priority queue ordering.

Source code in vllm_mlx/request.py

def __lt__(self, other: "Request") -> bool:
    """Compare requests for priority queue ordering."""
    if self.priority != other.priority:
        return self.priority < other.priority
    return self.arrival_time < other.arrival_time

vllm_mlx.request.Request.__hash__

__hash__() -> int

Source code in vllm_mlx/request.py

def __hash__(self) -> int:
    return hash(self.request_id)

vllm_mlx.request.Request.__eq__

__eq__(other: object) -> bool

Source code in vllm_mlx/request.py

def __eq__(self, other: object) -> bool:
    if not isinstance(other, Request):
        return False
    return self.request_id == other.request_id

vllm_mlx.request.RequestOutput dataclass

RequestOutput(request_id: str, new_token_ids: List[int] = list(), new_text: str = '', output_token_ids: List[int] = list(), output_text: str = '', finished: bool = False, finish_reason: Optional[str] = None, prompt_tokens: int = 0, completion_tokens: int = 0, mtp_drafts: int = 0, mtp_accepted: int = 0)

Output for a single request after a generation step.

This is returned by the engine to communicate results back to the API layer.

vllm_mlx.request.RequestOutput.request_id instance-attribute

request_id: str

vllm_mlx.request.RequestOutput.new_token_ids class-attribute instance-attribute

new_token_ids: List[int] = field(default_factory=list)

vllm_mlx.request.RequestOutput.new_text class-attribute instance-attribute

new_text: str = ''

vllm_mlx.request.RequestOutput.output_token_ids class-attribute instance-attribute

output_token_ids: List[int] = field(default_factory=list)

vllm_mlx.request.RequestOutput.output_text class-attribute instance-attribute

output_text: str = ''

vllm_mlx.request.RequestOutput.finished class-attribute instance-attribute

finished: bool = False

vllm_mlx.request.RequestOutput.finish_reason class-attribute instance-attribute

finish_reason: Optional[str] = None

vllm_mlx.request.RequestOutput.prompt_tokens class-attribute instance-attribute

prompt_tokens: int = 0

vllm_mlx.request.RequestOutput.completion_tokens class-attribute instance-attribute

completion_tokens: int = 0

vllm_mlx.request.RequestOutput.mtp_drafts class-attribute instance-attribute

mtp_drafts: int = 0

vllm_mlx.request.RequestOutput.mtp_accepted class-attribute instance-attribute

mtp_accepted: int = 0

vllm_mlx.request.RequestOutput.usage property

usage: Dict[str, int]

Return usage statistics compatible with OpenAI API.

Complete contract reference

Expand any definition for its exact inputs, annotations, defaults, return contract, directly raised exceptions, source-grounded behavior, and immutable line link. This section includes private and nested definitions that ordinary API generators omit.

vllm_mlx.request.RequestStatus · class

vllm_mlx.request.RequestStatus()

Status of a request in the scheduling system.

Parameters

This callable has no explicit inputs.

Returns

• Constructs: vllm_mlx.request.RequestStatus

Exceptions and behavior

Class RequestStatus derives from enum.IntEnum and declares 2 direct member(s). No direct raise statement appears in this definition.

View source #L18-L48.

vllm_mlx.request.RequestStatus.is_finished · method

vllm_mlx.request.RequestStatus.is_finished(status: 'RequestStatus') -> bool

Check if the status indicates a finished request.

Parameters

Name	Type	Required	Default	Description
status	'RequestStatus'	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: status > RequestStatus.PREEMPTED

Exceptions and behavior

Method RequestStatus.is_finished returns status > RequestStatus.PREEMPTED. No direct raise statement appears in this definition.

View source #L35-L37.

vllm_mlx.request.RequestStatus.get_finish_reason · method

vllm_mlx.request.RequestStatus.get_finish_reason(status: 'RequestStatus') -> Optional[str]

Get the finish reason string for a finished status.

Parameters

Name	Type	Required	Default	Description
status	'RequestStatus'	yes	none	Required positional or keyword input.

Returns

• Type: Optional[str]
• Direct return expressions: 'stop'; 'length'; 'abort'; None

Exceptions and behavior

Method RequestStatus.get_finish_reason has 4 explicit return paths. No direct raise statement appears in this definition.

View source #L40-L48.

vllm_mlx.request.SamplingParams · class

vllm_mlx.request.SamplingParams(max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, top_k: int = 0, min_p: float = 0.0, presence_penalty: float = 0.0, repetition_penalty: float = 1.0, stop: Optional[List[str]] = None, stop_token_ids: Optional[List[int]] = None, logits_processors: Optional[List[Callable]] = None)

Sampling parameters for text generation.

Parameters

Name	Type	Required	Default	Description
max_tokens	int	no	256	Optional constructor field; defaults to 256.
temperature	float	no	0.7	Optional constructor field; defaults to 0.7.
top_p	float	no	0.9	Optional constructor field; defaults to 0.9.
top_k	int	no	0	Optional constructor field; defaults to 0.
min_p	float	no	0.0	Optional constructor field; defaults to 0.0.
presence_penalty	float	no	0.0	Optional constructor field; defaults to 0.0.
repetition_penalty	float	no	1.0	Optional constructor field; defaults to 1.0.
stop	Optional[List[str]]	no	None	Optional constructor field; defaults to None.
stop_token_ids	Optional[List[int]]	no	None	Optional constructor field; defaults to None.
logits_processors	Optional[List[Callable]]	no	None	Optional constructor field; defaults to None.

Returns

• Constructs: vllm_mlx.request.SamplingParams

Exceptions and behavior

Class SamplingParams declares 1 direct member(s). No direct raise statement appears in this definition.

View source #L52-L73.

vllm_mlx.request.SamplingParams.__post_init__ · method

vllm_mlx.request.SamplingParams.__post_init__() -> not annotated

Method SamplingParams.__post_init__ updates self.stop, self.stop_token_ids.

Parameters

This callable has no explicit inputs.

Returns

• Type: not annotated

Exceptions and behavior

Method SamplingParams.__post_init__ updates self.stop, self.stop_token_ids. No direct raise statement appears in this definition.

View source #L69-L73.

vllm_mlx.request.Request · class

vllm_mlx.request.Request(request_id: str, prompt: Union[str, List[int]], sampling_params: SamplingParams, arrival_time: float = field(default_factory=time.time), priority: int = 0, prompt_token_ids: Optional[List[int]] = None, num_prompt_tokens: int = 0, status: RequestStatus = RequestStatus.WAITING, num_computed_tokens: int = 0, output_token_ids: List[int] = field(default_factory=list), output_text: str = '', batch_uid: Optional[int] = None, prompt_cache: Optional[List[Any]] = None, cached_tokens: int = 0, remaining_tokens: Optional[List[int]] = None, prefix_boundary: int = 0, block_table: Optional['BlockTable'] = None, shared_prefix_blocks: int = 0, images: Optional[List[Any]] = None, videos: Optional[List[Any]] = None, pixel_values: Optional[Any] = None, image_grid_thw: Optional[Any] = None, attention_mask: Optional[Any] = None, multimodal_kwargs: Optional[Dict[str, Any]] = None, is_multimodal: bool = False, finish_reason: Optional[str] = None, first_token_time: Optional[float] = None, cache_hit_type: Optional[str] = None)

Represents a single inference request in the scheduling system.

Parameters

Name	Type	Required	Default	Description
request_id	str	yes	none	Required constructor field.
prompt	Union[str, List[int]]	yes	none	Required constructor field.
sampling_params	SamplingParams	yes	none	Required constructor field.
arrival_time	float	no	field(default_factory=time.time)	Optional constructor field; defaults to field(default_factory=time.time).
priority	int	no	0	Optional constructor field; defaults to 0.
prompt_token_ids	Optional[List[int]]	no	None	Optional constructor field; defaults to None.
num_prompt_tokens	int	no	0	Optional constructor field; defaults to 0.
status	RequestStatus	no	RequestStatus.WAITING	Optional constructor field; defaults to RequestStatus.WAITING.
num_computed_tokens	int	no	0	Optional constructor field; defaults to 0.
output_token_ids	List[int]	no	field(default_factory=list)	Optional constructor field; defaults to field(default_factory=list).
output_text	str	no	''	Optional constructor field; defaults to ''.
batch_uid	Optional[int]	no	None	Optional constructor field; defaults to None.
prompt_cache	Optional[List[Any]]	no	None	Optional constructor field; defaults to None.
cached_tokens	int	no	0	Optional constructor field; defaults to 0.
remaining_tokens	Optional[List[int]]	no	None	Optional constructor field; defaults to None.
prefix_boundary	int	no	0	Optional constructor field; defaults to 0.
block_table	Optional['BlockTable']	no	None	Optional constructor field; defaults to None.
shared_prefix_blocks	int	no	0	Optional constructor field; defaults to 0.
images	Optional[List[Any]]	no	None	Optional constructor field; defaults to None.
videos	Optional[List[Any]]	no	None	Optional constructor field; defaults to None.
pixel_values	Optional[Any]	no	None	Optional constructor field; defaults to None.
image_grid_thw	Optional[Any]	no	None	Optional constructor field; defaults to None.
attention_mask	Optional[Any]	no	None	Optional constructor field; defaults to None.
multimodal_kwargs	Optional[Dict[str, Any]]	no	None	Optional constructor field; defaults to None.
is_multimodal	bool	no	False	Optional constructor field; defaults to False.
finish_reason	Optional[str]	no	None	Optional constructor field; defaults to None.
first_token_time	Optional[float]	no	None	Optional constructor field; defaults to None.
cache_hit_type	Optional[str]	no	None	Optional constructor field; defaults to None.

Returns

• Constructs: vllm_mlx.request.Request

Exceptions and behavior

Class Request declares 10 direct member(s). No direct raise statement appears in this definition.

View source #L77-L192.

vllm_mlx.request.Request.num_output_tokens · method

vllm_mlx.request.Request.num_output_tokens() -> int

Number of output tokens generated so far.

Parameters

This callable has no explicit inputs.

Returns

• Type: int
• Direct return expressions: len(self.output_token_ids)

Exceptions and behavior

Method Request.num_output_tokens calls len; returns len(self.output_token_ids). No direct raise statement appears in this definition.

View source #L146-L148.

vllm_mlx.request.Request.num_tokens · method

vllm_mlx.request.Request.num_tokens() -> int

Total number of tokens (prompt + output).

Parameters

This callable has no explicit inputs.

Returns

• Type: int
• Direct return expressions: self.num_prompt_tokens + self.num_output_tokens

Exceptions and behavior

Method Request.num_tokens returns self.num_prompt_tokens + self.num_output_tokens. No direct raise statement appears in this definition.

View source #L151-L153.

vllm_mlx.request.Request.max_tokens · method

vllm_mlx.request.Request.max_tokens() -> int

Maximum output tokens for this request.

Parameters

This callable has no explicit inputs.

Returns

• Type: int
• Direct return expressions: self.sampling_params.max_tokens

Exceptions and behavior

Method Request.max_tokens returns self.sampling_params.max_tokens. No direct raise statement appears in this definition.

View source #L156-L158.

vllm_mlx.request.Request.is_finished · method

vllm_mlx.request.Request.is_finished() -> bool

Check if request has finished.

Parameters

This callable has no explicit inputs.

Returns

• Type: bool
• Direct return expressions: RequestStatus.is_finished(self.status)

Exceptions and behavior

Method Request.is_finished calls RequestStatus.is_finished; returns RequestStatus.is_finished(self.status). No direct raise statement appears in this definition.

View source #L160-L162.

vllm_mlx.request.Request.get_finish_reason · method

vllm_mlx.request.Request.get_finish_reason() -> Optional[str]

Get the finish reason if finished.

Parameters

This callable has no explicit inputs.

Returns

• Type: Optional[str]
• Direct return expressions: self.finish_reason; RequestStatus.get_finish_reason(self.status)

Exceptions and behavior

Method Request.get_finish_reason calls RequestStatus.get_finish_reason; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L164-L168.

vllm_mlx.request.Request.append_output_token · method

vllm_mlx.request.Request.append_output_token(token_id: int) -> None

Append a generated token to the output.

Parameters

Name	Type	Required	Default	Description
token_id	int	yes	none	Required positional or keyword input.

Returns

• Type: None

Exceptions and behavior

Method Request.append_output_token updates self.num_computed_tokens; calls self.output_token_ids.append. No direct raise statement appears in this definition.

View source #L170-L173.

vllm_mlx.request.Request.set_finished · method

vllm_mlx.request.Request.set_finished(status: RequestStatus, reason: Optional[str] = None) -> None

Mark the request as finished.

Parameters

Name	Type	Required	Default	Description
status	RequestStatus	yes	none	Required positional or keyword input.
reason	Optional[str]	no	None	Optional positional or keyword input; defaults to None.

Returns

• Type: None

Exceptions and behavior

Method Request.set_finished updates self.status, self.finish_reason; calls RequestStatus.get_finish_reason. No direct raise statement appears in this definition.

View source #L175-L178.

vllm_mlx.request.Request.__lt__ · method

vllm_mlx.request.Request.__lt__(other: 'Request') -> bool

Compare requests for priority queue ordering.

Parameters

Name	Type	Required	Default	Description
other	'Request'	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: self.priority < other.priority; self.arrival_time < other.arrival_time

Exceptions and behavior

Method Request.__lt__ has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L180-L184.

vllm_mlx.request.Request.__hash__ · method

vllm_mlx.request.Request.__hash__() -> int

Method Request.__hash__ calls hash; returns hash(self.request_id).

Parameters

This callable has no explicit inputs.

Returns

• Type: int
• Direct return expressions: hash(self.request_id)

Exceptions and behavior

Method Request.__hash__ calls hash; returns hash(self.request_id). No direct raise statement appears in this definition.

View source #L186-L187.

vllm_mlx.request.Request.__eq__ · method

vllm_mlx.request.Request.__eq__(other: object) -> bool

Method Request.__eq__ calls isinstance; has 2 explicit return paths.

Parameters

Name	Type	Required	Default	Description
other	object	yes	none	Required positional or keyword input.

Returns

• Type: bool
• Direct return expressions: False; self.request_id == other.request_id

Exceptions and behavior

Method Request.__eq__ calls isinstance; has 2 explicit return paths. No direct raise statement appears in this definition.

View source #L189-L192.

vllm_mlx.request.RequestOutput · class

vllm_mlx.request.RequestOutput(request_id: str, new_token_ids: List[int] = field(default_factory=list), new_text: str = '', output_token_ids: List[int] = field(default_factory=list), output_text: str = '', finished: bool = False, finish_reason: Optional[str] = None, prompt_tokens: int = 0, completion_tokens: int = 0, mtp_drafts: int = 0, mtp_accepted: int = 0)

Output for a single request after a generation step.

Parameters

Name	Type	Required	Default	Description
request_id	str	yes	none	Required constructor field.
new_token_ids	List[int]	no	field(default_factory=list)	Optional constructor field; defaults to field(default_factory=list).
new_text	str	no	''	Optional constructor field; defaults to ''.
output_token_ids	List[int]	no	field(default_factory=list)	Optional constructor field; defaults to field(default_factory=list).
output_text	str	no	''	Optional constructor field; defaults to ''.
finished	bool	no	False	Optional constructor field; defaults to False.
finish_reason	Optional[str]	no	None	Optional constructor field; defaults to None.
prompt_tokens	int	no	0	Optional constructor field; defaults to 0.
completion_tokens	int	no	0	Optional constructor field; defaults to 0.
mtp_drafts	int	no	0	Optional constructor field; defaults to 0.
mtp_accepted	int	no	0	Optional constructor field; defaults to 0.

Returns

• Constructs: vllm_mlx.request.RequestOutput

Exceptions and behavior

Class RequestOutput declares 1 direct member(s). No direct raise statement appears in this definition.

View source #L196-L227.

vllm_mlx.request.RequestOutput.usage · method

vllm_mlx.request.RequestOutput.usage() -> Dict[str, int]

Return usage statistics compatible with OpenAI API.

Parameters

This callable has no explicit inputs.

Returns

• Type: Dict[str, int]
• Direct return expressions: {'prompt_tokens': self.prompt_tokens, 'completion_tokens': self.completion_tokens, 'total_tokens': self.prompt_tokens +…

Exceptions and behavior

Method RequestOutput.usage returns {'prompt_tokens': self.prompt_tokens, 'completion_tokens': self.completion_tokens, 'total_tokens': self.prompt_tokens +…. No direct raise statement appears in this definition.

View source #L221-L227.

Complete symbol map

This map also includes private definitions and nested helpers. The signature column exposes every explicit input even when an internal helper has no dedicated parameter prose.

Symbol	Kind	Signature and inputs	What it does	Source
RequestStatus	class	RequestStatus()	Status of a request in the scheduling system.	#L18-L48
RequestStatus.is_finished	method	RequestStatus.is_finished(status: 'RequestStatus') -> bool	Check if the status indicates a finished request.	#L35-L37
RequestStatus.get_finish_reason	method	RequestStatus.get_finish_reason(status: 'RequestStatus') -> Optional[str]	Get the finish reason string for a finished status.	#L40-L48
SamplingParams	class	SamplingParams(max_tokens: int = 256, temperature: float = 0.7, top_p: float = 0.9, top_k: int = 0, min_p: float = 0.0, presence_penalty: float = 0.0, repetition_penalty: float = 1.0, stop: Optional[List[str]] = None, stop_token_ids: Optional[List[int]] = None, logits_processors: Optional[List[Callable]] = None)	Sampling parameters for text generation.	#L52-L73
SamplingParams.__post_init__	method	SamplingParams.__post_init__() -> not annotated	Method SamplingParams.__post_init__ updates self.stop, self.stop_token_ids.	#L69-L73
Request	class	Request(request_id: str, prompt: Union[str, List[int]], sampling_params: SamplingParams, arrival_time: float = field(default_factory=time.time), priority: int = 0, prompt_token_ids: Optional[List[int]] = None, num_prompt_tokens: int = 0, status: RequestStatus = RequestStatus.WAITING, num_computed_tokens: int = 0, output_token_ids: List[int] = field(default_factory=list), output_text: str = '', batch_uid: Optional[int] = None, prompt_cache: Optional[List[Any]] = None, cached_tokens: int = 0, remaining_tokens: Optional[List[int]] = None, prefix_boundary: int = 0, block_table: Optional['BlockTable'] = None, shared_prefix_blocks: int = 0, images: Optional[List[Any]] = None, videos: Optional[List[Any]] = None, pixel_values: Optional[Any] = None, image_grid_thw: Optional[Any] = None, attention_mask: Optional[Any] = None, multimodal_kwargs: Optional[Dict[str, Any]] = None, is_multimodal: bool = False, finish_reason: Optional[str] = None, first_token_time: Optional[float] = None, cache_hit_type: Optional[str] = None)	Represents a single inference request in the scheduling system.	#L77-L192
Request.num_output_tokens	method	Request.num_output_tokens() -> int	Number of output tokens generated so far.	#L146-L148
Request.num_tokens	method	Request.num_tokens() -> int	Total number of tokens (prompt + output).	#L151-L153
Request.max_tokens	method	Request.max_tokens() -> int	Maximum output tokens for this request.	#L156-L158
Request.is_finished	method	Request.is_finished() -> bool	Check if request has finished.	#L160-L162
Request.get_finish_reason	method	Request.get_finish_reason() -> Optional[str]	Get the finish reason if finished.	#L164-L168
Request.append_output_token	method	Request.append_output_token(token_id: int) -> None	Append a generated token to the output.	#L170-L173
Request.set_finished	method	Request.set_finished(status: RequestStatus, reason: Optional[str] = None) -> None	Mark the request as finished.	#L175-L178
Request.__lt__	method	Request.__lt__(other: 'Request') -> bool	Compare requests for priority queue ordering.	#L180-L184
Request.__hash__	method	Request.__hash__() -> int	Method Request.__hash__ calls hash; returns hash(self.request_id).	#L186-L187
Request.__eq__	method	Request.__eq__(other: object) -> bool	Method Request.__eq__ calls isinstance; has 2 explicit return paths.	#L189-L192
RequestOutput	class	RequestOutput(request_id: str, new_token_ids: List[int] = field(default_factory=list), new_text: str = '', output_token_ids: List[int] = field(default_factory=list), output_text: str = '', finished: bool = False, finish_reason: Optional[str] = None, prompt_tokens: int = 0, completion_tokens: int = 0, mtp_drafts: int = 0, mtp_accepted: int = 0)	Output for a single request after a generation step.	#L196-L227
RequestOutput.usage	method	RequestOutput.usage() -> Dict[str, int]	Return usage statistics compatible with OpenAI API.	#L221-L227