HTTP API reference

The server implements OpenAI-compatible, Anthropic-compatible, operational, cache, audio, reranking, and MCP routes in vllm_mlx.server. Request and response schemas are documented in vllm_mlx.api.

Default base URL:

http://127.0.0.1:8000

When --api-key is configured, protected routes require Authorization: Bearer <key>.

Generation protocols

Method	Path	Purpose	Implementation
POST	/v1/completions	OpenAI-compatible text completions	create_completion
POST	/v1/chat/completions	OpenAI-compatible chat, tools, reasoning, and multimodal input	create_chat_completion
POST	/v1/responses	OpenAI-compatible Responses API	create_response
POST	/v1/messages	Anthropic-compatible Messages API	create_anthropic_message
POST	/v1/messages/count_tokens	Count Anthropic message tokens	count_anthropic_tokens

OpenAI streaming uses Server-Sent Events and terminates with data: [DONE]. Anthropic and Responses API streams emit their protocol-specific typed terminal events.

Vector and ranking protocols

Method	Path	Purpose	Implementation
POST	/v1/embeddings	OpenAI-compatible embeddings	create_embeddings
POST	/v1/rerank	Score query-document relevance	rerank_documents

Audio protocols

Method	Path	Purpose	Implementation
POST	/v1/audio/transcriptions	Speech-to-text transcription	create_transcription
POST	/v1/audio/speech	Text-to-speech synthesis	create_speech
GET	/v1/audio/voices	List available voices for a TTS model	list_voices

Upload and input limits are applied before model execution. Audio dependencies are installed separately with the audio extra.

MCP protocols

Method	Path	Purpose	Implementation
GET	/v1/mcp/tools	List tools discovered from configured MCP servers	list_mcp_tools
GET	/v1/mcp/servers	List MCP server connection state	list_mcp_servers
POST	/v1/mcp/execute	Execute a named MCP tool	execute_mcp_tool

MCP execution is a trust boundary. Use explicit server configuration and review the MCP guide before exposing it to untrusted clients.

Models and operations

Method	Path	Purpose	Implementation
GET	/health	Basic health and residency readiness	health
GET	/metrics	Prometheus metrics when enabled	metrics
GET	/v1/status	Server, model, engine, and lifecycle status	status
GET	/v1/models	List API-visible model IDs	list_models
POST	/v1/requests/{request_id}/cancel	Cancel active generation	cancel_request
DELETE	/v1/requests/{request_id}	Delete or cancel active generation	delete_request

Cache operations

Method	Path	Purpose	Implementation
GET	/v1/cache/stats	Return active engine cache statistics	cache_stats
DELETE	/v1/cache	Clear all supported engine caches	clear_cache
DELETE	/v1/cache/prefix	Clear the prefix cache	clear_prefix_cache

Error behavior

Validation failures use HTTP 4xx responses. Authentication failures return 401. Rate limits return 429. Busy, timeout, model-loading, and internal generation failures are mapped by the endpoint to a protocol-compatible error body where possible.

For complete fields and validation rules, inspect the Pydantic model reference rather than inferring support from an upstream OpenAI or Anthropic schema.