# vllm-mlx

> vllm-mlx is an OpenAI and Anthropic compatible inference server for Apple Silicon. It provides continuous batching, multimodal generation, tool calling, structured output, and several KV-cache strategies on top of MLX.

Canonical documentation: https://vllm-mlx.is-a.dev/

The documentation site publishes human-readable pages, Markdown mirrors, a complete single-file corpus, and a line-precise JSON inventory of every tracked Python module, class, function, method, and nested helper.

## Start here

- [Documentation home](https://vllm-mlx.is-a.dev/index.md): Project overview, requirements, and navigation.
- [Installation](https://vllm-mlx.is-a.dev/getting-started/installation.md): Supported environment and installation choices.
- [Quick start](https://vllm-mlx.is-a.dev/getting-started/quickstart.md): Start a server and make the first request.
- [Runtime architecture](https://vllm-mlx.is-a.dev/concepts/runtime-architecture.md): Components, engine modes, and ownership boundaries.
- [Request lifecycle](https://vllm-mlx.is-a.dev/concepts/request-lifecycle.md): Request validation, scheduling, generation, streaming, and cleanup.

## User guides

- [Server guide](https://vllm-mlx.is-a.dev/guides/server.md): OpenAI and Anthropic compatible serving.
- [Python API](https://vllm-mlx.is-a.dev/guides/python-api.md): Direct engine and model usage.
- [Multimodal](https://vllm-mlx.is-a.dev/guides/multimodal.md): Image and video requests.
- [Audio](https://vllm-mlx.is-a.dev/guides/audio.md): Speech-to-text and text-to-speech.
- [Embeddings](https://vllm-mlx.is-a.dev/guides/embeddings.md): Embedding model setup and API usage.
- [Reasoning](https://vllm-mlx.is-a.dev/guides/reasoning.md): Reasoning parsers and thinking controls.
- [Tool calling](https://vllm-mlx.is-a.dev/guides/tool-calling.md): Parser selection and tool-call formats.
- [MCP tools](https://vllm-mlx.is-a.dev/guides/mcp-tools.md): Tool discovery, execution, and security controls.
- [Continuous batching](https://vllm-mlx.is-a.dev/guides/continuous-batching.md): Concurrent inference configuration.
- [Multi-model serving](https://vllm-mlx.is-a.dev/guides/model-registry.md): Registry-backed loading and memory budgets.

## Reference

- [HTTP API](https://vllm-mlx.is-a.dev/reference/http-api.md): Endpoint catalog and protocol behavior.
- [CLI](https://vllm-mlx.is-a.dev/reference/cli.md): Commands and options.
- [Complete CLI option inventory](https://vllm-mlx.is-a.dev/reference/cli-options.md): Every argparse declaration with defaults, choices, help, and source links.
- [CLI JSON inventory](https://vllm-mlx.is-a.dev/cli-inventory.json): Structured command-line option metadata.
- [Configuration](https://vllm-mlx.is-a.dev/reference/configuration.md): Server, cache, batching, and request settings.
- [Supported models](https://vllm-mlx.is-a.dev/reference/models.md): Model families and selection guidance.
- [Python API index](https://vllm-mlx.is-a.dev/reference/api/index.md): Generated module reference and source maps.
- [JSON API inventory](https://vllm-mlx.is-a.dev/api-inventory.json): Structured symbols, signatures, docstrings, and `#Lx-Ly` source URLs.
- [Repository source inventory](https://vllm-mlx.is-a.dev/source-inventory.json): Runtime package, maintenance scripts, and runnable examples in one structured source map.
- [Complete documentation corpus](https://vllm-mlx.is-a.dev/llms-full.txt): All documentation and all Python symbols in one Markdown document.

## Maintainers and coding agents

- [Codebase map](https://vllm-mlx.is-a.dev/development/codebase-map.md): Ownership and responsibilities by module group.
- [Agent guide](https://vllm-mlx.is-a.dev/development/agent-guide.md): Efficient reading order, invariants, tests, and change workflows.
- [Documentation guide](https://vllm-mlx.is-a.dev/development/documentation.md): Build, coverage, and publishing rules.
- [Contributing](https://vllm-mlx.is-a.dev/development/contributing.md): Development and pull request workflow.

## Languages

- [Español](https://vllm-mlx.is-a.dev/es/index.md): Spanish documentation.
- [Français](https://vllm-mlx.is-a.dev/fr/index.md): French documentation.
- [中文](https://vllm-mlx.is-a.dev/zh/index.md): Chinese documentation.