MLX Server

Deploy MLX models on Apple Silicon fast: a Python library for easy, OpenAI-compatible API serving.

MLX Server is your direct path to production for MLX models on Apple Silicon. This Python library simplifies deployment, loading models like Mistral 7B Instruct or Mixtral 8x7B directly into memory. It then initiates an HTTP server, providing a crucial OpenAI-compatible API (e.g., /v1/chat/completions) for seamless integration with existing tools and clients. Leverage the speed of MLX (Apple's unified memory array framework) to run high-performance, local AI applications.

https://mlxserver.com