MLX-LM

The Python package for efficient text generation and fine-tuning of Large Language Models (LLMs) directly on Apple silicon via the MLX framework.

MLX-LM is a high-performance Python package engineered for text generation and fine-tuning of Large Language Models (LLMs) on Apple silicon, leveraging the core MLX array framework. It provides seamless integration with the Hugging Face Hub, allowing users to easily access and run thousands of LLMs with a single command. Key features include native support for 4-bit quantization to reduce model memory footprint and efficient low-rank or full model fine-tuning. This package enables developers to maximize the unified memory architecture of Apple silicon for faster, on-device machine learning workflows.

https://github.com/ml-explore/mlx-lm