Candle ML

Candle is a minimalist, Rust-based ML framework, designed for high-performance, serverless inference, and GPU-accelerated deployment.

Candle, developed by Hugging Face, is your solution for high-efficiency machine learning inference: It’s a minimalist ML framework written in Rust. This design eliminates the performance drag and Global Interpreter Lock (GIL) issues common with Python-centric stacks like PyTorch. Candle focuses on serverless deployment, generating lightweight binaries for fast instance creation and optimized CPU or CUDA GPU backends. It supports WebAssembly (WASM) for running models directly in the browser, demonstrating its versatility with examples like LLaMA2, Whisper, and YOLOv8. The framework has quickly gained traction, securing thousands of stars on GitHub, proving its value for production environments where speed and minimal overhead are critical.

https://github.com/huggingface/candle