Candle

Minimalist ML framework for Rust: optimized for high-performance, serverless inference and lightweight binary deployment.

Candle is Hugging Face's minimalist, high-performance machine learning framework, purpose-built in Rust. It directly addresses production bottlenecks: specifically, eliminating Python overhead and the GIL for faster, more efficient inference. The framework supports multiple backends (optimized CPU, CUDA, and WASM) and focuses on deploying lightweight binaries, making serverless ML a reality. We're seeing successful implementation with state-of-the-art models: LLaMA, Whisper, and YOLOv8 are already running in browser demos.

https://huggingface.github.io/candle/