Technology

Mistral 7B

Mistral 7B: A 7.3 billion parameter large language model (LLM) that outperforms Meta's Llama 2 13B across all benchmarks, released under the permissive Apache 2.0 license.

Mistral 7B is a high-performance, 7.3 billion parameter LLM from Mistral AI: it sets a new standard for efficiency and capability at its size. The model leverages two key architectural innovations, Grouped-query Attention (GQA) for faster inference and Sliding Window Attention (SWA) for managing longer sequences at a lower cost. Benchmarks confirm it surpasses the larger Llama 2 13B model on all metrics and approaches CodeLlama 7B performance on code tasks. Released openly under the Apache 2.0 license, this technology is built for unrestricted use and easy fine-tuning across diverse applications.