Jamba

Jamba is AI21 Labs' groundbreaking large language model: the world's first production-grade hybrid architecture (Mamba-Transformer) featuring a 256K token context window.

Jamba, developed by AI21 Labs, is a state-of-the-art large language model (LLM) that leverages a unique hybrid architecture, combining the efficiency of a Mamba Structured State Space model (SSM) with the performance of a traditional Transformer. This novel design delivers superior efficiency and throughput, reportedly offering three times higher throughput on long-context tasks compared to same-size transformer models. Its key feature is the massive 256,000 token context window, the longest among open models, making it ideal for enterprise use cases like analyzing lengthy financial reports or enhancing Retrieval-Augmented Generation (RAG) workflows. The model weights are also open-source under an Apache 2.0 license, promoting broad adoption and customization.

https://docs.ai21.com/jamba