Mamba

Mamba is a Selective State Space Model (SSM) architecture that delivers Transformer-level quality with linear-time scaling ($O(L)$) and up to 5x faster inference throughput.

Mamba redefines sequence modeling: it's a Selective State Space Model (SSM) that introduces an input-dependent selection mechanism to filter information along the sequence length dimension (Source 1.2, 1.4). This design overcomes the limitations of prior SSMs, enabling content-based reasoning (Source 1.4). The result is a streamlined, attention-free architecture that achieves linear scaling in sequence length, dramatically improving efficiency for long-context data up to million-length sequences (Source 1.1, 1.4). Specifically, the Mamba-3B model matches the quality of Transformers twice its size and provides up to 5x higher generation throughput (Source 1.2, 1.4). It serves as a general-purpose backbone for diverse modalities: language, audio, and genomics (Source 1.1, 1.2).

https://github.com/state-spaces/mamba