Sparse Autoencoders

Sparse Autoencoders (SAEs) enforce a sparsity constraint on the hidden layer to learn disentangled, interpretable features from high-dimensional, unlabeled data.

Sparse Autoencoders (SAEs) are a powerful variant of the standard autoencoder structure. The core mechanism is a sparsity constraint: only a small fraction of hidden layer neurons activate for any given input, forcing the network to focus on the most informative features. This is enforced by adding a penalty term, such as L1 regularization or KL divergence, to the reconstruction loss function. The result is a compact, overcomplete, and *disentangled* feature representation, which is critical for model interpretability. Research teams at Anthropic and OpenAI now deploy SAEs as a primary tool to decompose the internal reasoning of large language models (LLMs) like Claude 3 and GPT-4, breaking down complex, polysemantic neurons into crisp, human-understandable features.

https://adamkarvonen.github.io/machine_learning/2024/06/11/sae-intuitions.html