sentence-transformers (local embeddings)

A Python framework for generating state-of-the-art dense vector representations for sentences, paragraphs, and images using transformer networks.

Sentence-transformers (SBERT) provides a streamlined interface to compute local embeddings for over 15,000 pre-trained models available via Hugging Face. By utilizing Siamese and triplet network structures, it transforms raw text into fixed-size vectors (embeddings) that reside in a continuous vector space where semantically similar items are positioned closely. This local execution enables high-throughput processing for tasks like semantic search, clustering, and paraphrase mining without the latency or privacy concerns of external APIs. Developers can implement a full retrieval pipeline by loading a model like all-MiniLM-L6-v2 with just two lines of code, leveraging GPU acceleration through PyTorch to handle massive datasets efficiently.

https://www.sbert.net