RAG-Token

RAG-Token is a sequence-to-sequence generation model that retrieves relevant documents to predict each target token based on a distinct latent document distribution.

RAG-Token (Retrieval-Augmented Generation) optimizes language generation by performing document retrieval at the individual token level rather than the entire sequence. Developed by researchers at Facebook AI Research (FAIR), University College London, and NYU, this architecture allows the model to shift its knowledge source for every word generated. By marginalizing over a set of top-k retrieved documents (typically k=5 or k=10), RAG-Token outperforms standard parametric models on knowledge-intensive tasks like Natural Questions and Jeopardy! while maintaining a significantly lower hallucination rate.

https://arxiv.org/abs/2005.11401