DeepEval

DeepEval is the open-source LLM evaluation framework: unit-test your AI applications with 30+ research-backed metrics, ensuring reliability from development to production.

DeepEval is your open-source LLM evaluation framework, designed for engineering teams. It lets you 'unit test' LLM outputs (similar to Pytest), ensuring reliability for RAG, agents, and chatbots. Leverage 30+ LLM-evaluated metrics—like Hallucination and Answer Relevancy—and custom G-Eval to rigorously benchmark performance. DeepEval supports both end-to-end and component-level tracing, integrating directly into CI/CD pipelines to prevent regressions and accelerate confident deployment. The companion Confident AI cloud platform centralizes results and debugging.

https://confident-ai.com