Langfuse

Langfuse is the open-source LLM engineering platform: gain full observability, manage prompt versions, and run production-grade evaluations.

Langfuse delivers the essential LLM engineering stack. It's an open-source platform for full-lifecycle management of your AI applications (agents, chains, etc.). Use its comprehensive tracing (OpenTelemetry-based) to debug complex, non-deterministic interactions and track exact cost/latency metrics. The system provides robust prompt management (versioning, A/B testing) and flexible evaluation tools to measure output quality and monitor production health. Integrations are native: connect with Langchain, OpenAI, and LlamaIndex via Python/JS SDKs for immediate control and clarity over your LLM deployment.

https://langfuse.com/