Dagster

Dagster is the open-source data orchestrator and unified control plane for building, scaling, and observing reliable, software-defined data assets.

Dagster is a modern, asset-centric data orchestrator, moving beyond simple task-based execution to focus on the data assets themselves (e.g., tables, ML models). It uses Software-Defined Assets (SDAs) to declaratively define what data should exist and how it's computed, primarily in Python. This framework provides a unified control plane for the entire data lifecycle: development, testing, and production operations. Key features include integrated observability, clear data lineage, and built-in support for complex workflows like ETL/ELT and AI/ML. This asset-first approach ensures data teams can deliver insights with confidence, often accelerating velocity by 20x.

https://dagster.io