Azure Databricks

A unified, Apache Spark-based analytics platform optimized for Azure to accelerate data engineering, machine learning, and SQL workloads.

Azure Databricks integrates the Photon engine and Delta Lake to deliver high-performance data processing within a Lakehouse architecture. It provides a collaborative workspace for Python, Scala, R, and SQL users to build ETL pipelines and train ML models using managed MLflow. Native integration with Azure Data Lake Storage (ADLS Gen2) and Microsoft Entra ID ensures secure, enterprise-grade scaling for petabyte-scale datasets.

https://azure.microsoft.com/en-us/products/databricks