Argo Workflows

The open source, container-native engine for orchestrating complex parallel jobs and DAGs on Kubernetes.

This CNCF graduated project transforms Kubernetes into a high-performance workflow engine by treating every step as an isolated container. You define logic via YAML (or Python using the Hera SDK) to manage intricate Directed Acyclic Graphs (DAGs) and sequential tasks. It automates the hard parts: retries, artifact management, and scheduling across massive clusters. Engineering teams rely on it for heavy-duty data processing, machine learning pipelines, and infrastructure automation. It scales natively with your cluster to handle thousands of concurrent pods without breaking a sweat.

https://argoproj.github.io/argo-workflows/