Seldon Core

An open-source framework to package, deploy, and monitor machine learning models at scale on Kubernetes.

Seldon Core transforms trained models (TensorFlow, PyTorch, or Scikit-learn) into production-ready microservices within Kubernetes clusters. It handles the heavy lifting of inference through advanced deployment strategies like Canary updates and A/B testing while managing complex request graphs. By providing out-of-the-box support for Prometheus monitoring and Jaeger tracing, it ensures high-throughput environments remain observable. The framework supports over 2.5 million installs, making it a standard for teams moving from experimental notebooks to hardened, scalable REST/gRPC endpoints.

https://github.com/SeldonIO/seldon-core