Apache Storm

Apache Storm is a distributed real-time computation system that processes unbounded streams of data with sub-second latency.

Storm handles high-velocity data streams by orchestrating a network of spouts (data sources) and bolts (processing units) across a distributed cluster. It guarantees message processing (at-least-once or exactly-once semantics) and scales horizontally to manage millions of messages per second. Engineering teams at Twitter, Spotify, and Flipboard use it for real-time analytics, continuous computation, and ETL tasks. The system integrates with existing queueing and database technologies (like Apache Kafka and MongoDB) to provide a robust backbone for live data processing.

https://storm.apache.org/