Hadoop

An open-source framework for distributed storage and processing of massive datasets across clusters of commodity hardware.

Hadoop serves as the backbone for big data operations by utilizing the Hadoop Distributed File System (HDFS) and the MapReduce programming model. It breaks large files into blocks (typically 128MB or 256MB) and distributes them across nodes to ensure high availability and fault tolerance. By moving computation to the data rather than the data to the computation, it enables companies like Yahoo and Facebook to manage petabytes of information efficiently. The ecosystem includes YARN for resource management and integrates seamlessly with tools like Hive and Spark for advanced analytics.

https://hadoop.apache.org/