Kafka Hadoop Integration

What Is Hadoop?

A large-scale distributed batch processing framework that use to parallelize the data processing among many nodes and also addresses the challenges for distributed computing, including big data, is what we call Hadoop.

Basically, it works on the principle of the MapReduce framework, which is introduced by Google. It offers a simple interface for the parallelization as well as the distribution of large-scale computations. In addition, it has its own distributed data filesystem which we call as HDFS (Hadoop Distributed File System). To understand HDFS, it splits the data into small pieces (called blocks) and further distributes it to all the nodes in any typical Hadoop cluster. Moreover, it creates the replication of these small pieces of data as well as it stores them to ensure that the data is available from another node if any node is down. https://goo.gl/q7RNNx