Ever heard of Big Data? Hadoop is the solution, and this is what it is.
So you must’ve wondered at least once till now what happens to all the pictures and paragraphs and tweets that we upload everyday. Where does it all go?
The modern day problem of the storage and management of the enormous volumes of data generated every second is called Big Data. So, contrary to our belief, Big Data is actually just the name of the problem. Now, if there is a problem, there must be a solution as well, right?
Hadoop is a project developed by Apache. It brings a great an innovative solution to this problem of big data handling. As we know, laptops and servers have a limited amount of storage space available, and none of it is enough to store all of Facebook or Instagram’s data. So what Hadoop does is, it makes a single network using multiple computers/ servers by clustering them in one unit. So you basically end up with a system where you can control all the data from a single device known as the master node (aka The Name Node).
Making use of a master-slave architecture, with the Name and Data nodes, Hadoop facilitates Distributed Storage Clusters, which is nothing but storing all the data in multiple slave-nodes (servers) to allow the distributed processing of large data sets across them using simple programming models. Its designs allows us to use it for a single or for thousands of machines with easy scalability, each offering local computation and storage.
The utility of Hadoop is truly easy to understand and it brings the following to the table:
- Easy scalability
- Automatic detection and handling of failures in the application layer
- High reliability
- Distributed processing of big data using simple programming models
- Storage of thousands of TBs within seconds
- High speed and efficiency
Having joint the ARTH program just 2 days back, by LinuxWorld, all this was explained in such a simple way by my mentor, Mr. Vimal Daga. So I would like to thank him for imparting right knowledge in layman terms.
To all the curious readers, hope you found this a little helpful. Have a great day, and keep learning!
