Showing posts with label Hadoop Map-reduce. Show all posts
Showing posts with label Hadoop Map-reduce. Show all posts

Monday, August 24, 2015

Comparison between RDBMS and Map-Reduce



RDBMS
Map-Reduce
Scalability
Scale UP
Scale Out
Data Size
GB
PB and more
Read/Write
Batch and Interactive
Batch
Update Type
Write many Read many
Write once Read many
Integrity
High
Low
Structure
Structural data/ Schema first Write later
Non Structural data/ Write first Schema later
Query
SQL
No SQL and SQL support too with add on tools
Response time
Faster for Less data/ slow once size increases
Faster for more data in comparison
Note: Slowly with the new sub projects being developed for Hadoop this gap is being filled up as people are developing an abstract layer on top of Map-Reduce and YARN frameworks which takes SQL and in turn convert it to Map-Reduce/Yarn.

Monday, September 24, 2012

Apache Hadoop NextGen MapReduce (YARN)

MapReduce has undergone a complete overhaul in hadoop-0.23 and we now have, what we call, MapReduce 2.0 (MRv2) or YARN.

The fundamental idea of MRv2 is to split up the two major functionalities of the JobTracker, resource management and job scheduling/monitoring, into separate daemons. The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM). An application is either a single job in the classical sense of Map-Reduce jobs or a DAG of jobs.

The ResourceManager and per-node slave, the NodeManager (NM), form the data-computation framework. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system.

The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.

 

Check this LINK for more detail



Featured Posts

Kali Linux Remote Desktop: Access GNOME from Windows Using Native RDP

  Kali Linux + GNOME 50 + GNOME Remote Desktop + Windows Remote Desktop (MSTSC) Getting a full GNOME desktop remotely on Kali Linux can be ...