Showing posts with label hadoop components. Show all posts
Showing posts with label hadoop components. Show all posts

Sunday, September 23, 2012

Demystifying Hadoop concepts Series: Safe mode


safemode
What is is safe mode of hadoop, may time we come across this exception “ org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.SafeModeException”  or some other exceptions Which contains safe mode in it Smile .

First let me tell what Safe mode is in context to Hadoop : as we all know Name node contains fsimage (metadata) of the data present on the cluster, which can be large or small based on the size of the cluster and the size of date present on the cluster, so when the name node starts it loads this fsimage and the edit logs from the disk in the Primary memory RAM for fast processing, and after loading it waits for data nodes to report about the present on those data nodes, so during this process that is loading the fsimage and edit logs and waiting for data nodes to report about the data block in safe mode, which is a read only mode for name node this is done to maintain the consistency of the data present, this is just like saying “ i will not receive any thing till i know what i already have”. And during this period no modification to the file blocks are allowed as to maintain the correctness of the data.

Sunday, April 15, 2012

Different components of hadoop system



·         Hadoop Common – The common utilities that support the other Hadoop sub – projects
·         HDFS – A distributed file system that provides high throughput access to application data
·         MapReduce – A software framework for distributed processing of large data sets on compute clusters.
·         Avro – A data serialization system.

Featured Posts

Kali Linux Remote Desktop: Access GNOME from Windows Using Native RDP

  Kali Linux + GNOME 50 + GNOME Remote Desktop + Windows Remote Desktop (MSTSC) Getting a full GNOME desktop remotely on Kali Linux can be ...