Reading sequence file/ compressed file / or TextRecordInputStream file from hadoop

As we know sequence file are binary files with key/value pair specially build for hadoop. If we want to read file on hadoop we have option of cat which will not show the output of sequence or compressed file correctly in this case we can use -text option with hadoop command which follows:

hadoop fs -text   <file path name/filename>

which will show they correct output of the sequence file content.

Demystifying Hadoop concepts Series: Safe mode

What is is safe mode of hadoop, may time we come across this exception “ org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.SafeModeException”  or some other exceptions Which contains safe mode in it .

First let me tell what Safe mode is in context to Hadoop : as we all know Name node contains fsimage (metadata) of the data present on the cluster, which can be large or small based on the size of the cluster and the size of date present on the cluster, so when the name node starts it loads this fsimage and the edit logs from the disk in the Primary memory RAM for fast processing, and after loading it waits for data nodes to report about the present on those data nodes, so during this process that is loading the fsimage and edit logs and waiting for data nodes to report about the data block in safe mode, which is a read only mode for name node this is done to maintain the consistency of the data present, this is just like saying “ i will not receive any thing till i know what i already have”. And during this period no modification to the file blocks are allowed as to maintain the correctness of the data.

Hadoop Shell Commands

namenode -format	format the DFS filesystem
secondarynamenode	run the DFS secondary namenode
namenode	run the DFS namenode
datanode	run a DFS datanode
dfsadminmradmin	run a DFS admin client
mradmin	run a Map-Reduce admin client