Better way to get a list of all files on Hadoop, using shell.

Hadoop provides an option OIV  that is offline image viewer, which can read the Hadoop image file and output it to a output file in human readable format.

Syntax and uses:

bin/hadoop  oiv –i <Hadoop image file name> -o <output file in human readable form>

Options with this command:

There is an additional option to output file format that is defined using switch –p <format>, the formats can be: -->

 

          XML

·         Indented

     Delimited

One more command line switch -skipBlocks, prevents the tool from explicitly enumerating all of the blocks that make up a file in the namespace. its is good option for file systems that have very huge files. Enabling this option can significantly reduce the size of the output, as individual blocks are not included

Example:

bin/hadoop oiv -i fsimagefile -o output.xml -p XML

bin/hadoop oiv -i fsimagefile -o output.xml -p Indented