Bench marking Hadoop

You have completed setting up your Hadoop cluster?? Now its time to benchmark it, as this will help you to understand that your Hadoop is production ready, and configured properly.
So how to proceed?

The benchmarking programs are there in Hadoop*-test.jar file which you can call.

So lets try TestDFSIO : this will test the read/write performance of HDFS.

How we can run this?


bin/hadoop jar $HADOOP_HOME/hadoop-test-*.jar TestDFSIO –write  -nrFiles 3 –fileSize 1000
here nrfiles parameter will say number of files to be written and the filesize will tell how big files to be it in MB
So after sucessful running it will produce a file called TestDFSIO_results.log which we can check for the test results
Like wise there are lot of test cases in thes jar files just go and explore

Some other options are

TestDFSIO:    This tests the input output (I/O) performance of HDFS
nnbench:         This checks the NameNode hardware
mrbench:        This runs many small jobs
TeraSort:        This sorts a one terabyte of data