Tuesday, December 6, 2011

HBase MapReduce Read Example

Configuration config = HBaseConfiguration.create();
Job job = new Job(config, "ExampleRead");
job.setJarByClass(MyReadJob.class);     // class that contains mapper
   
Scan scan = new Scan();
scan.setCaching(500);        // 1 is the default in Scan, which will be bad for MapReduce jobs
scan.setCacheBlocks(false);  // don't set to true for MR jobs
// set other scan attrs

 
TableMapReduceUtil.initTableMapperJob(
  tableName,        // input HBase table name
  scan,             // Scan instance to control CF and attribute selection
  MyMapper.class,   // mapper
  null,             // mapper output key
  null,             // mapper output value
  job);
job.setOutputFormatClass(NullOutputFormat.class);   // because we aren't emitting anything from mapper
       
boolean b = job.waitForCompletion(true);
if (!b) {
  throw new IOException("error with job!");
}

No comments:

Post a Comment

Thank you for Commenting Will reply soon ......

Featured Posts

Enhancing Unix Proficiency: A Deeper Look at the 'Sleep' Command and Signals

Hashtags: #Unix #SleepCommand #Signals #UnixTutorial #ProcessManagement In the world of Unix commands, there are often tools that, at first ...