Tuesday, December 6, 2011

HBase MapReduce Read/Write Example

Configuration config = HBaseConfiguration.create();
Job job = new Job(config,"ExampleReadWrite");
job.setJarByClass(MyReadWriteJob.class);    // class that contains mapper
                       
Scan scan = new Scan();
scan.setCaching(500);        // 1 is the default in Scan, which will be bad for MapReduce jobs
scan.setCacheBlocks(false);  // don't set to true for MR jobs
// set other scan attrs
           


TableMapReduceUtil.initTableMapperJob(
    sourceTable,      // input table
    scan,              // Scan instance to control CF and attribute selection
    MyMapper.class,   // mapper class
    null,              // mapper output key
    null,              // mapper output value
    job);
TableMapReduceUtil.initTableReducerJob(
    targetTable,      // output table
    null,             // reducer class
    job);
job.setNumReduceTasks(0);
           
boolean b = job.waitForCompletion(true);
if (!b) {
    throw new IOException("error with job!");
}

No comments:

Post a Comment

Thank you for Commenting Will reply soon ......

Featured Posts

#HighAvailability, #Scalability, #Elasticity, #Agility, #Fault Tolerance

High Availability High Availability refers to systems that are continuously operational and accessible, minimizing downtime. Imagine a resta...