Tuesday, December 6, 2011

HBase MapReduce Read/Write Example

Configuration config = HBaseConfiguration.create();
Job job = new Job(config,"ExampleReadWrite");
job.setJarByClass(MyReadWriteJob.class);    // class that contains mapper
                       
Scan scan = new Scan();
scan.setCaching(500);        // 1 is the default in Scan, which will be bad for MapReduce jobs
scan.setCacheBlocks(false);  // don't set to true for MR jobs
// set other scan attrs
           


TableMapReduceUtil.initTableMapperJob(
    sourceTable,      // input table
    scan,              // Scan instance to control CF and attribute selection
    MyMapper.class,   // mapper class
    null,              // mapper output key
    null,              // mapper output value
    job);
TableMapReduceUtil.initTableReducerJob(
    targetTable,      // output table
    null,             // reducer class
    job);
job.setNumReduceTasks(0);
           
boolean b = job.waitForCompletion(true);
if (!b) {
    throw new IOException("error with job!");
}

No comments:

Post a Comment

Thank you for Commenting Will reply soon ......

Featured Posts

🌫️ Project Title: "Fog Buster – AI-Powered Visibility Enhancement System"

🔍 Project Vision: To design a device that allows vehicles (cars, trucks) and aircraft (planes, helicopters, drones) to see clearly during f...