WebHosting

Tuesday, December 6, 2011

HBase MapReduce Read/Write Example

Configuration config = HBaseConfiguration.create();
Job job = new Job(config,"ExampleReadWrite");
job.setJarByClass(MyReadWriteJob.class);    // class that contains mapper
                       
Scan scan = new Scan();
scan.setCaching(500);        // 1 is the default in Scan, which will be bad for MapReduce jobs
scan.setCacheBlocks(false);  // don't set to true for MR jobs
// set other scan attrs
           


TableMapReduceUtil.initTableMapperJob(
    sourceTable,      // input table
    scan,              // Scan instance to control CF and attribute selection
    MyMapper.class,   // mapper class
    null,              // mapper output key
    null,              // mapper output value
    job);
TableMapReduceUtil.initTableReducerJob(
    targetTable,      // output table
    null,             // reducer class
    job);
job.setNumReduceTasks(0);
           
boolean b = job.waitForCompletion(true);
if (!b) {
    throw new IOException("error with job!");
}

No comments:

Post a Comment

Thank you for Commenting Will reply soon ......

Featured Posts

How to Enable and Configure Remote Desktop (RDP) on Ubuntu 24.04.4 LTS

Remote Desktop Protocol (RDP) support in Ubuntu has become considerably easier with the GNOME Remote Desktop stack included in modern Ubuntu...