Sunday, January 15, 2012

How to Avoid Sorting and Partitioning in Map only Job

We can define MapReduce job with no reducer. In this case, all the mappers write their outputs under specified job output directory. So; there will be no sorting and no partitioning.
Just set the number of reduces to 0.

job.setNumReduceTasks(0);

No comments:

Post a Comment