When we set the number of reduce tasks as zero, reduce tasks will not be executed. The output of the mapper will be copied to the hdfs and it will be the output of the job. Suppose 10 mappers were spawned for a job, if we set the number of reduce tasks as zero, we will get 10 output files.
The output files will be with a name similar to part-m-00000, part-m-00001 ..... part-m-00009.
We can set the number of reduce tasks as zero either from the program or from the commandline.
In the program we can set this by setting the following configuration
job.setNumReduceTasks(0);
From the command line also we can achieve the same result by using the property below
-Dmapred.reduce.tasks=0
The output files will be with a name similar to part-m-00000, part-m-00001 ..... part-m-00009.
We can set the number of reduce tasks as zero either from the program or from the commandline.
In the program we can set this by setting the following configuration
job.setNumReduceTasks(0);
From the command line also we can achieve the same result by using the property below
-Dmapred.reduce.tasks=0
No comments:
Post a Comment