CCD-410 · Question #56
In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many…
The correct answer is D. Write a custom FileInputFormat and override the method isSplitable to always return false. FileInputFormat is the base class for all file-based InputFormats. This provides a generic implementation of getSplits(JobContext). Subclasses of FileInputFormat can also override the isSplitable(JobContext, Path) method to ensure input-files are not split-up and are processed…
Question
In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?
Options
- AIncrease the parameter that controls minimum split size in the job configuration.
- BWrite a custom MapRunner that iterates over all key-value pairs in the entire file.
- CSet the number of mappers equal to the number of input files you want to process.
- DWrite a custom FileInputFormat and override the method isSplitable to always return false.
How the community answered
(30 responses)- A3% (1)
- B3% (1)
- C10% (3)
- D83% (25)
Explanation
FileInputFormat is the base class for all file-based InputFormats. This provides a generic implementation of getSplits(JobContext). Subclasses of FileInputFormat can also override the isSplitable(JobContext, Path) method to ensure input-files are not split-up and are processed as a whole by Mappers.
Topics
Community Discussion
No community discussion yet for this question.