PROFESSIONAL-DATA-ENGINEER · Question #141
You architect a system to analyze seismic data. Your extract, transform, and load (ETL) process runs as a series of MapReduce jobs on an Apache Hadoop cluster. The ETL process takes days to process…
The correct answer is B. Introduce a new MapReduce job to apply sensor calibration to raw data, and ensure all other MapReduce jobs are chained after this. Explanation/Reference: It is a cleaner approach with single job to handle the calibration before the data is used in the pipeline. Second, doing this step in later stages can be complex and maintenance of those jobs in the future will become challenging.
Question
Options
- AModify the transformMapReduce jobs to apply sensor calibration before they do anything else.
- BIntroduce a new MapReduce job to apply sensor calibration to raw data, and ensure all other MapReduce jobs are chained after this.
- CAdd sensor calibration data to the output of the ETL process, and document that all users need to apply sensor calibration themselves.
- DDevelop an algorithm through simulation to predict variance of data output from the last MapReduce job based on calibration factors, and apply the correction
How the community answered
(33 responses)- A27% (9)
- B55% (18)
- C12% (4)
- D6% (2)
Explanation
Explanation/Reference: It is a cleaner approach with single job to handle the calibration before the data is used in the pipeline. Second, doing this step in later stages can be complex and maintenance of those jobs in the future will become challenging.
Topics
Community Discussion
No community discussion yet for this question.