DEA-C01 · Question #235
A company builds a new data pipeline to process data for business intelligence reports. Users have noticed that data is missing from the reports. A data engineer needs to add a data quality check…
The correct answer is B. Use AWS Glue ETL jobs to perform a data quality evaluation transform on the data. Use an. AWS Glue's built-in data quality evaluation transform lets you declaratively apply DQDL rules, like IsComplete for null checks and ReferentialIntegrity for joins, directly in your ETL job. This requires minimal custom code and no separate reporting or profiling infrastructure…
Question
A company builds a new data pipeline to process data for business intelligence reports. Users have noticed that data is missing from the reports. A data engineer needs to add a data quality check for columns that contain null values and for referential integrity at a stage before the data is added to storage. Which solution will meet these requirements with the LEAST operational overhead?
Options
- AUse Amazon SageMaker Data Wrangler to create a Data Quality and Insights report.
- BUse AWS Glue ETL jobs to perform a data quality evaluation transform on the data. Use an
- CUse AWS Glue ETL jobs to perform a SQL transform on the data to determine whether requested
- DUse Amazon SageMaker Data Wrangler and a custom Python transform to create custom rules to
How the community answered
(50 responses)- A16% (8)
- B72% (36)
- C8% (4)
- D4% (2)
Explanation
AWS Glue's built-in data quality evaluation transform lets you declaratively apply DQDL rules, like IsComplete for null checks and ReferentialIntegrity for joins, directly in your ETL job. This requires minimal custom code and no separate reporting or profiling infrastructure, giving you pre-load validations with the least operational overhead.
Topics
Community Discussion
No community discussion yet for this question.