DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE · Question #17
A data engineer has three tables in a Delta Live Tables (DLT) pipeline. They have configured the pipeline to drop invalid records at each table. They notice that some data is being dropped due to…
The correct answer is D. They can navigate to the DLT pipeline page, click on each table, and view the data quality. Data engineers can identify which table is dropping records in a DLT pipeline by examining the data quality metrics available for each table in the DLT pipeline's user interface.
Question
Options
- AThey can set up separate expectations for each table when developing their DLT pipeline.
- BThey cannot determine which table is dropping the records.
- CThey can set up DLT to notify them via email when records are dropped.
- DThey can navigate to the DLT pipeline page, click on each table, and view the data quality
- EThey can navigate to the DLT pipeline page, click on the 揈rror?button, and review the present
How the community answered
(39 responses)- A5% (2)
- C3% (1)
- D82% (32)
- E10% (4)
Why each option
Data engineers can identify which table is dropping records in a DLT pipeline by examining the data quality metrics available for each table in the DLT pipeline's user interface.
While setting separate expectations is good practice for data quality, it doesn't inherently provide the visibility into *which* table dropped records without inspecting the resulting metrics.
Delta Live Tables provides comprehensive data quality observability features, making it possible to determine where records are dropped.
DLT can be configured for notifications, but identifying dropped records typically involves reviewing pipeline health and data quality metrics within the UI, not automated email notifications specific to row drops by default.
Delta Live Tables automatically captures and displays data quality metrics, including the number of dropped records, for each table in the DLT pipeline page, allowing engineers to pinpoint where data quality issues occur.
The 'Error' button typically highlights pipeline execution failures or critical errors, not the specific data quality metrics for dropped rows, which are found on individual table details.
Concept tested: Delta Live Tables data quality monitoring
Source: https://docs.databricks.com/en/delta-live-tables/monitor.html#data-quality-metrics
Topics
Community Discussion
No community discussion yet for this question.