A team of data engineer are adding tables to a DLT pipeline that contain repetitive expectations for many of the same data quality checks. One member of the team suggests reusing these data quality…

Question

Options

• AMaintain data quality rules in a Delta table outside of this pipeline's target schema, providing the
• BUse global Python variables to make expectations visible across DLT notebooks included in the
• CAdd data quality constraints to tables in this pipeline using an external job with access to pipeline
• DMaintain data quality rules in a separate Databricks notebook that each DLT notebook of file.

• A
  80% (48)
• B
  5% (3)
• C
  3% (2)
• D
  12% (7)

Explanation

In Delta Live Tables (DLT), data quality expectations are defined as Python dictionaries or expressions. To share rules across multiple tables and pipelines, teams can store those rules in an external Delta table (outside the pipeline's target schema) and read from it at pipeline initialization time to dynamically construct expectations. This is the recommended, production-grade approach because it decouples rule management from pipeline code and allows versioning. Option B (global Python variables) only works within a single notebook and does not scale across multiple DLT notebooks. Option C (external job modifying pipeline constraints) is not a valid DLT pattern. Option D (a separate notebook) does not enable sharing unless the rules are read from a shared, persistent store - making it functionally equivalent to Option A, but less explicit about the mechanism.

No community discussion yet for this question.