A junior data engineer seeks to leverage Delta Lake's Change Data Feed functionality to create a Type 1 table representing all of the values that have ever been valid for all rows in a bronze table…

Question

Exhibit

Options

• AEach time the job is executed, newly updated records will be merged into the target table,
• BEach time the job is executed, the entire available history of inserted or updated records will be
• CEach time the job is executed, the target table will be overwritten using the entire history of
• DEach time the job is executed, the differences between the original and current versions are
• EEach time the job is executed, only those records that have been inserted or updated since the

• B
  94% (32)
• C
  3% (1)
• E
  3% (1)

Explanation

The code reads from the Change Data Feed without tracking which CDF versions have already been processed (no checkpoint or startingVersion tied to the last processed version). Each time the daily job runs, it reads the CDF from the beginning of the available history (e.g., starting at version 0 or the earliest available version) and merges all historical inserts and updates into the target. This means on every run, the full available history is re-read and re-merged-not just the incremental changes since the last run. A correct implementation would record the last processed CDF version and use it as startingVersion on subsequent runs.

No community discussion yet for this question.