CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #108
A DLT pipeline includes the following streaming tables: Raw_lot ingest raw device measurement data from a heart rate tracking device. Bgm_stats incrementally computes user statistics based on BPM meas
The correct answer is D. Set the pipelines, reset, allowed property to false on raw_iot. Setting pipelines.reset.allowed = false on the raw_iot table prevents the DLT pipeline from truncating and reloading that table during a full refresh or pipeline reset. This preserves any manually deleted or updated records in raw_iot. Meanwhile, downstream tables like bpm_stats
Question
A DLT pipeline includes the following streaming tables:
Raw_lot ingest raw device measurement data from a heart rate tracking device. Bgm_stats incrementally computes user statistics based on BPM measurements from raw_lot. How can the data engineer configure this pipeline to be able to retain manually deleted or updated records in the raw_iot table while recomputing the downstream table when a pipeline update is run?
Options
- ASet the skipChangeCommits flag to true on bpm_stats
- BSet the SkipChangeCommits flag to true raw_lot
- CSet the pipelines, reset, allowed property to false on bpm_stats
- DSet the pipelines, reset, allowed property to false on raw_iot
How the community answered
(31 responses)- A3% (1)
- B10% (3)
- C16% (5)
- D71% (22)
Explanation
Setting pipelines.reset.allowed = false on the raw_iot table prevents the DLT pipeline from truncating and reloading that table during a full refresh or pipeline reset. This preserves any manually deleted or updated records in raw_iot. Meanwhile, downstream tables like bpm_stats can still be recomputed from whatever data exists in raw_iot. Setting skipChangeCommits (options A/B) controls how streaming tables handle change data capture events, which is a different concern. Setting reset.allowed = false on bpm_stats (option C) would protect the downstream table, not the source table.
Topics
Community Discussion
No community discussion yet for this question.