CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #113
A data architect has heard about lake's built-in versioning and time travel capabilities. For auditing purposes they have a requirement to maintain a full of all valid street addresses as they…
The correct answer is A. Delta Lake time travel does not scale well in cost or latency to provide a long-term versioning. The critical information is that Delta Lake time travel does not scale well for long-term auditing. By default, Delta Lake retains transaction log history for only 30 days (configurable via delta.logRetentionDuration), and old data files are vacuumed away. Relying on it as a…
Question
A data architect has heard about lake's built-in versioning and time travel capabilities. For auditing purposes they have a requirement to maintain a full of all valid street addresses as they appear in the customers table. The architect is interested in implementing a Type 1 table, overwriting existing records with new values and relying on Delta Lake time travel to support long-term auditing. A data engineer on the project feels that a Type 2 table will provide better performance and scalability. Which piece of information is critical to this decision?
Options
- ADelta Lake time travel does not scale well in cost or latency to provide a long-term versioning
- BDelta Lake time travel cannot be used to query previous versions of these tables because Type 1
- CShallow clones can be combined with Type 1 tables to accelerate historic queries for long-term
- DData corruption can occur if a query fails in a partially completed state because Type 2 tables
- EDelta Lake only supports Type 0 tables; once records are inserted to a Delta Lake table, they
How the community answered
(56 responses)- A77% (43)
- B13% (7)
- C2% (1)
- D7% (4)
- E2% (1)
Explanation
The critical information is that Delta Lake time travel does not scale well for long-term auditing. By default, Delta Lake retains transaction log history for only 30 days (configurable via delta.logRetentionDuration), and old data files are vacuumed away. Relying on it as a permanent audit log is architecturally unsound and becomes expensive at scale. This is why the data engineer's suggestion of a Type 2 table (which physically stores historical records as rows) is better for long-term, scalable auditing. Options B, C, D, and E contain inaccurate statements about Delta Lake's capabilities.
Topics
Community Discussion
No community discussion yet for this question.