CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #116
The data governance team is reviewing user for deleting records for compliance with GDPR. The following logic has been implemented to propagate deleted requests from the user_lookup table to the…
The correct answer is A. No; files containing deleted records may still be accessible with time travel until a BACUM. The DELETE operation in Delta Lake is ACID compliant, which means that once the operation is successful, the records are logically removed from the table. However, the underlying files that contained these records may still exist and be accessible via time travel to older…
Question
The data governance team is reviewing user for deleting records for compliance with GDPR. The following logic has been implemented to propagate deleted requests from the user_lookup table to the user aggregate table. Assuming that user_id is a unique identifying key and that all users have requested deletion have been removed from the user_lookup table, which statement describes whether successfully executing the above logic guarantees that the records to be deleted from the user_aggregates table are no longer accessible and why?
Exhibit
Options
- ANo; files containing deleted records may still be accessible with time travel until a BACUM
- BYes; Delta Lake ACID guarantees provide assurance that the DELETE command successed fully
- CNo; the change data feed only tracks inserts and updates not deleted records.
- DNo; the Delta Lake DELETE command only provides ACID guarantees when combined with the
- EYes; the change data feed uses foreign keys to ensure delete consistency throughout the
How the community answered
(44 responses)- A75% (33)
- B7% (3)
- C2% (1)
- D14% (6)
- E2% (1)
Explanation
The DELETE operation in Delta Lake is ACID compliant, which means that once the operation is successful, the records are logically removed from the table. However, the underlying files that contained these records may still exist and be accessible via time travel to older versions of the table. To ensure that these records are physically removed and compliance with GDPR is maintained, a VACUUM command should be used to clean up these data files after a certain retention period. The VACUUM command will remove the files from the storage layer, and after this, the records will no longer be accessible.
Topics
Community Discussion
No community discussion yet for this question.