CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #44
The data governance team is reviewing code used for deleting records for compliance with GDPR. They note the following logic is used to delete records from the Delta Lake table named users. Assuming t
The correct answer is E. No; files containing deleted records may still be accessible with time travel until a vacuum. The code uses the DELETE FROM command to delete records from the users table that match a condition based on a join with another table called delete_requests, which contains all users that have requested deletion. The DELETE FROM command deletes records from a Delta Lake table by
Question
The data governance team is reviewing code used for deleting records for compliance with GDPR. They note the following logic is used to delete records from the Delta Lake table named users. Assuming that user_id is a unique identifying key and that delete_requests contains all users that have requested deletion, which statement describes whether successfully executing the above logic guarantees that the records to be deleted are no longer accessible and why?
Options
- AYes; Delta Lake ACID guarantees provide assurance that the delete command succeeded fully
- BNo; the Delta cache may return records from previous versions of the table until the cluster is
- CYes; the Delta cache immediately updates to reflect the latest data files recorded to disk.
- DNo; the Delta Lake delete command only provides ACID guarantees when combined with the
- ENo; files containing deleted records may still be accessible with time travel until a vacuum
How the community answered
(40 responses)- A3% (1)
- B15% (6)
- C5% (2)
- D8% (3)
- E70% (28)
Explanation
The code uses the DELETE FROM command to delete records from the users table that match a condition based on a join with another table called delete_requests, which contains all users that have requested deletion. The DELETE FROM command deletes records from a Delta Lake table by creating a new version of the table that does not contain the deleted records. However, this does not guarantee that the records to be deleted are no longer accessible, because Delta Lake supports time travel, which allows querying previous versions of the table using a timestamp or version number. Therefore, files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files from physical storage.
Topics
Community Discussion
No community discussion yet for this question.