A company analyzes data in a data lake every quarter to perform inventory assessments. A data engineer uses AWS Glue DataBrew to detect any personally identifiable formation (PII) about customers…

Question

Options

• AManually review the data for custom PII categories.
• BImplement custom data quality rules in DataBrew. Apply the custom rules across datasets.
• CDevelop custom Python scripts to detect the custom PII categories. Call the scripts from
• DImplement regex patterns to extract PII information from fields during extract transform, and load

• A
  9% (3)
• B
  82% (28)
• C
  6% (2)
• D
  3% (1)

Explanation

AWS Glue DataBrew allows users to create custom data quality rules, including rules for detecting non-standard PII categories. By implementing custom rules, the data engineer can automate the detection of PII (including the custom categories) across multiple datasets without needing to manually review or develop custom scripts. This provides a solution with the least operational overhead, as it leverages the existing DataBrew framework and automation Manually reviewing the data is highly labor-intensive, error-prone, and not scalable, especially across multiple datasets. While possible, writing custom Python scripts adds operational overhead, as it requires developing, maintaining, and integrating scripts with DataBrew. This increases complexity compared to implementing custom rules directly within DataBrew. While regex can be useful, incorporating it directly into ETL processes adds operational complexity and doesn't leverage the capabilities of DataBrew, which is already designed for data cleaning and quality validation.

No community discussion yet for this question.