The data engineering team maintains the following code: Assuming that this code produces logically correct results and the data in the source table has been de-duplicated and validated, which…

Question

Exhibit

Options

• AThe silver_customer_sales table will be overwritten by aggregated values calculated from all
• BA batch job will update the gold_customer_lifetime_sales_summary table, replacing only those
• CThe gold_customer_lifetime_sales_summary table will be overwritten by aggregated values
• DAn incremental job will leverage running information in the state store to update aggregate values
• EAn incremental job will detect if new rows have been written to the silver_customer_sales table; if

• C
  94% (15)
• D
  6% (1)

Explanation

This code is using the pyspark.sql.functions library to group the silver_customer_sales table by customer_id and then aggregate the data using the minimum sale date, maximum sale total, and sum of distinct order ids. The resulting aggregated data is then written to the gold_customer_lifetime_sales_summary table, overwriting any existing data in that table. This is a batch job that does not use any incremental or streaming logic, and does not perform any merge or update operations. Therefore, the code will overwrite the gold table with the aggregated values from the silver table every time it is executed.

No community discussion yet for this question.