The data engineering team maintains the following code: Assuming that this code produces logically correct results and the data in the source tables has been de-duplicated and validated, which stateme

Question

Exhibit

Options

• AA batch job will update the enriched_itemized_orders_by_account table, replacing only those
• BThe enriched_itemized_orders_by_account table will be overwritten using the current valid
• CAn incremental job will leverage information in the state store to identify unjoined rows in the
• DAn incremental job will detect if new rows have been written to any of the source tables; if new
• ENo computation will occur until enriched_itemized_orders_by_account is queried; upon query

• B
  96% (23)
• D
  4% (1)

Explanation

The correct answer is B (the table will be overwritten using the current valid records). The code (not fully visible) uses Structured Streaming with an aggregation and complete output mode. In Spark Structured Streaming, outputMode('complete') causes the entire result table to be recomputed and overwritten on every trigger. This is required for stateful aggregations because the aggregated result represents the full current state, not just incremental changes. As a result, the enriched_itemized_orders_by_account table is fully replaced on each execution rather than incrementally appended to or merged with existing data.

No community discussion yet for this question.