A Structured Streaming job deployed to production has been experiencing delays during peak hours of the day. At present, during normal execution, each microbatch of data is processed in less than 3…

Question

Options

• ADecrease the trigger interval to 5 seconds; triggering batches more frequently allows idle
• BIncrease the trigger interval to 30 seconds; setting the trigger interval near the maximum
• CThe trigger interval cannot be modified without modifying the checkpoint directory; to maintain the
• DUse the trigger once option and configure a Databricks job to execute the query every 10
• EDecrease the trigger interval to 5 seconds; triggering batches more frequently may prevent

• A
  5% (3)
• B
  14% (8)
• C
  2% (1)
• D
  7% (4)
• E
  72% (42)

Explanation

Decreasing the trigger interval to 5 seconds causes Spark to fire microbatches more frequently. Because each trigger interval is shorter, less data accumulates between triggers, resulting in smaller microbatches. Smaller batches are faster to process, which helps keep individual batch execution times under the 10-second requirement during peak load. Option A is wrong because 'triggering batches more frequently allows idle' is a nonsensical or misleading justification. Option B (increase to 30s) would make batches larger and slower, violating the <10s requirement. Option C is incorrect because checkpoints can be reused when changing only the trigger interval. Option D (trigger once) is for one-time batch execution, not continuous streaming. Option E correctly identifies that more frequent, smaller batches can prevent the processing time from exceeding 10 seconds.

No community discussion yet for this question.