CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #121
CERTIFIED-DATA-ENGINEER-PROFESSIONAL Question #121: Real Exam Question with Answer & Explanation
Sign in or unlock CERTIFIED-DATA-ENGINEER-PROFESSIONAL to reveal the answer and full explanation for question #121. The question stem and answer options stay visible for context.
Question
A Structured Streaming job deployed to production has been resulting in higher than expected cloud storage costs. At present, during normal execution, each microbatch of data is processed in less than 3s; at least 12 times per minute, a microbatch is processed that contains 0 records. The streaming write was configured using the default trigger settings. The production job is currently scheduled alongside many other Databricks jobs in a workspace with instance pools provisioned to reduce start-up time for jobs with batch execution. Holding all other variables constant and assuming records need to be processed in less than 10 minutes, which adjustment will meet the requirement?
Options
- ASet the trigger interval to 3 seconds; the default trigger interval is consuming too many records
- BIncrease the number of shuffle partitions to maximize parallelism, since the trigger interval cannot
- CSet the trigger interval to 10 minutes; each batch calls APIs in the source storage account, so
- DSet the trigger interval to 500 milliseconds; setting a small but non-zero trigger interval ensures
- EUse the trigger once option and configure a Databricks job to execute the query every 10
Unlock CERTIFIED-DATA-ENGINEER-PROFESSIONAL to see the answer
You've previewed enough free CERTIFIED-DATA-ENGINEER-PROFESSIONAL questions. Unlock CERTIFIED-DATA-ENGINEER-PROFESSIONAL for full answers, explanations, the timed quiz mode, progress tracking, and the master PDF. Question stem and options stay visible so you can still see what's on the exam.