PROFESSIONAL-DATA-ENGINEER · Question #48
You work for a manufacturing plant that batches application log files together into a single log file once a day at 2:00 AM. You have written a Google Cloud Dataflow job to process that log file. You
The correct answer is C. Create a cron job with Google App Engine Cron Service to run the Cloud Dataflow job.. Option C is correct because App Engine Cron Service provides a managed, serverless scheduling mechanism that can trigger the Dataflow job automatically at a fixed time each day (e.g., shortly after 2:00 AM), requiring no infrastructure to maintain and incurring minimal cost - you
Question
Options
- AChange the processing job to use Google Cloud Dataproc instead.
- BManually start the Cloud Dataflow job each morning when you get into the office.
- CCreate a cron job with Google App Engine Cron Service to run the Cloud Dataflow job.
- DConfigure the Cloud Dataflow job as a streaming job so that it processes the log data immediately.
How the community answered
(37 responses)- A8% (3)
- B16% (6)
- C70% (26)
- D5% (2)
Explanation
Option C is correct because App Engine Cron Service provides a managed, serverless scheduling mechanism that can trigger the Dataflow job automatically at a fixed time each day (e.g., shortly after 2:00 AM), requiring no infrastructure to maintain and incurring minimal cost - you only pay when the job actually runs.
Why the others are wrong:
- A (Dataproc) is a cluster-based Hadoop/Spark service; switching to it adds complexity and cost without solving the scheduling problem, and clusters often sit idle between jobs.
- B (Manual start) is not automated, is error-prone, depends on a person being available, and is the opposite of "inexpensive" in operational terms.
- D (Streaming job) keeps the pipeline running continuously 24/7, which is far more expensive than a batch job that runs once a day - it's the wrong tool when data only arrives once daily.
Memory tip: Think "batch data = batch job = scheduled trigger." When you see "once per day" + "as inexpensively as possible," map that to a cron-style scheduler (App Engine Cron) launching a bounded Dataflow batch job - streaming is for continuous data, not daily dumps.
Topics
Community Discussion
No community discussion yet for this question.