PROFESSIONAL-DATA-ENGINEER · Question #187
You used Cloud Dataprep to create a recipe on a sample of data in a BigQuery table. You want to reuse this recipe on a daily upload of data with the same schema, after the load job with variable…
The correct answer is D. Export the Cloud Dataprep job as a Cloud Dataflow template, and incorporate it into a Cloud Composer job. Option D is correct because Cloud Dataprep jobs run on Cloud Dataflow under the hood, so exporting as a Dataflow template allows Cloud Composer (managed Apache Airflow) to orchestrate the pipeline with proper dependency management - specifically, Composer can wait for the…
Question
Options
- ACreate a cron schedule in Cloud Dataprep.
- BCreate an App Engine cron job to schedule the execution of the Cloud Dataprep job.
- CExport the recipe as a Cloud Dataprep template, and create a job in Cloud Scheduler.
- DExport the Cloud Dataprep job as a Cloud Dataflow template, and incorporate it into a Cloud Composer job.
How the community answered
(28 responses)- A14% (4)
- B4% (1)
- C11% (3)
- D71% (20)
Explanation
Option D is correct because Cloud Dataprep jobs run on Cloud Dataflow under the hood, so exporting as a Dataflow template allows Cloud Composer (managed Apache Airflow) to orchestrate the pipeline with proper dependency management - specifically, Composer can wait for the upstream BigQuery load job to complete before triggering the Dataflow/Dataprep job, regardless of how long the load takes.
Options A, B, and C are wrong for the same core reason: they are all time-based scheduling approaches (fixed cron or Cloud Scheduler), which cannot account for a load job with variable execution time - the Dataprep job might fire before the data is ready, or waste idle time waiting for a worst-case window.
Option C has an additional flaw: Cloud Dataprep doesn't export "templates" in the way described - you export as a Cloud Dataflow template, making the terminology itself a red flag.
Memory tip: Whenever an exam question mentions "variable execution time" or "after job X completes," that's a signal that time-based cron is disqualified - you need event-driven or dependency-aware orchestration, which is exactly what Cloud Composer (Airflow DAGs) provides. Think: variable time = Composer.
Topics
Community Discussion
No community discussion yet for this question.