Google
PROFESSIONAL-DATA-ENGINEER · Question #166
You have data pipelines running on BigQuery, Cloud Dataflow, and Cloud Dataproc. You need to perform health checks and monitor their behavior, and then notify the team managing the pipelines if they…
The correct answer is A. Export the information to Cloud Stackdriver, and set up an Alerting policy. Explanation/Reference: Monitoring does not only provide you with access to Dataflow-related metrics, but also lets you to create alerting policies and dashboards so you can chart time series of metrics and choose to be notified when these metrics reach specified values.
Submitted by fernanda_arg· Mar 30, 2026Building and operationalizing data processing systems
Question
You have data pipelines running on BigQuery, Cloud Dataflow, and Cloud Dataproc. You need to perform health checks and monitor their behavior, and then notify the team managing the pipelines if they fail. You also need to be able to work across multiple projects. Your preference is to use managed products of features of the platform. What should you do?
Options
- AExport the information to Cloud Stackdriver, and set up an Alerting policy
- BRun a Virtual Machine in Compute Engine with Airflow, and export the information to Stackdriver
- CExport the logs to BigQuery, and set up App Engine to read that information and send emails if you find a failure in the logs
- DDevelop an App Engine application to consume logs using GCP API calls, and send emails if you find a failure in the logs
How the community answered
(34 responses)- A91% (31)
- B6% (2)
- C3% (1)
Explanation
Explanation/Reference: Monitoring does not only provide you with access to Dataflow-related metrics, but also lets you to create alerting policies and dashboards so you can chart time series of metrics and choose to be notified when these metrics reach specified values.
Topics
#Cloud Monitoring#Stackdriver alerting#pipeline health checks#multi-project monitoring
Community Discussion
No community discussion yet for this question.