nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #338

You are deploying an Apache Airflow directed acyclic graph (DAG) in a Cloud Composer 2 instance. You have incoming files in a Cloud Storage bucket that the DAG processes, one file at a time. The…

The correct answer is C. 1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance. Option C is correct because triggering a DAG reactively requires the Airflow REST API (not the Cloud Composer API), which allows external systems to programmatically start a DAG run. A Cloud Function serves as the ideal bridge: Cloud Storage sends a notification when a new file…

Submitted by paula_co· Mar 30, 2026Designing data processing systems

Question

You are deploying an Apache Airflow directed acyclic graph (DAG) in a Cloud Composer 2 instance. You have incoming files in a Cloud Storage bucket that the DAG processes, one file at a time. The Cloud Composer instance is deployed in a subnetwork with no Internet access. Instead of running the DAG based on a schedule, you want to run the DAG in a reactive way every time a new file is received. What should you do?

Options

  • A
    1. Enable Private Google Access in the subnetwork, and set up Cloud Storage notifications to a Pub/Sub topic.
  • B
    1. Enable the Cloud Composer API, and set up Cloud Storage notifications to trigger a Cloud Function.
  • C
    1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance.
  • D
    1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance.

How the community answered

(22 responses)
  • A
    5% (1)
  • B
    9% (2)
  • C
    82% (18)
  • D
    5% (1)

Explanation

Option C is correct because triggering a DAG reactively requires the Airflow REST API (not the Cloud Composer API), which allows external systems to programmatically start a DAG run. A Cloud Function serves as the ideal bridge: Cloud Storage sends a notification when a new file arrives, the Cloud Function is invoked, and it calls the Airflow REST API to trigger the DAG for that specific file.

Why the distractors fail:

  • A is wrong because routing notifications to a Pub/Sub topic alone doesn't trigger a DAG - something still needs to consume the message and call the Airflow REST API; Pub/Sub without a downstream trigger is an incomplete solution.
  • B is wrong because the Cloud Composer API manages the environment itself (create, update, delete Composer instances) - it cannot trigger individual DAG runs; that's what the Airflow REST API is for.
  • D appears identical to C as written, which is likely a question formatting error; in real exam form, D would differ (e.g., a wrong API or trigger mechanism).

Memory tip: Think of it as two distinct APIs - the Cloud Composer API manages the house (the environment), while the Airflow REST API manages the work (the DAGs). Whenever you need to trigger a DAG from outside, always reach for the Airflow REST API + an event-driven intermediary like Cloud Functions.

Topics

#Cloud Composer#Event-driven Architecture#Cloud Functions#Airflow REST API

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-DATA-ENGINEER Practice