You are designing a pipeline that publishes application events to a Pub/Sub topic. You need to aggregate events across hourly intervals before loading the results to BigQuery for analysis. Your…

Question

Options

• ACreate a streaming Dataflow job to continually read from the Pub/Sub topic and perform the necessary aggregations using tumbling windows
• BSchedule a batch Dataflow job to run hourly, pulling all available messages from the Pub-Sub topic and performing the necessary aggregations
• CSchedule a Cloud Function to run hourly, pulling all avertable messages from the Pub/Sub topic and performing the necessary aggregations
• DCreate a Cloud Function to perform the necessary data processing that executes using the Pub/Sub trigger every time a new message is published to the topic.

• A
  86% (24)
• B
  4% (1)
• C
  7% (2)
• D
  4% (1)

Explanation

Option A is correct because a streaming Dataflow job is purpose-built for continuous, scalable stream processing - it can ingest Pub/Sub messages in real time and apply tumbling windows (fixed, non-overlapping hourly intervals) to aggregate events before loading results to BigQuery, all without manual scheduling or message backlog risk.

B is wrong because a batch Dataflow job runs on a schedule, meaning messages accumulate in Pub/Sub between runs; at large volumes this creates lag, potential message expiry (Pub/Sub retains messages for only 7 days by default), and doesn't truly scale continuously with load.

C is wrong because Cloud Functions have execution time limits (up to 9 minutes for Gen 1, 60 minutes for Gen 2) and memory constraints that make them unsuitable for processing large volumes of messages in a single hourly pull - they aren't designed for heavy batch aggregation workloads.

D is wrong because triggering a Cloud Function per message processes events individually with no aggregation window - you'd need to aggregate elsewhere, and at high throughput the per-invocation overhead makes this expensive and unscalable.

Memory tip: Think "stream + window = Dataflow" - whenever an exam question combines Pub/Sub, time-based aggregation, scalability, and BigQuery loading, a streaming Dataflow job with windowing is almost always the answer. Batch solutions (B, C) can't keep up at scale; per-event functions (D) skip the aggregation requirement entirely.

No community discussion yet for this question.