You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static reference data from BigQuery. The reference data is small enough to fit in memory on a single worker. The…

Question

Options

• ABatch job, PubSubIO, side-inputs
• BStreaming job, PubSubIO, JdbcIO, side-outputs
• CStreaming job, PubSubIO, BigQueryIO, side-inputs
• DStreaming job, PubSubIO, BigQueryIO, side-outputs

• A
  7% (3)
• B
  5% (2)
• C
  70% (30)
• D
  19% (8)

Explanation

C is correct because Pub/Sub is an unbounded data source requiring a streaming job, PubSubIO reads from it, BigQueryIO loads the static reference data, and side-inputs are the purpose-built mechanism for broadcasting small, read-only datasets (like a lookup table) to all workers during a transform - exactly the "small enough to fit in memory" scenario described.

A is wrong because Pub/Sub is inherently unbounded/real-time; a batch job cannot continuously consume it.

B is wrong on two counts: JdbcIO targets JDBC-compliant relational databases, not BigQuery; and side-outputs (tagged outputs) split a stream into multiple output branches - they have nothing to do with enrichment lookups.

D is wrong because side-outputs route processed records to different destination collections (e.g., good vs. bad records); they do not inject reference data into a transform.

Memory tip: Think of the words literally - side-inputs bring data into a transform for enrichment (reads), while side-outputs send data out to separate destinations (writes). If the question involves a lookup or join with static data, it's always side-inputs.

No community discussion yet for this question.