nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #286

You want to create a machine learning model using BigQuery ML and create an endpoint for hosting the model using Vertex AI. This will enable the processing of continuous streaming data in near-real ti

The correct answer is D. Create a Pub/Sub topic and send all vendor data to it. Use Dataflow to process and sanitize the Pub/Sub data and stream it to BigQuery.. D is correct because it follows the canonical GCP real-time streaming architecture: Pub/Sub acts as a durable, scalable message broker that decouples the multiple vendor sources from downstream processing, and Dataflow provides a fully managed, autoscaling pipeline specifically d

Submitted by lucia.co· Mar 30, 2026Designing data processing systems

Question

You want to create a machine learning model using BigQuery ML and create an endpoint for hosting the model using Vertex AI. This will enable the processing of continuous streaming data in near-real time from multiple vendors. The data may contain invalid values. What should you do?

Options

  • ACreate a new BigQuery dataset and use streaming inserts to land the data from multiple vendors. Configure your BigQuery ML model to use the "ingestion"
  • BUse BigQuery streaming inserts to land the data from multiple vendors where your BigQuery dataset ML model is deployed.
  • CCreate a Pub/Sub topic and send all vendor data to it. Connect a Cloud Function to the topic to process the data and store it in BigQuery.
  • DCreate a Pub/Sub topic and send all vendor data to it. Use Dataflow to process and sanitize the Pub/Sub data and stream it to BigQuery.

How the community answered

(36 responses)
  • A
    8% (3)
  • B
    6% (2)
  • C
    3% (1)
  • D
    83% (30)

Explanation

D is correct because it follows the canonical GCP real-time streaming architecture: Pub/Sub acts as a durable, scalable message broker that decouples the multiple vendor sources from downstream processing, and Dataflow provides a fully managed, autoscaling pipeline specifically designed for exactly-once stream processing and data sanitization - ensuring invalid values are handled before clean data lands in BigQuery for ML training and inference via the Vertex AI endpoint.

A is wrong because "ingestion" is not a real BigQuery ML configuration option - this answer fabricates a feature that doesn't exist, and BigQuery streaming inserts alone provide no mechanism to validate or sanitize invalid values mid-stream.

B is wrong for the same sanitization reason: streaming inserts dump data directly into BigQuery with no transformation layer, so invalid values would corrupt your dataset before the ML model ever sees it.

C is wrong because Cloud Functions are event-driven and stateless - they're ill-suited for high-throughput, continuous stream processing at scale; Dataflow is purpose-built for this workload with windowing, aggregation, and fault tolerance that Cloud Functions can't reliably provide.

Memory tip: Remember "Pub/Sub → Dataflow → BigQuery" as GCP's golden streaming trio - whenever a question involves real-time ingestion + transformation + analysis, this pipeline is almost always the answer.

Topics

#Streaming Data Processing#Data Ingestion#Data Transformation#BigQuery ML

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-DATA-ENGINEER Practice