nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #204

You currently have a single on-premises Kafka cluster in a data center in the us-east region that is responsible for ingesting messages from IoT devices globally. Because large parts of globe have…

The correct answer is C. An IoT gateway connected to Cloud Pub/Sub, with Cloud Dataflow to read and process the messages from Cloud Pub/Sub. Option C is correct because it replaces the single on-premises Kafka cluster with a fully managed, globally scalable stack: an IoT gateway (Cloud IoT Core or a regional MQTT broker) ingests device messages into Cloud Pub/Sub, which is designed to absorb massive, bursty traffic…

Submitted by asante_acc· Mar 30, 2026Designing data processing systems

Question

You currently have a single on-premises Kafka cluster in a data center in the us-east region that is responsible for ingesting messages from IoT devices globally. Because large parts of globe have poor internet connectivity, messages sometimes batch at the edge, come in all at once, and cause a spike in load on your Kafka cluster. This is becoming difficult to manage and prohibitively expensive. What is the Google-recommended cloud native architecture for this scenario?

Options

  • AEdge TPUs as sensor devices for storing and transmitting the messages.
  • BCloud Dataflow connected to the Kafka cluster to scale the processing of incoming messages.
  • CAn IoT gateway connected to Cloud Pub/Sub, with Cloud Dataflow to read and process the messages from Cloud Pub/Sub.
  • DA Kafka cluster virtualized on Compute Engine in us-east with Cloud Load Balancing to connect to the devices around the world.

How the community answered

(38 responses)
  • A
    5% (2)
  • B
    18% (7)
  • C
    66% (25)
  • D
    11% (4)

Explanation

Option C is correct because it replaces the single on-premises Kafka cluster with a fully managed, globally scalable stack: an IoT gateway (Cloud IoT Core or a regional MQTT broker) ingests device messages into Cloud Pub/Sub, which is designed to absorb massive, bursty traffic without manual scaling, and Cloud Dataflow provides a managed Apache Beam pipeline to process those messages reliably.

  • A is wrong because Edge TPUs are machine-learning inference chips, not message-queuing or connectivity solutions - they don't solve the ingestion spike problem.
  • B is wrong because adding Dataflow downstream of the existing Kafka cluster only helps with processing; it does nothing to absorb the upstream ingestion spikes hitting your on-prem cluster, and it keeps the expensive single-cluster bottleneck in place.
  • D is wrong because simply virtualizing Kafka on Compute Engine and adding a load balancer moves the problem to the cloud but doesn't eliminate it - you still own cluster management, capacity planning, and cost spikes; it's lift-and-shift, not cloud-native.

Memory tip: Think "ingest → buffer → process" as the cloud-native IoT pattern. Pub/Sub is Google's infinitely scalable buffer (the "shock absorber" for bursty IoT traffic), and Dataflow is the managed processor - whenever a question mentions spiky, global IoT ingestion, that pairing is the Google-recommended answer.

Topics

#IoT#Messaging#Stream Processing#Scalability#Cloud Native Architecture

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-DATA-ENGINEER Practice