nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #15

You need to store and analyze social media postings in Google BigQuery at a rate of 10,000 messages per minute in near real-time. Initially, design the application to use streaming inserts for individ

The correct answer is D. Estimate the average latency for data availability after streaming inserts, and always run queries after waiting twice as long.. Explanation/Reference: The data is first comes to buffer and then written to Storage. If we are running queries in buffer we will face above mentioned issues. If we wait for the bigquery to write the data to storage then we won’t face the issue. So We need to wait till it’s writt

Submitted by femi9· Mar 30, 2026Building and operationalizing data processing systems

Question

You need to store and analyze social media postings in Google BigQuery at a rate of 10,000 messages per minute in near real-time. Initially, design the application to use streaming inserts for individual postings. Your application also performs data aggregations right after the streaming inserts. You discover that the queries after streaming inserts do not exhibit strong consistency, and reports from the queries might miss in-flight data. How can you adjust your application design?

Options

  • ARe-write the application to load accumulated data every 2 minutes.
  • BConvert the streaming insert code to batch load for individual messages.
  • CLoad the original message to Google Cloud SQL, and export the table every hour to BigQuery via streaming inserts.
  • DEstimate the average latency for data availability after streaming inserts, and always run queries after waiting twice as long.

How the community answered

(36 responses)
  • A
    3% (1)
  • B
    19% (7)
  • C
    11% (4)
  • D
    67% (24)

Explanation

Explanation/Reference: The data is first comes to buffer and then written to Storage. If we are running queries in buffer we will face above mentioned issues. If we wait for the bigquery to write the data to storage then we won’t face the issue. So We need to wait till it’s written to storage.

Topics

#BigQuery streaming consistency#eventual consistency#query latency#data freshness

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full PROFESSIONAL-DATA-ENGINEER Practice