nerdexam
Google

PROFESSIONAL-CLOUD-DATABASE-ENGINEER · Question #153

You are building a data warehouse on BigQuery. Sources of data include several MySQL databases located on-premises. You need to transfer data from these databases into BigQuery for analytics. You want

The correct answer is A. Use Datastream to connect to your on-premises database and create a stream. Have Datastream. The easiest, lowest-latency fully-managed path is to stand up a Datastream capture from your on- prem MySQL into BigQuery via the built-in Dataflow templates. In practice you would: - Configure Datastream to connect to your on-prem MySQL and stream change-data-capture into a Clou

Designing and Implementing Data Ingestion Solutions

Question

You are building a data warehouse on BigQuery. Sources of data include several MySQL databases located on-premises. You need to transfer data from these databases into BigQuery for analytics. You want to use a managed solution that has low latency and is easy to set up. What should you do-

Options

  • AUse Datastream to connect to your on-premises database and create a stream. Have Datastream
  • BUse Cloud Data Fusion and scheduled workflows to extract data from MySQL. Transform this data
  • CUse Database Migration Service to replicate data to a Cloud SQL for MySQL instance. Create
  • DCreate extracts from your on-premises databases periodically, and push these extracts to Cloud

How the community answered

(59 responses)
  • A
    83% (49)
  • B
    5% (3)
  • C
    10% (6)
  • D
    2% (1)

Explanation

The easiest, lowest-latency fully-managed path is to stand up a Datastream capture from your on- prem MySQL into BigQuery via the built-in Dataflow templates. In practice you would: - Configure Datastream to connect to your on-prem MySQL and stream change-data-capture into a Cloud Storage staging area. - Use the Google-provided Dataflow template to read that staged stream and write it directly into BigQuery tables. This gives you a serverless, auto-scaling, near-real-time ELT pipeline with minimal setup and no hand-rolled extracts or VMs.

Topics

#Data Ingestion#Real-time Data Transfer#BigQuery#On-premises Integration

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-CLOUD-DATABASE-ENGINEER Practice