ASSOCIATE-CLOUD-ENGINEER · Question #321
The core business of your company is to rent out construction equipment at large scale. All the equipment that is being rented out has been equipped with multiple sensors that send event information e
The correct answer is D. Ingest the data into Bigtable. Create a row key based on the event timestamp.. For high-throughput sensor data with consistent, atomic, time-based retrieval, Bigtable is the ideal choice due to its massive scalability, low-latency access, and suitability for time-series data using appropriately designed row keys.
Question
Options
- ACreate files in Cloud Storage as data comes in.
- BCreate a file in Filestore per device, and append new data to that file.
- CIngest the data into Cloud SQL. Use multiple read replicas to match the throughput.
- DIngest the data into Bigtable. Create a row key based on the event timestamp.
How the community answered
(17 responses)- A12% (2)
- B6% (1)
- C29% (5)
- D53% (9)
Why each option
For high-throughput sensor data with consistent, atomic, time-based retrieval, Bigtable is the ideal choice due to its massive scalability, low-latency access, and suitability for time-series data using appropriately designed row keys.
Creating individual files in Cloud Storage for every few seconds of data from thousands of devices would lead to an overwhelming number of small files and poor performance for querying or retrieving specific time-based events consistently and atomically.
Filestore is a managed file service (NFS), primarily for applications requiring shared file systems, not designed for high-throughput, low-latency individual event ingestion and atomic operations at scale. Appending to files would also lead to contention and poor performance for thousands of concurrent writes.
Cloud SQL, while a relational database, would struggle with the extremely high write throughput (thousands of events per hour per device across many devices) and scale of this time-series workload, even with read replicas, as the primary instance would become a bottleneck for writes, and it's not optimized for this type of data ingestion pattern.
Bigtable is a massively scalable, low-latency NoSQL database ideal for high-throughput time-series data like sensor events. Creating a row key based on the event timestamp (or a combination with device ID) allows for efficient and consistent retrieval of data based on time, while Bigtable's atomic row operations ensure individual signals are stored and retrieved consistently.
Concept tested: Choosing a database for high-throughput time-series data
Source: https://cloud.google.com/bigtable/docs/overview
Topics
Community Discussion
No community discussion yet for this question.