CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #124
A large company seeks to implement a near real-time solution involving hundreds of pipelines with parallel updates of many tables with extremely high volume and high velocity data. Which of the follow
The correct answer is A. Use Databricks High Concurrency clusters, which leverage optimized cloud storage connections. High Concurrency clusters in Databricks are designed for multiple concurrent users and workloads. They provide fine-grained sharing of cluster resources and are optimized for operations such as running multiple parallel queries and updates. This would be suitable for a solution t
Question
A large company seeks to implement a near real-time solution involving hundreds of pipelines with parallel updates of many tables with extremely high volume and high velocity data. Which of the following solutions would you implement to achieve this requirement?
Options
- AUse Databricks High Concurrency clusters, which leverage optimized cloud storage connections
- BPartition ingestion tables by a small time duration to allow for many data files to be written in
- CConfigure Databricks to save all data to attached SSD volumes instead of object storage,
- DIsolate Delta Lake tables in their own storage containers to avoid API limits imposed by cloud
- EStore all tables in a single database to ensure that the Databricks Catalyst Metastore can load
How the community answered
(37 responses)- A76% (28)
- B14% (5)
- C3% (1)
- D3% (1)
- E5% (2)
Explanation
High Concurrency clusters in Databricks are designed for multiple concurrent users and workloads. They provide fine-grained sharing of cluster resources and are optimized for operations such as running multiple parallel queries and updates. This would be suitable for a solution that involves many pipelines with parallel updates, especially with high volume and high
Topics
Community Discussion
No community discussion yet for this question.