DP-420 · Question #124
You are designing an Azure Cosmos DB for NoSQL solution to store data from IoT devices. Writes from the devices will occur every second. Data will be retained indefinitely. The following is a sample…
The correct answer is B. Create a new synthetic key that contains deviceId and timestamp. To achieve even distribution and avoid hot partitions for high-frequency IoT writes in Azure Cosmos DB, a synthetic partition key combining deviceId and timestamp is ideal.
Question
You are designing an Azure Cosmos DB for NoSQL solution to store data from IoT devices. Writes from the devices will occur every second. Data will be retained indefinitely. The following is a sample of the data. You need to select a partition key that meets the following requirements for writes:
- Minimizes the partition skew
- Avoids capacity limits
- Avoids hot partitions
What should you do?
Exhibit
Options
- AUse deviceManufacturer as the partition key.
- BCreate a new synthetic key that contains deviceId and timestamp.
- CCreate a new synthetic key that contains deviceId and deviceManufacturer.
- DUse deviceId as the partition key.
How the community answered
(45 responses)- A7% (3)
- B76% (34)
- C13% (6)
- D4% (2)
Why each option
To achieve even distribution and avoid hot partitions for high-frequency IoT writes in Azure Cosmos DB, a synthetic partition key combining `deviceId` and `timestamp` is ideal.
Using `deviceManufacturer` as the partition key is likely to lead to partition skew and hot partitions because there are usually far fewer manufacturers than devices, concentrating writes onto a small number of logical partitions.
Creating a synthetic partition key by combining `deviceId` and `timestamp` (or a time bucket) provides high cardinality and ensures writes are distributed across a large number of logical partitions over time. This approach effectively minimizes partition skew and avoids hot partitions, which is crucial for frequent writes from many devices.
Combining `deviceId` and `deviceManufacturer` increases cardinality but doesn't introduce the necessary time-based distribution to prevent hot partitions when a single device writes frequently, as all writes from a given `deviceId` would still go to the same logical partition.
Using `deviceId` as the partition key would lead to hot partitions because each device writes every second, meaning all writes from a single `deviceId` would land on the same logical partition, potentially exceeding its throughput limit.
Concept tested: Azure Cosmos DB partition key selection for IoT workloads
Source: https://learn.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#choose-partitionkey
Topics
Community Discussion
No community discussion yet for this question.
