DAS-C01 · Question #141
A bank is using Amazon Managed Streaming for Apache Kafka (Amazon MSK) to populate real- time data into a data lake. The data lake is built on Amazon S3, and data must be accessible from the data…
The correct answer is B. Create an Amazon CloudWatch alarm that monitors the KafkaDataLogsDiskUsed metric. To prevent an Amazon MSK cluster from running out of disk space given recent volume increases and an alert, a data analytics specialist should create an Amazon CloudWatch alarm that monitors the KafkaDataLogsDiskUsed metric at a proactive threshold.
Question
A bank is using Amazon Managed Streaming for Apache Kafka (Amazon MSK) to populate real- time data into a data lake. The data lake is built on Amazon S3, and data must be accessible from the data lake within 24 hours. Different microservices produce messages to different topics in the cluster. The cluster is created with 8 TB of Amazon Elastic Block Store (Amazon EBS) storage and a retention period of 7 days. The customer transaction volume has tripled recently and disk monitoring has provided an alert that the cluster is almost out of storage capacity. What should a data analytics specialist do to prevent the cluster from running out of disk space1?
Options
- AUse the Amazon MSK console to triple the broker storage and restart the cluster
- BCreate an Amazon CloudWatch alarm that monitors the KafkaDataLogsDiskUsed metric
- CCreate a custom Amazon MSK configuration
- DTriple the number of consumers to ensure that data is consumed as soon as it is added to a topic.
How the community answered
(31 responses)- A10% (3)
- B81% (25)
- C3% (1)
- D6% (2)
Why each option
To prevent an Amazon MSK cluster from running out of disk space given recent volume increases and an alert, a data analytics specialist should create an Amazon CloudWatch alarm that monitors the `KafkaDataLogsDiskUsed` metric at a proactive threshold.
Tripling the broker storage is a reactive solution for an imminent disk space issue, but the question asks what to do to *prevent* running out of space, which implies a proactive measure. Additionally, MSK storage scaling typically does not require a cluster restart.
While an immediate storage increase might be needed, to *prevent* the cluster from *running out* of disk space in the future, the specialist should create or refine a CloudWatch alarm on the `KafkaDataLogsDiskUsed` metric. This allows for proactive notification at an earlier threshold (e.g., 80% instead of 90%), enabling timely scaling of broker storage or adjustment of retention policies before capacity becomes critical.
Creating a custom Amazon MSK configuration can adjust Kafka parameters like log retention, but it is not the primary mechanism for scaling underlying EBS storage capacity to handle increased data volume, nor is it the most direct 'preventative' measure against full disks compared to monitoring.
Tripling the number of consumers helps process data faster but does not reduce the amount of disk space Amazon MSK uses to retain data according to its configured retention period; Kafka stores data regardless of consumer speed for its log retention settings.
Concept tested: Proactive monitoring and capacity management for Amazon MSK
Source: https://docs.aws.amazon.com/msk/latest/developerguide/monitoring.html
Topics
Community Discussion
No community discussion yet for this question.