DAS-C01 · Question #103
A company wants to provide its data analysts with uninterrupted access to the data in its Amazon Redshift cluster. All data is streamed to an Amazon S3 bucket with Amazon Kinesis Data Firehose. An…
The correct answer is A. Increase the number of retries. Decrease the timeout value. Increase the job concurrency. To optimize fault tolerance and improve data availability for an AWS Glue job ingesting data into Amazon Redshift, the data analytics specialist should increase the number of retries, decrease the timeout value, and increase the job concurrency.
Question
A company wants to provide its data analysts with uninterrupted access to the data in its Amazon Redshift cluster. All data is streamed to an Amazon S3 bucket with Amazon Kinesis Data Firehose. An AWS Glue job that is scheduled to run every 5 minutes issues a COPY command to move the data into Amazon Redshift. The amount of data delivered is uneven throughout then day, and cluster utilization is high during certain periods. The COPY command usually completes within a couple of seconds. However, when load spike occurs, locks can exist and data can be missed. Currently, the AWS Glue job is configured to run without retries, with timeout at 5 minutes and concurrency at 1. How should a data analytics specialist configure the AWS Glue job to optimize fault tolerance and improve data availability in the Amazon Redshift cluster?
Options
- AIncrease the number of retries. Decrease the timeout value. Increase the job concurrency.
- BKeep the number of retries at 0. Decrease the timeout value. Increase the job concurrency.
- CKeep the number of retries at 0. Decrease the timeout value. Keep the job concurrency at 1.
- DKeep the number of retries at 0. Increase the timeout value. Keep the job concurrency at 1.
How the community answered
(55 responses)- A75% (41)
- B15% (8)
- C7% (4)
- D4% (2)
Why each option
To optimize fault tolerance and improve data availability for an AWS Glue job ingesting data into Amazon Redshift, the data analytics specialist should increase the number of retries, decrease the timeout value, and increase the job concurrency.
Increasing retries improves fault tolerance by allowing the job to re-attempt `COPY` commands that fail due to intermittent Redshift locks. Decreasing the timeout ensures that a stuck job fails faster and retries sooner, improving data availability. Increasing job concurrency allows subsequent scheduled runs to start and process new data even if a previous run is temporarily stalled, thus maintaining uninterrupted data flow.
Keeping the number of retries at 0 means the job will not automatically recover from intermittent failures caused by Redshift locks, directly compromising fault tolerance and data availability.
Keeping the number of retries at 0 means the job will not automatically recover from intermittent failures, and keeping concurrency at 1 could cause a backlog if a single job instance gets stuck.
Keeping the number of retries at 0 compromises fault tolerance, and increasing the timeout value would worsen data availability by prolonging the time a stuck job blocks new data processing.
Concept tested: AWS Glue job configuration for Redshift data ingestion
Source: https://docs.aws.amazon.com/glue/latest/dg/monitor-debug-configure-jobs-job-parameters.html
Topics
Community Discussion
No community discussion yet for this question.