A company is Running Apache Spark on an Amazon EMR cluster. The Spark job writes to an Amazon S3 bucket. The job fails and returns an HTTP 503 "Slow Down" AmazonS3Exception error. Which actions will…

Question

Options

• AAdd additional prefixes to the S3 bucket
• BReduce the number of prefixes in the S3 bucket
• CIncrease the EMR File System (EMRFS) retry limit
• DDisable dynamic partition pruning in the Spark configuration for the cluster
• EAdd more partitions in the Spark configuration for the cluster

• A
  79% (19)
• B
  13% (3)
• D
  4% (1)
• E
  4% (1)

Explanation

An HTTP 503 'Slow Down' error from Amazon S3 indicates request rate throttling. S3 automatically scales to handle high request rates, but it partitions based on key prefixes. Two actions resolve this: (A) Adding additional prefixes distributes write requests across more S3 partitions, increasing the aggregate throughput your job can sustain without being throttled. (C) Increasing the EMRFS retry limit allows the Spark job to back off and retry throttled requests rather than immediately failing after 30 minutes of partial success. Option B (reducing prefixes) would worsen throttling. Option D (disabling dynamic partition pruning) affects query planning for reads, not S3 write throughput. Option E (adding more Spark partitions) would actually increase the number of concurrent S3 write requests, potentially making throttling worse.

No community discussion yet for this question.