Which of the following Spark properties is used to configure the maximum size of an automatically broadcasted DataFrame when performing a join?

Question

Options

• Aspark.sql.broadcastTimeout
• Bspark.sql.autoBroadcastJoinThreshold
• Cspark.sql.shuffle.partitions
• Dspark.sql.inMemoryColumnarStorage.batchSize
• Espark.sql.adaptive.skewedJoin.enabled

• B
  90% (43)
• C
  4% (2)
• D
  2% (1)
• E
  4% (2)

Explanation

spark.sql.autoBroadcastJoinThreshold controls the maximum byte size threshold for a DataFrame to be automatically broadcast to all worker nodes during a join, avoiding a costly shuffle. The default value is 10MB. If a DataFrame's size is below this threshold, Spark will choose a broadcast hash join strategy automatically. spark.sql.broadcastTimeout controls how long a broadcast can take before it times out. spark.sql.shuffle.partitions controls the number of partitions for shuffles. The other options are unrelated to broadcast size.

No community discussion yet for this question.