CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #101
Which statement regarding spark configuration on the Databricks platform is true?
The correct answer is A. Spark configuration properties set for an interactive cluster with the Clusters UI will impact all. Spark configuration properties set at the cluster level via the Clusters UI are applied to the SparkContext when the cluster starts, and they affect all SparkSessions created on that cluster. This is because all SparkSessions on a cluster share the same underlying SparkContext. O
Question
Which statement regarding spark configuration on the Databricks platform is true?
Options
- ASpark configuration properties set for an interactive cluster with the Clusters UI will impact all
- BWhen the same spar configuration property is set for an interactive to the same interactive
- CSpark configuration set within an notebook will affect all SparkSession attached to the same
- DThe Databricks REST API can be used to modify the Spark configuration properties for an
- ESpark configuration properties can only be set for an interactive cluster by creating a global init
How the community answered
(27 responses)- A96% (26)
- C4% (1)
Explanation
Spark configuration properties set at the cluster level via the Clusters UI are applied to the SparkContext when the cluster starts, and they affect all SparkSessions created on that cluster. This is because all SparkSessions on a cluster share the same underlying SparkContext. Option C is incorrect-Spark configuration set within a single notebook only affects that notebook's SparkSession, not all sessions on the cluster. Option E is incorrect because cluster-level Spark configs can be set directly in the cluster configuration UI, not only through init scripts. Option D is partially true but incomplete and misleading as written.
Topics
Community Discussion
No community discussion yet for this question.