Which of the following describes a partition?

Question

Options

• AA partition is the amount of data that fits in a single executor.
• BA partition is an automatically-sized segment of data that is used to create efficient logical plans.
• CA partition is the amount of data that fits on a single worker node.
• DA partition is a portion of a Spark application that is made up of similar jobs.
• EA partition is a collection of rows of data that fit on a single machine in a cluster.

• A
  2% (1)
• C
  2% (1)
• D
  3% (2)
• E
  94% (59)

Explanation

A Spark partition is a logical chunk of data - specifically, a collection of rows that resides on a single machine in the cluster at any given time. One task processes exactly one partition. Option A is wrong - partitions are not defined by executor memory capacity. Option B is wrong - partitions are not automatically sized for logical plans; logical plans operate independently of partitioning. Option C is wrong - a worker node can hold many partitions simultaneously across multiple executor slots. Option D is wrong - partitions are data subdivisions, not groupings of jobs. Option E is correct: a partition is simply a subset of rows that fits on (and is processed by) a single machine.

No community discussion yet for this question.