A Delta Lake table representing metadata about content from user has the following schema: Based on the above schema, which column is a good candidate for partitioning the Delta Table?

The correct answer is A. Date. Good partition columns for Delta Lake tables have low-to-moderate cardinality and are frequently used as filter predicates in queries. Date satisfies both criteria: it has bounded cardinality (finite calendar dates), enables efficient partition pruning for time-range queries…

Optimizing Delta Lake Tables

Question

A Delta Lake table representing metadata about content from user has the following schema:

Based on the above schema, which column is a good candidate for partitioning the Delta Table?

Options

ADate
BPost_id
CUser_id
DPost_time
Elatitude

How the community answered

(59 responses)

A
85% (50)
B
7% (4)
C
2% (1)
D
2% (1)
E
5% (3)

Explanation

Good partition columns for Delta Lake tables have low-to-moderate cardinality and are frequently used as filter predicates in queries. Date satisfies both criteria: it has bounded cardinality (finite calendar dates), enables efficient partition pruning for time-range queries, and distributes data evenly over time. Post_id (Option B) and User_id (Option C) are high-cardinality identifiers that would create millions of tiny partitions - a classic over-partitioning anti-pattern. Post_time (Option D) is a high-precision timestamp with effectively unbounded cardinality. Latitude (Option E) is a continuous floating-point value with very high cardinality and would create an enormous number of partitions with minimal pruning benefit.

Topics

#Delta Lake#Data Partitioning#Performance Optimization#Data Modeling

Community Discussion

No community discussion yet for this question.

Full CERTIFIED-DATA-ENGINEER-PROFESSIONAL Practice