You need to create the product dimension. How should you complete the Apache Spark SQL code? To answer, select the appropriate options in the answer area.

Question

Explanation

This question tests your ability to write Apache Spark SQL code to create a product dimension table, a core component of a star schema used in dimensional data modeling. It evaluates knowledge of Spark SQL DDL syntax, Delta Lake table creation, and common dimension-loading patterns.

Approach. To create a product dimension in Apache Spark SQL (typically in a Databricks/Delta Lake context), you would use CREATE OR REPLACE TABLE ... USING DELTA to define the table, or CREATE TABLE ... AS SELECT (CTAS) to populate it from a source. If incrementally loading, a MERGE INTO statement is the correct pattern - matching on a surrogate or natural key (e.g., product_id), updating changed attributes on MATCHED, and inserting new records on NOT MATCHED. For Slowly Changing Dimension Type 2, you would add columns like is_current, effective_date, and end_date, and use window functions (ROW_NUMBER or RANK OVER PARTITION BY) combined with MERGE to version records correctly.

Concept tested. Apache Spark SQL dimensional modeling - specifically creating and populating a product dimension table using Delta Lake DDL (CREATE TABLE / CTAS) and/or MERGE INTO for upsert/SCD logic in a star schema data warehouse pattern.

Reference. Databricks Documentation: Delta Lake MERGE INTO, CREATE TABLE (Delta), and Dimensional Modeling with Apache Spark SQL

No community discussion yet for this question.