nerdexam
Databricks

CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #105

The following table consists of items found in user carts within an e-commerce website. The following MERGE statement is used to update this table using an updates view, with schema evaluation enabled

The correct answer is D. The new nested field is added to the target schema, and files underlying existing records are. With schema evolution enabled in Databricks Delta tables, when a new field is added to a record through a MERGE operation, Databricks automatically modifies the table schema to include the new field. In existing records where this new field is not present, Databricks will insert

Schema Management in Delta Lake

Question

The following table consists of items found in user carts within an e-commerce website. The following MERGE statement is used to update this table using an updates view, with schema evaluation enabled on this table. How would the following update be handled?

Options

  • AThe update is moved to separate ''restored'' column because it is missing a column expected in
  • BThe new restored field is added to the target schema, and dynamically read as NULL for existing
  • CThe update throws an error because changes to existing columns in the target schema are not
  • DThe new nested field is added to the target schema, and files underlying existing records are

How the community answered

(62 responses)
  • A
    2% (1)
  • B
    3% (2)
  • C
    6% (4)
  • D
    89% (55)

Explanation

With schema evolution enabled in Databricks Delta tables, when a new field is added to a record through a MERGE operation, Databricks automatically modifies the table schema to include the new field. In existing records where this new field is not present, Databricks will insert NULL values for that field. This ensures that the schema remains consistent across all records in the table, with the new field being present in every record, even if it is NULL for records that did not originally include it.

Topics

#Delta Lake#Schema Evolution#MERGE Statement#Data Engineering

Community Discussion

No community discussion yet for this question.

Full CERTIFIED-DATA-ENGINEER-PROFESSIONAL Practice