nerdexam
Databricks

CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #7

The data science team has created and logged a production model using MLflow. The following code correctly imports and applies the production model to output the predictions as a new DataFrame named p

The correct answer is A. preds.write.mode("append").saveAsTable("churn_preds"). Option A (preds.write.mode('append').saveAsTable('churn_preds')) is correct for several reasons. In Databricks, saveAsTable creates a managed Delta table by default - no need to explicitly specify .format('delta'). Using mode('append') preserves all historical prediction rows, wh

Delta Lake Data Management

Question

The data science team has created and logged a production model using MLflow. The following code correctly imports and applies the production model to output the predictions as a new DataFrame named preds with the schema "customer_id LONG, predictions DOUBLE, date DATE". The data science team would like predictions saved to a Delta Lake table with the ability to compare all predictions across time. Churn predictions will be made at most once per day. Which code block accomplishes this task while minimizing potential compute costs?

Exhibits

CERTIFIED-DATA-ENGINEER-PROFESSIONAL question #7 exhibit 1
CERTIFIED-DATA-ENGINEER-PROFESSIONAL question #7 exhibit 2

Options

  • Apreds.write.mode("append").saveAsTable("churn_preds")
  • Bpreds.write.format("delta").save("/preds/churn_preds")

How the community answered

(25 responses)
  • A
    84% (21)
  • B
    16% (4)

Explanation

Option A (preds.write.mode('append').saveAsTable('churn_preds')) is correct for several reasons. In Databricks, saveAsTable creates a managed Delta table by default - no need to explicitly specify .format('delta'). Using mode('append') preserves all historical prediction rows, which is required to compare predictions across time. Since churn predictions run at most once per day, appending new rows daily builds a full history with minimal overhead. Option B writes to an unmanaged path without appending, so it would overwrite or require additional partitioning logic to support time-based comparisons. Managed tables also reduce operational overhead (no manual path management), and append operations are cheaper than full rewrites.

Topics

#Delta Lake#Spark DataFrame API#Data Ingestion#Data Persistence

Community Discussion

No community discussion yet for this question.

Full CERTIFIED-DATA-ENGINEER-PROFESSIONAL Practice