A Databricks job has been configured with 3 tasks, each of which is a Databricks notebook. Task A does not depend on other tasks. Tasks B and C run in parallel, with each having a serial dependency…

Question

Options

• ABecause all tasks are managed as a dependency graph, no changes will be committed to the
• BTasks B and C will attempt to run as configured; any changes made in task A will be rolled back
• CUnless all tasks complete successfully, no changes will be committed to the Lakehouse; because
• DTasks B and C will be skipped; some logic expressed in task A may have been committed before
• ETasks B and C will be skipped; task A will not commit any changes because of stage failure.

• A
  4% (1)
• D
  93% (26)
• E
  4% (1)

Explanation

When a Databricks job runs multiple tasks with dependencies, the tasks are executed in a dependency graph. If a task fails, the downstream tasks that depend on it are skipped and marked as Upstream failed. However, the failed task may have already committed some changes to the Lakehouse before the failure occurred, and those changes are not rolled back automatically. Therefore, the job run may result in a partial update of the Lakehouse. To avoid this, you can use the transactional writes feature of Delta Lake to ensure that the changes are only committed when the entire job run succeeds. Alternatively, you can use the Run if condition to configure tasks to run even when some or all of their dependencies have failed, allowing your job to recover from failures and continue running.

No community discussion yet for this question.