DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE · Question #18
A data engineer has a single-task Job that runs each morning before they begin working. After identifying an upstream data issue, they need to set up another task to run a new notebook prior to the…
The correct answer is B. They can create a new task in the existing Job and then add it as a dependency of the original. To add a new task that runs prior to an existing task in a Databricks Job, the new task should be created and then configured as a dependency for the original task.
Question
Options
- AThey can clone the existing task in the existing Job and update it to run the new notebook.
- BThey can create a new task in the existing Job and then add it as a dependency of the original
- CThey can create a new task in the existing Job and then add the original task as a dependency of
- DThey can create a new job from scratch and add both tasks to run concurrently.
- EThey can clone the existing task to a new Job and then edit it to run the new notebook.
How the community answered
(33 responses)- A6% (2)
- B82% (27)
- D3% (1)
- E9% (3)
Why each option
To add a new task that runs prior to an existing task in a Databricks Job, the new task should be created and then configured as a dependency for the original task.
Cloning and updating the existing task would create a separate task or a modified version of the original, not a new preceding task with a dependency.
By creating a new task and then setting the original task to be dependent on it, the new task will execute first, and only upon its successful completion will the original task begin, fulfilling the requirement.
Adding the original task as a dependency of the new task would mean the new task runs *after* the original, which is the opposite of the desired order.
Creating a new job is unnecessary overhead for simply adding a preceding task, and running tasks concurrently would not ensure the new task runs *prior* to the original.
Cloning to a new job would separate the tasks into different job contexts, preventing them from being part of a single coordinated workflow with dependencies.
Concept tested: Databricks Jobs task dependencies
Source: https://docs.databricks.com/en/workflows/jobs/create-run-jobs.html#create-a-job
Topics
Community Discussion
No community discussion yet for this question.