CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #1
An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with…
The correct answer is E. dbutils.widgets.text("date", "null"). When parameters are passed to a Databricks notebook via the Jobs API, the correct mechanism is Databricks Widgets. dbutils.widgets.text("date", "null") declares a text widget named 'date' with a default of 'null'; the Jobs API then injects the runtime value into this widget…
Question
An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with the following code:
df = spark.read.format("parquet").load(f"/mnt/source/(date)") Which code block should be used to create the date Python variable used in the above code block?
Options
- Adate = spark.conf.get("date")
- Binput_dict = input()
- Cimport sys
- Ddate = dbutils.notebooks.getParam("date")
- Edbutils.widgets.text("date", "null")
How the community answered
(29 responses)- B14% (4)
- C3% (1)
- D7% (2)
- E76% (22)
Explanation
When parameters are passed to a Databricks notebook via the Jobs API, the correct mechanism is Databricks Widgets. dbutils.widgets.text("date", "null") declares a text widget named 'date' with a default of 'null'; the Jobs API then injects the runtime value into this widget. The date variable is subsequently retrieved with dbutils.widgets.get("date"). Option A (spark.conf.get) reads Spark configuration properties, not Jobs API parameters. Option D (dbutils.notebooks.getParam) is used when one notebook calls another - not for Jobs API parameter passing. Options B and C are not relevant Databricks parameter-retrieval patterns.
Topics
Community Discussion
No community discussion yet for this question.