A data analyst has developed a query that runs against Delta table. They want help from the data engineering team to implement a series of tests to ensure the data returned by the query is clean…

The correct answer is C. spark.sql. spark.sql() is the PySpark method that accepts a SQL string, executes it against the Spark SQL engine, and returns the result as a PySpark DataFrame. This makes it the natural bridge between SQL logic and Python-based testing: the data engineer can pass the analyst's exact SQL…

Submitted by daniela_cl· Apr 18, 2026ELT with Spark SQL and Python

Question

A data analyst has developed a query that runs against Delta table. They want help from the data engineering team to implement a series of tests to ensure the data returned by the query is clean. However, the data engineering team uses Python for its tests rather than SQL. Which of the following operations could the data engineering team use to run the query and operate with the results in PySpark?

Options

ASELECT * FROM sales
Bspark.delta.table
Cspark.sql
DThere is no way to share data between PySpark and SQL.
Espark.table

How the community answered

(25 responses)

C
96% (24)
D
4% (1)

Explanation

spark.sql() is the PySpark method that accepts a SQL string, executes it against the Spark SQL engine, and returns the result as a PySpark DataFrame. This makes it the natural bridge between SQL logic and Python-based testing: the data engineer can pass the analyst's exact SQL query into spark.sql() and then use DataFrame operations or assertions in Python to validate the results. spark.table() loads an existing named table as a DataFrame but does not execute an arbitrary SQL query. spark.delta.table is not a valid PySpark API. SQL and PySpark are fully interoperable in Spark, so option D is incorrect.

Topics

#PySpark#Spark SQL#DataFrame API#Query execution

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Practice