The code block shown below should use SQL to return a new DataFrame containing column storeId and column managerName from a table created from DataFrame storesDF. Choose the response that correctly…

The correct answer is E. 1. storesDF. To query a DataFrame using Spark SQL, you must first register it as a temporary view, then use the SparkSession's sql() method. The two-step process is: (1) storesDF.createOrReplaceTempView('stores') - registers the DataFrame as a temp view named 'stores' accessible within the…

Working with Spark SQL and DataFrames

Question

The code block shown below should use SQL to return a new DataFrame containing column storeId and column managerName from a table created from DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. Code block:

1.2("stores") 3.4("SELECT storeId, managerName FROM stores")

Options

A
1. spark
B
1. spark
C
1. storesDF
D
1. spark
E
1. storesDF

How the community answered

(30 responses)

B
3% (1)
D
3% (1)
E
93% (28)

Explanation

To query a DataFrame using Spark SQL, you must first register it as a temporary view, then use the SparkSession's sql() method. The two-step process is: (1) storesDF.createOrReplaceTempView('stores') - registers the DataFrame as a temp view named 'stores' accessible within the current session; (2) spark.sql('SELECT storeId, managerName FROM stores') - executes the SQL query and returns a new DataFrame. Answer E correctly places storesDF (not spark) for blank 1, createOrReplaceTempView for blank 2, spark for blank 3, and sql for blank 4. Using spark.createOrReplaceTempView (option D) would be incorrect because createOrReplaceTempView is a DataFrame method, not a SparkSession method.

Topics

#Spark SQL#DataFrames#Temporary Views#SQL Integration

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice