nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #30

The code block shown below contains an error. The code block is intended to return a DataFrame containing a column openDateString, a string representation of Java's SimpleDateFormat. Identify the…

The correct answer is A. The from_unixtime() operation only accepts two parameters - the TimestampTime() arguments. from_unixtime() accepts exactly two parameters: the column containing the Unix timestamp (in seconds) and a format string (e.g., 'EEE, MMM d, yyyy h:mm a'). It always returns a string-not a timestamp type-so passing TimestampType() as a third argument is invalid and will cause…

Data Manipulation with Spark SQL and DataFrames

Question

The code block shown below contains an error. The code block is intended to return a DataFrame containing a column openDateString, a string representation of Java's SimpleDateFormat. Identify the error. Note that column openDate is of type integer and represents a date in the UNIX epoch format - the number of seconds since midnight on January 1st, 1970. An example of Java's SimpleDateFormat is "Sunday, Dec 4, 2008 1:05 PM". A sample of storesDF is displayed below:

Code block:

storesDF.withColumn("openDateString", from_unixtime(col("openDate"), "EEE, MMM d, yyyy h:mm a", TimestampType()))

Options

  • AThe from_unixtime() operation only accepts two parameters - the TimestampTime() arguments
  • BThe from_unixtime() operation only works if column openDate is of type long rather than integer -
  • CThe second argument to from_unixtime() is not correct - it should be a variant of
  • DThe from_unixtime() operation automatically places the input column in java's SimpleDateFormat
  • EThe column openDate must first be converted to a timestamp, and then the Date() function can

How the community answered

(30 responses)
  • A
    90% (27)
  • B
    3% (1)
  • C
    3% (1)
  • D
    3% (1)

Explanation

from_unixtime() accepts exactly two parameters: the column containing the Unix timestamp (in seconds) and a format string (e.g., 'EEE, MMM d, yyyy h:mm a'). It always returns a string-not a timestamp type-so passing TimestampType() as a third argument is invalid and will cause an error. The fix is to drop the TimestampType() argument: from_unixtime(col('openDate'), 'EEE, MMM d, yyyy h:mm a').

Topics

#Spark SQL Functions#Date/Time Conversion#DataFrame API

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice