nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #175

Which of the following code blocks returns a DataFrame containing a column month, an integer representation of the month from column openDate from DataFrame storesDF? Note that column openDate is of t

The correct answer is D. (storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp")). Because openDate is an integer representing Unix epoch seconds, it must first be cast to Timestamp (not Date) before date functions like month() can interpret it correctly. Choice D casts to Timestamp with col('openDate').cast('Timestamp') and then extracts the month. Casting dir

Transforming and manipulating data within Spark DataFrames, specifically handling date and time data types by converting UNIX epoch to a timestamp.

Question

Which of the following code blocks returns a DataFrame containing a column month, an integer representation of the month from column openDate from DataFrame storesDF? Note that column openDate is of type integer and represents a date in the UNIX epoch format - the number of seconds since midnight on January 1 st, 1970. A sample of storesDF is displayed below:

Options

  • AstoresDF.withColumn("month", getMonth(col("openDate")))
  • BstoresDF.withColumn("month", substr(col("openDate"), 4, 2))
  • C(storesDF.withColumn("openDateFormat", col("openDate").cast("Date"))
  • D(storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp"))
  • EstoresDF.withColumn("month", month(col("openDate")))

How the community answered

(24 responses)
  • A
    13% (3)
  • C
    8% (2)
  • D
    75% (18)
  • E
    4% (1)

Explanation

Because openDate is an integer representing Unix epoch seconds, it must first be cast to Timestamp (not Date) before date functions like month() can interpret it correctly. Choice D casts to Timestamp with col('openDate').cast('Timestamp') and then extracts the month. Casting directly to Date (choice C) may not correctly interpret Unix epoch seconds. Choice E applies month() directly to an integer column, which will not work. Choice A uses a non-existent getMonth() function. Choice B uses substr() which is for string manipulation, not numeric epoch values.

Topics

#Spark DataFrames#Date/Time Functions#Type Casting#UNIX Epoch

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice