nerdexam
Microsoft

DP-500 · Question #10

You are using a Python notebook in an Apache Spark pool in Azure Synapse Analytics. You need to present the data distribution statistics from a DataFrame in a tabular view. Which method should you…

The correct answer is D. describe. To display statistical summaries of a Spark DataFrame in a tabular view, the describe() method should be invoked.

Explore and visualize data

Question

You are using a Python notebook in an Apache Spark pool in Azure Synapse Analytics. You need to present the data distribution statistics from a DataFrame in a tabular view. Which method should you invoke on the DataFrame?

Options

  • AfreqItems
  • Bexplain
  • Crollup
  • Ddescribe

How the community answered

(41 responses)
  • A
    2% (1)
  • B
    2% (1)
  • C
    5% (2)
  • D
    90% (37)

Why each option

To display statistical summaries of a Spark DataFrame in a tabular view, the describe() method should be invoked.

AfreqItems

freqItems is used to find frequent items in a DataFrame, not to generate general statistical summaries.

Bexplain

explain is used to print the physical and logical plans of a DataFrame operation, useful for debugging and optimization, not for data distribution statistics.

Crollup

rollup is an aggregation function used to create sub-total and grand-total rows, similar to SQL's ROLLUP, and does not directly provide overall distribution statistics.

DdescribeCorrect

The describe() method on a Spark DataFrame computes and displays common summary statistics for numerical columns, such as count, mean, standard deviation, min, and max, providing a tabular view of data distribution.

Concept tested: PySpark DataFrame descriptive statistics

Source: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.describe.html

Topics

#PySpark DataFrame#Descriptive Statistics#Azure Synapse Analytics#Data Exploration

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full DP-500 Practice