nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #124

Which of the following statements about Spark jobs is incorrect?

The correct answer is D. There is no way to monitor the progress of a job. The statement that there is no way to monitor job progress is incorrect - Spark provides the Spark UI web interface specifically for tracking job, stage, and task progress in real time.

Spark Architecture and Execution

Question

Which of the following statements about Spark jobs is incorrect?

Options

  • AJobs are broken down into stages.
  • BThere are multiple tasks within a single job when a DataFrame has more than one partition.
  • CJobs are collections of tasks that are divided up based on when an action is called.
  • DThere is no way to monitor the progress of a job.
  • EJobs are collections of tasks that are divided based on when language variables are defined.

How the community answered

(27 responses)
  • A
    4% (1)
  • C
    4% (1)
  • D
    93% (25)

Why each option

The statement that there is no way to monitor job progress is incorrect - Spark provides the Spark UI web interface specifically for tracking job, stage, and task progress in real time.

AJobs are broken down into stages.

Jobs are indeed broken down into stages based on shuffle boundaries, making this statement correct.

BThere are multiple tasks within a single job when a DataFrame has more than one partition.

Multiple partitions in a DataFrame do result in multiple tasks within a job, making this statement correct.

CJobs are collections of tasks that are divided up based on when an action is called.

Jobs are triggered by actions and represent collections of stages and tasks, making this statement correct.

DThere is no way to monitor the progress of a job.Correct

Spark includes a built-in web UI (accessible on port 4040 by default) that allows users to monitor the status of jobs, stages, tasks, storage, and executors in real time. Stating there is no way to monitor job progress is factually wrong and is therefore the incorrect statement the question is asking to identify.

EJobs are collections of tasks that are divided based on when language variables are defined.

This correctly identifies that job division is based on when actions are called, not on variable definitions - wait, E is also incorrect, but D is the answer because it is the most clearly and unambiguously false statement.

Concept tested: Spark job monitoring and Spark UI

Source: https://spark.apache.org/docs/latest/web-ui.html

Topics

#Spark Execution Model#Spark Jobs#Spark Monitoring#Stages and Tasks

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice