Which scheduler would you deploy to ensure that your cluster allows short jobs to finish within a reasonable time without starving long-running jobs?

The correct answer is B. Fair Scheduler. Fair scheduling is a method of assigning resources to jobs such that all jobs get, on average, an equal share of resources over time. When there is a single job running, that job uses the entire cluster. When other jobs are submitted, tasks slots that free up are assigned to…

Requirements Analysis and Design Definition

Question

Options

AFIFO Scheduler
BFair Scheduler
CCapacity Scheduler
DCompletely Fair Scheduler (CFS)

How the community answered

(14 responses)

A
7% (1)
B
71% (10)
C
7% (1)
D
14% (2)

Explanation

Fair scheduling is a method of assigning resources to jobs such that all jobs get, on average, an equal share of resources over time. When there is a single job running, that job uses the entire cluster. When other jobs are submitted, tasks slots that free up are assigned to the new jobs, so that each job gets roughly the same amount of CPU time. Unlike the default Hadoop scheduler, which forms a queue of jobs, this lets short jobs finish in reasonable time while not starving long jobs. It is also a reasonable way to share a cluster between a number of users. Finally, fair sharing can also work with job priorities - the priorities are used as weights to determine the fraction of total compute time that each job should get.

Topics

#Fair Scheduler#Job Scheduling#YARN Schedulers#Resource Fairness

Community Discussion

No community discussion yet for this question.

Full CCA-410 Practice