You has a cluster running with the Fail Scheduler enabled. There are currently no jobs running on the cluster you submit a job A, so that only job A is running on the cluster. A while later, you…

Question

Options

• AWhen job A gets submitted, it consumes all the task slot
• BWhen job A gets submitted, it doesn't consume all the task slot
• CWhen job B gets submitted, Job A has to finish first, before job it can get scheduled.
• DWhen job B gets submitted, it will get assigned tasks, while job A continues to run with fewer tasks.

• A
  13% (5)
• B
  5% (2)
• C
  11% (4)
• D
  71% (27)

Explanation

Fair scheduling is a method of assigning resources to jobs such that all jobs get, on average, an equal share of resources over time. When there is a single job running, that job uses the entire cluster. When other jobs are submitted, tasks slots that free up are assigned to the new jobs, so that each job gets roughly the same amount of CPU time. Unlike the default Hadoop scheduler, which forms a queue of jobs, this lets short jobs finish in reasonable time while not starving long jobs. It is also a reasonable way to share a cluster between a number of users. Finally, fair sharing can also work with job priorities - the priorities are used as weights to determine the fraction of total compute time that each job should get.

No community discussion yet for this question.