What four functions do scheduling algorithms perform on hadoop cluster?

Question

Options

• ARun jobs at periodic times of the day
• BReduce the job latencies in environment with multiple jobs of different sizes
• CAllow multiple users to share clusters in a predictable policy-guided manner
• Dsupport the implementation of service-level agreements for multiple cluster users
• Eallow short jobs to complete even when large, long jobs (consuming a lot of resources are running)
• FReduce the total amount of computation necessary to complete a job.
• GEnsure data locality by ordering map tasks so that they run on data local maps slots

• A
  84% (16)
• C
  5% (1)
• D
  5% (1)
• G
  5% (1)

Explanation

Hadoop schedulers Since the pluggable scheduler was implemented, several scheduler algorithms have been developed for it. / FIFO scheduler The original scheduling algorithm that was integrated within the JobTracker was called FIFO. In FIFO scheduling, a JobTracker pulled jobs from a work queue, oldest job first. This schedule had no concept of the priority or size of the job, but the approach was simple to implement and / Fair scheduler The core idea behind the fair share scheduler was to assign resources to jobs such that on average over time, each job gets an equal share of the available resources. The result is that jobs that require less time are able to access the CPU and finish intermixed with the execution of jobs that require more time to execute. This behavior allows for some interactivity among Hadoop jobs and permits greater responsiveness of the Hadoop cluster to the variety of job types submitted. The fair scheduler was developed by Facebook. / Capacity scheduler The capacity scheduler shares some of the principles of the fair scheduler but has distinct differences, too. First, capacity scheduling was defined for large clusters, which may have multiple, independent consumers and target applications. For this reason, capacity scheduling provides greater control as well as the ability to provide a minimum capacity guarantee and share excess capacity among users. The capacity scheduler was developed by Yahoo!. In capacity scheduling, instead of pools, several queues are created, each with a configurable number of map and reduce slots. Each queue is also assigned a guaranteed capacity (where the overall capacity of the cluster is the sum of each queue's capacity). * The introduction of the pluggable scheduler was yet another evolution in cluster computing with Hadoop. The pluggable scheduler permits the use (and development) of schedulers optimized for the particular workload and application. The new schedulers have also made it possible to create multi-user data warehouses with Hadoop, given the ability to share the overall Hadoop infrastructure with multiple users and organizations.

No community discussion yet for this question.