You have a cluster running with the fair scheduler enabled and configured. You submit multiple jobs to the cluster. Each job is assigned to a pool. What are the two key points to remember about how…

Question

Options

• AEach pool gets 1/M of the total available tasks slots, where M is the no. of nodes in the cluster
• BPools are assigned priorites.pools with higher priorities an executed b4 pools with lower priorities
• CEach pool gets 1/N of the total available tasks slots, where N is the no of jobs running on the cluster
• DPools get a dynamically-allocated share of the available task slots (subject to additional constraints)
• EEach pools share of the tasks slots remains static within the execution of any individual job
• FEach pools share of task slots may change throughout the course of job execution

• B
  6% (2)
• C
  11% (4)
• D
  80% (28)
• E
  3% (1)

Explanation

By default, the fair scheduler will evenly split available slots between all pools that have "demand." In this context, demand means there are tasks in the pool that are currently eligible to run. Tasks within a pool may be freely reordered by the scheduler, and often are. As task trackers heartbeat advertising available slots, the scheduler looks for a task that wants to process data on that task tracker. This is largely how it achieves data locality. There are some other details around how tasks are are selected out of a pool (e.g. tasks that will process larger amounts of data receive preference, etc.). Pools can have both weights as well as minimum shares allocated to them. The min share is how you guarantee slots for a pool. The scheduler will always allocate min share slots to all pools first. Any left over slots are spread evenly across pools. If a pool has a weight, it will receive more or less slots, based on the value of the weight, during this extra or "free share" assignment process. Interesting tidbit about weights: the job's priority is really just changing the weight of its tasks. So far this only refers to how slots are allocated across pools. It is also possible that there are multiple jobs submitted to the same pool. The scheduler actually just runs another instance of the fair scheduler within each pool and the default behavior that applies across pools now applies to multiple jobs within a single pool; slots are simply split evenly across all jobs in the pool. This mostly works as expected. If you open two sqlplus sessions to Oracle and ran two queries at the same time, you'd expect them both to make progress, albeit contending for available resources.

No community discussion yet for this question.