Showing posts with label hadoop scheduling task. Show all posts
Showing posts with label hadoop scheduling task. Show all posts

Thursday, March 14, 2013

Adding Scheduler to Hadoop Cluster

 

As we know we we execute task or jobs on hadoop it follows FIFO Scheduling, but if you are in multi user hadoop environment the you will need better scheduler for the consistency and correctness of the task scheduling.

Hadoop comes with other schedulers too those are:

Fair Schedulers : This defines pools and over time; each pool gets around the same amount of resources.

Capacity Schedulers : This defines queues, and each queue has a guaranteed capacity. The capacity scheduler shares computer resources allocated to a queue with other queues if those resources are not in use.

For changing the scheduler you need to take your cluster offline and make some configuration changes, first make sure that the correct scheduler jar files are there. In older version of hadoop you need to put the jar file if not ther in lib directory but from hadoop 1 these jars available in the lib folder and if you are using the newer hadoop good news for you Smile

Steps will be:

Featured Posts

Kali Linux Remote Desktop: Access GNOME from Windows Using Native RDP

  Kali Linux + GNOME 50 + GNOME Remote Desktop + Windows Remote Desktop (MSTSC) Getting a full GNOME desktop remotely on Kali Linux can be ...