Highly AvailableNG Hadoop MapReduce

Arinto MurdopoMário Almeida

Umit CavusStrahinja Lazetic

Supervisor: Jim Dowling

AimHigh Availability in YARN

AimHA with ZooKeeper

AimHA with NDB

Outcome

● Highly Available implementation of YARN○ with Zookeeper before the hadoop team○ with NDB MySql-Cluster before Jim kicks us out

● (Secondary goal) make RM stateless so we can have more than one running at each moment.○ So we can have failover when there is failure in a

RM

● (Eventual goal) Get a job!

Expected graphs (fictional values)

Expected graphs (fictional values)