savanna project update jan 2014

OpenStack Data Processing (Savanna) Project Update

January 16, 2014Sergey Lukjanov, Data Processing Program PTL

Havana Release Overview● 69 blueprints implemented● ~150 bugs fixed● 880 code commits from 31 people● 3000+ code reviews● officially incubated for Icehouse!● more info: https://launchpad.net/savanna/havana

● 0.2 and 0.3 releases● https://wiki.openstack.org/wiki/Savanna/ReleaseNotes

Havana Highlights● from PoC to community-backed project● contributors

○ core: Mirantis, Hortonworks, Red Hat○ plus: HP, IBM, UnitedStack, Rackspace

● two plugins:○ Vanilla○ Hortonworks Data Platform

● clustering● Elastic Data Processing● OpenStack Dashboard plugin

Havana Highlights: Clustering● cluster provisioning● configuration management (templates)● cluster scaling● data nodes anti-affinity● 200 nodes in 5 mins

Havana Highlights: EDP● API to execute MapReduce jobs without exposing

details of underlying infrastructure ○ pluggable data sources: Swift○ supported job types: jar, Pig, Hive○ integration with Oozie for workflow management

● user-friendly UI for ad-hoc analytics queries based on Hive or Pig

Icehouse Overview● working on i-2 dev milestone now (release next week)● so far:

○ 24 blueprints implemented○ ~43 fixed○ 220 code commits from 26 people○ 1530+ code reviews

● more info: https://launchpad.net/savanna/icehouse

Icehouse Highlights● new contributors

○ eNovance, Samsung, Intel, EPAM Systems, Zurich University of Applied Science

● Heat integration for orchestration● Intel Distribution of Hadoop plugin● Gating Savanna (async)

○ DevStack integration○ Tempest tests (base API)

● CLI proof of concept

Icehouse plans● Heat by default for orchestration● Tempest: scenarios, CLI ● v2 API based on Pecan/WSME● complete CLI implementation● guest agents PoC

Q&A

email: slukjanov@mirantis.comirc: SergeyLukjanov on Freenode