InMobi Private Cloud: Going the

CloudPlatform Way

Iliyas Shirol

InMobi

#whoami

• BigData/Cloud Evangelist at InMobi

• Working in CloudStack Since 2012

• CloudStack Community Gardener

@iliyas_shirol

Agenda

• About InMobi

• Going Cloud Way

• Architecture & Design

• Questions

About InMobi

About InMobi

Going Cloud Way

• Development and Test Environment

• Scalable Web Applications and Content Services

• High Speed Server Resource Deployment

• And More..

Why Cloud ?

• Disconnected Tools, Servers, People

• Slow Build and Release Cycles

• Multiple Build and Test Systems Across Locations

• Home-Built Systems are Brittle, Error-Prone, and Not Scalable

• Difficult to Build, Test, and Recreate the Data

• Difficult to Audit Processes and Predict Schedules

• Time, Money Wasted on Broken Builds

• Dedicated Build/Test Servers are not Fully Utilized

Why Dev / Test in the Cloud ?

• Self-Service

Developers can provision their own environments

• Usage Measurement

We worry about VM sprawl

• Isolated Networks

Must not let dev/test interfere with the real world

• Commodity

As cheap as practical

• May also House Production Workloads

How Does a Dev/Test Cloud Look Like ?

• Budget

• Integration with Public Cloud

• Scaling

• Reconfiguration

• Legacy Hardware

Early Assessments & Results

• Leverage Existing Infrastructure

• Customization and Control

• Regulatory/Security Requirements

• Cost

Why Private Cloud ?

• ScalableGrow-as-you-need, expand across borders

• Open Source Community SupportActive user and development support base

• Advanced Networking VLAN integrated, SDN ready

• Multi-Tenancy

• Multi-Hypervisor Support

Why Citrix CloudPlatform ?

• Easy to Deploy

• Speed to Production

• Documentation

Why Citrix CloudPlatform ?

Architecture & Design

• Bring Everyone in the Room

• Know the Fine Print

• Understand Workloads

• Understand Bottlenecks

The Design Phase

• CloudStack Management Server

• MySQL Servers

• Primary & Secondary Storage

• Hypervisor Farm

Redundancy and Resilience

Management Layout & Design

• Dell PE R610’s & R720’s

• 96 – 256 GB RAM

• 40-80 CPU Cores

• 2 Advanced + 1 Basic Zones

• 1 Pod in Each Zone

• 1 – 5 Clusters in Each Pod

• XenServer 6.2

Compute

• Gigabytes

• Management Uses 1 Bonded NIC

• Public + Guest Uses 1 Bonded NIC

• Storage Uses 1 Bonded NIC (10G)

• Access & Storage Networks are Isolated

• Dell 6248

• Dell 8132

Network

• ZFS Filesystem, Based on OpenSolaris and Debian

• Linux Open Source Code Bases

( Moving to illumos)

• Software Only

• Platform, Plugins and Adapters

• Compelling Advantages over Proprietary Filesystems

Primary Storage Design (NexentaStor)

What is NexentaStor ?

• Simple Installation and Management.

• Optimized Base Configuration that Allows for:

‣ Easily reproducible configuration on a massive scale

‣ Generalized storage skill set required to maintain extensive infrastructure

‣ Little or no need for modification to achieve high performance

Advantages of NexentaStor Storage Appliance

Head Nodes – 2 Numbers

2x Dell R720 with Onboard Intel 1Gbps NIC, Perc H710

4x Intel® Xeon® Hex Core – 2.3Ghz or higher

24x 16GB 1600Mhz Memory

4x 300GB 2.5" 10K RPM SAS disks (Nexenta Syspool Configuration)

4x 2port Dell (LSI) SAS HBA

4x 10Gbps Intel NIC's low profile

Primary Storage – Hardware List

JBODs – 2 Numbers

PowerVault(TM) MD1220 Disk Storage Enclosure, 2U

24 Disk Slots

SSD Disks

30 x 2TB SSD Drives

Primary Storage – Hardware List

HeadNode-JBOD SAS Connectivity

HeadNode Network Connectivity

• Expose a Volume Using NFS/iSCSI as per the Cluster

Requirement

• Add the Respective Volume as a ‘Primary Storage’ from the

CloudPlatform Management Portal

• CloudPlatform Provisioning with NexentaStor is Automated

Using Enterprise Chef™ Cookbooks, using the Knife API Plug-ins

Provisioning Primary Storage

• GlusterFS with One-to-One Replication

• uCarp to Manage Floating IP

• Hardware Specs

• SuperMicro MBD-XS8DTE-F-O

• 32GB Memory

• 24x1TB SATA

Secondary Storage Design

GlusterFS Network Layout

• Cassandra Cluster

• Client Facing Applications

• Database Servers (PostgreSQL, MySQL, Redis etc.)

• And More …

Current Workload

• Preparation

• Compatibility Testing

• Stable Network

• Pay Attention on HA

• Staging Cluster

• Schedule Backup Solution

• Contribute Back to the Community

What We Learnt ?

• Migrate traditional data centers into Cloud

• Evaluate and build a Hyper-V zone

Steps Ahead

Questions ?

Thank You!mailto: iliyas.shirol@inmobi.com