ebay's private cloud journey

Copyright © 1995 – 2016 eBay Inc. Confidential and proprietary. All Rights Reserved.

Suneet Nandwani March 23, 2016

Developing OpenStack at ScaleeBay’s Connected Commerce Cloud

2

Agenda

2

3

4

?

Introduction1

eBay Private Cloud

Challenges and Learning

Future Directions

Q&A

3

BOOSTED INNOVATION

AUTOMATION

OPERATIONAL EFFICIENCY

PRIVATECLOUD

eBay Private CloudOur Vision

AGILITYfor

TOP LINE REVENUE GROWTH

4

eBay Private CloudOur Journey

6 Years

3 Cloud Generations

2011 Bare Metal Cloud

Front End Traffic to

Cloud

eBay eBox Application

Provisioning20102012 eBay Search to

Cloud

30% eBay Traffic on

Cloud

1st GENERATION

Hadoop to Internal Cloud

Launched OpenStack

Cloud

90% eBay Traffic on

Cloud

Kubernetes Journey Starts

Availability & Reliability

2013

SW Defined Network

capability

OpenStack Manageability Scale Progr.

70% eBay Traffic on

Cloud

2014

2015 2016

2nd GENERATION

Kubernetes Investment …

3rd GENERATION

5

eBay Private CloudDeveloper Agility

DIMENSION 2012 NOWPROVISIONING

FLEXIBILITY

DEPLOYMENT

APP DEPLOYMENT

CONTINUOUS INTEGRATION

APP PROVISIONING

SELF SERVICE CAPABILITIES

TIME1 6 0 0. hours

4 0. hours

0 5. hours

0 2. hours

6

eBay Private CloudOur Cloud Scale

IaaS PaaS Frameworks Monitoring

4PBManaged Storage

Managed VMs2 1 0 0 0 0,

Managed BMs5 9 0 0 0,

Deployments per Month

5 0 0 0 0 0,

Pools1 2 0 0 0,

Pools2 7 0 0,

8 8BHits per Day

VMs6 0 0 0 0,

4MMetrics per Second

7BURL Logs per Day

7 0 0TBLogs per Day

Applications4 0 0 0,

Challenges and LearningFrom OpenStack to a Cloud Service able to support our needs

7

Network Design

Net/Info Sec

Infra. On boarding

CI

Builds

Config. Mgmt.

High Availability

Cloud Monitoring

Metrics

Log Processing

Alerting

Incident Resolution

Customer Support

SLA

Upgrades

Capacity Planning

Remediation

Auto-Scaling

Scale Out

Metering and

Chargeback

Monitoring and

Alerting

CLOUD IS A

SERVICE!

User Experience

OpenStack APIs

OpenStack native components

Required eBay developments

8

Operating Principles

Customer oriented Open source first Simplify,simplify, simplify

Automate everything, minimize manual

workObsess over

operational metricsEngineered systems

with automation patterns – no one offs

9

Lessons Learned

Hard part is turning OpenStack into a Cloud with ecosystem of tools

and processes

Upgrades

Debugging

Tech Difficulties

OpenStack for an enterprise requires long term journey and

committment

Infrastructure programmability can hold you back

Reliability at scale is low without a lot of work

Complexities

Ramp up time is high

Requires top notch engineers who are tuned into open source and

community

More engineers & dev. ops, less tradtional ops teams

Personnel

10

Future Directions

Consistent create-build-ship-run experience

Enable Containers, Micro-Services

Scalable XaaS interfaces

Declarative intent

Lights Out Management: auto everything

Efficient scheduling

Enforce policy on each interaction of an app

Adaptive to dynamic app changes

Distributed policy enforcement points

Self contained and independent

Fully automated infrastructure (last mile)

Software defined networks

Enable Cloud Native Stack

Enable Cluster Management

Programmable Application

Security

Enable SW Defined

Data Center

Agile & Efficient Available & Efficient Secure & Efficient Available & Efficient

11

Q&A

Copyright © 1995 – 2016 eBay Inc. Confidential and proprietary. All Rights Reserved.

Suneet Nandwani

Thank you