Moving Enterprise Application Dev/Test to VMware’s
Internal Private Cloud – Architecture,
Implementation and Integration
Thirumalesh Reddy, VMware
Padmaja Vrudhula, VMware
VSVC4948
#VSVC4948
2 2
Executive Summary
Key Lesson Learned
Invest in Agility, and Service Quality and Cost will Improve
AppOps Team Deploy integrated, complex SDLC instances to support 600 developers.
Challenge Process is manual, siloed, slow, unreliable. Reduces developer efficiency. Increases risk.
Two Fundamentally Different Options
1. Fix the “human middleware” on traditional infrastructure
2. Replace and automate on private cloud SDDC
Results From Choice to Replace and Automate on SDDC
Process time – dropped from 4 weeks to 36 hours
Developer productivity – increased 20% or more
Project schedule risk - eliminated
Annual infrastructure and operating costs - reduced by $6M annually
3 3
Session Overview
Agenda
Executive Summary
Fundamental Challenges: VMware IT
Options
Cloud Automation and Management – Architecture, Implementation:
VMware IT
Cost Savings & Delight Customers
Key Takeaways
Session Goals
Automate Workload Provisioning life-cycle
Understand new paradigm of instance refresh – Just re-provision.
Understand resource optimization - Different Service profiles
Dramatically reduce Problem Resolution – Integrated Monitoring from
the outset
4 4
Corporate IT Application Group
Manage portfolio of enterprise
applications used by global
business functions
Team
27 DevOps engineers
Customer
600 developers
Role
Provision 16 different dev/test instances that
include 130+ app components.
Infrastructure footprint
~4000 non-production VMs
~500 production VMs
~290 physical hosts
~4 Petabytes of storage
Enterprise Application Portfolio
SaaS 65
IT tools 50
Business 100
Total 215
5 5
AppOps provisions environments across SDLC
Support 30 to 50 major development projects per year
Team of 27 engineers manually builds each Dev/Test instance
Each project needs Dev/Test multiple times per project
Enterprise App Development Project - 9 months
20 Major Steps
3 to 5 Weeks in Traditional Virtualized Environment
Request for
Dev/Test
Infrastructure
Verification
Hardware
Setup
Build VMs –
new or clone
DNS
Entries
Install,
Setup,
Configure
Workload
Database
Refresh
Latest Code
Deployment
Load
Balancer
Entries
Web Server
Configuration
Firewall
Changes
External
Interface &
Integration
PPM
Tasks
Workload
Monitoring
Setup
Security – VM
access control
Functional
Testing
Environmental
Testing
Production Dev Test UAT Stage Load
Test
6 6
Human Middleware Problem
Variable Quality
Variations in
calendar and
service quality
Schedule Risk
Late projects cause
domino effect
with constrained
resource
Unpredictable
“No we are busy”
is response to
many developer
requests
Disruptive
Developer must
work around
3+ weeks gaps up
to 5x per project
Customer View
7 7
Human Middleware Problem – AppOps team view
Global Team
Management
Project manage
around PTO,
holidays,
variable skills
Capacity
Constrained
Only 4-6 projects
in parallel
Slow and
Error Prone
Many manual steps.
Ticketing systems.
Human error.
Handoffs
Silos. Globally
distributed teams.
Multiple application
experts.
8 8
Two Fundamentally Different Options
Fix
The “human middleware”
on traditional infrastructure
Replace and Automate
End-to-end provisioning
on SDDC Private Cloud
Option 1 Option 2
9 9
Project OneCloud - Explosive Tenant Growth
Corp IT DevOps = Tenant #4
Very low cost per VM
“Cloud first” policy in IT
Corp IT
Ent App
dev/test
Hands
On Labs Hol.vmware.com
Services &
Support Customer environment
reproduction
Sales
Engineering
Demo Pods
VMworld 2013
Management
BU Field
Testing
TechSummit
2013
Tech Ops
Mini R&D
Cloud
vSEL
Private Cloud IaaS Software Defined Data Center
June
2012
Jan.
2013
Today End
2013
2014
Launched
Built on
vCloud Suite
4 tenants
10,000 VMs
9 tenants
38,000 VMs
12 tenants
50,000 VMs
More
services
Timeline
10 10
Application Ops Requirements Vary By Workload Type
…are different than Ops
requirements for this App
Ops requirements for this instance…
Corp IT– App Dev/Test
Provision: Application Stack
Manage: Yes
Duration: 3+ months
Provision: VM
Manage: No
Duration: 3 weeks
One App/Sales – Demo Pod
11 11
Private Cloud IaaS Software Defined Data Center
Bring Your Own - Application Ops
Three-tier Ops Model
Different tenants
Different Application Ops
Application Ops (Provided by Tenant)
Now an infrastructure service consumer. Provisioning.
Monitoring. Configuring. Upgrades. Maintenance.
Many typical ops tasks still required.
Infrastructure Ops (Provided by OneCloud infrastructure team)
Network, storage, compute availability. Deliver to SLA.
Tenant/Service Ops (Provided by OneCloud service team)
Common service definitions, SLA, tenant onboarding, tenant management
Private Cloud IaaS Software Defined Data Center
12 12
Bring Your Own - Cloud Automation and Management
Tenant - needs different service
levels, automation and
management capabilities
IaaS - needs automation and
management capabilities
Service Manager
Decides what goes in service catalog.
Service Catalog
Mechanism to request service.
Policy
Logic used to guide automation.
Cloud Automation and Management
Manage workloads and underlying services.
Tenant 1 Tenant 2 Tenant 3
Private Cloud IaaS Software Defined Data Center
13 13
Two Project Goals
Transition
to Private Cloud
Automate
the Process
Key Dependency
Need SDDC to automate the process
14 14
• Cloud Automation
• Dev, Test instances
• Policy based provisioning
• Storage Mgmt.
• 30+ App Blueprints
• 500+ VM’s
• Non-prod Environments
• One Cloud
Management
Cloud Virtual Infrastructure
Provisioning Automation
Cloud Storage Virtualization
Done
Q3 – Q4 12
• Cloud Automation & Management
• Dev, Test, UAT instances
• Scaling/Upgrades
• Policy based provisioning
• Storage Mgmt.
• Security Mgmt.
• Monitoring & Analytics
• VM Asset Mgmt.
• 50+ App Blueprints
• 2500+ VM’s
• Non-prod Environments
• One Cloud
• VCHS IaaS Validation
Management
Cloud Virtual Infrastructure
Provisioning/Scaling/Upgrade Automation
Monitoring
Cloud Storage Virtualization
Cloud Security
Done
Q1 – Q2 13
Service Catalog
Phased Project Approach
• Stage and load test instances
• Service catalog
• Performance management
• Network, Storage & Security
Virtualization
• 80+ App Blueprints
• 3500+ VM’s
• VCHS IaaS
Management
Cloud Virtual Infrastructure
Monitoring
Cloud Storage Virtualization
Cloud Network Virtualization
Cloud Security
Provisioning/Scaling/Upgrade Automation
Performance Mgmt
Service Catalog
In-progress
Q3 – Q4 13 • Cloud Automation Scaling/Upgrades
• Cloud Storage & Network Mgmt. & Scaling
• Cloud Security Mgmt.
• Cloud Performance Mgmt.
• Usage & Charge-back
• Analytic and Correlations
• 100+ App Blueprints
• 4500+ VM’s
• All Non-prod & Prod Environments
• One Cloud/VCHS IaaS
Cloud Virtual Infrastructure
Management
Monitoring
Cloud Storage Virtualization
Cloud Network Virtualization
Cloud Security
Provisioning/Scaling/Upgrade Automation
Performance Management
Big Data Operational and Biz Analytic and Correlations
Service Catalog
2014
15 15
vCenter Orchestrator
VMware Application Director (VCAC Enterprise)
Dev Test UAT Stage Mgmt.
Cloud Providers
App Blueprints (130+)
VMware vCloud Director / VCAC
VMware vCloud Automation Center
Access Control
Provisioning Policies
Service Catalog
vCAC Workflows
Cloud Administrator
Blueprint
Manager
AppOps/ Biz/Dev Consumer
One Cloud - Private Cloud IaaS Software Defined Data Center
Deployment Plans
Automation and Management – Based on vCloud
vCloud Suite
16 16
Cloud Automation and Management - Extensibility
Load Balancer
(F5/vShield)
IPAM
(Men & Mice)
LDAP
(Lotus)
Config
(GIT Repo)
Other
3rd Party
Plug-Ins
Configure
VCAC Service
Catalogs and
Policies
Configure
Monitor Agents
and Collector
Feeds
Export
App D Services
and Blueprints
Export
Provision and
De-provision
Workflows
Export App D
Update Harness
Container
VMware /
Non-VMware
Components
Plug-In Mgmt.
Service Profile
and Provision
Tasks/Services
Mgmt.
Provisioned
Workloads
Unified Asset
Inventory
Provisioned
Workloads
Monitoring
Mgmt.
Provisioned
Workloads
Monitoring
Provision/
De-provisioning
Audits
Extension 3rd party
components
Private Cloud IaaS Software-defined Data Center
vCloud Suite
…
17 17
Cloud Automation & Mgmt. Platform: Life-cycle Diagram E
xte
nsio
n
vC
lou
d
Dir
ecto
r
Ap
pli
cati
on
Dir
ecto
r (V
CA
C
En
terp
rise)
vC
en
ter
Orc
hestr
ato
r
vC
lou
d
Au
tom
ati
on
Cen
ter
Build Configure Phase
Provision Phase
Monitor & Manage Phase
18 18
Cloud Automation & Management Capabilities
Capability VMware Extension
Service
Catalog vCloud Automation Center - VCAC
Blueprinting Application Director (Part of VCAC Enterprise)
Provision, Scale-out, In-place Upgrade a blue print multiple times with in the same vLAN
Provisioning
Geared towards production ready workloads
Policy Based Automated Provisioning - Balance Costs w.r.t Performance requirements:
• Use Fast Provisioning for Low Cost Envs.
• Use full clones with custom choice of Storage, Networks, Load Balancing etc.
• Highly customizable.
• De-provisioning – Tear down and reclaim all the resources for optimized resource usage
Monitoring Single Pane of Glass - Monitors resource consumption as well as application behavior
Management Comprehensive unified platform for pre-provision set-up, configuration, provision-time
auditing and management, post provisioning workload inventory, monitoring etc.
Use Cases Any Use Case.
Targeted towards Use cases like "Dev. Test, UAT”, “Sales Demos", "Training", Bug
Fixing” etc.
Portability Is completely portable as an OVF and can be setup in-house
Hybrid Cloud Multiple options for Hybrid cloud –
With in house - IPSec with any cloud provider + Private Cloud
CAaMP on Cloud - IPSec with any other cloud + Private Cloud
Χ
Χ
19 19
Policy Driven Business Workloads Provisioning
Self-serviced Portal for consumers to
request Services on-demand from the
published Service Catalog
Request Services for a specific term
more of a leased consumption model
Request Services with the choice of
Service Profiles based on cost &
Performance needs
Policy driven approval process
Fully configured Business Workloads
provisioned based on requests with
out manual intervention
Workloads provisioned with integrated
monitoring provide deep insight &
visibility
Policy driven alerts, notifications
Rabbit MQ Server MQ
Spring Module
Controller
Modules
Lotus LDAP
Server vShield Edge
vCOps
Log Insight
Hyperic
LSPA
Client
Cloud Automation & Management
Access Control
Provisioning Policies
Users
Catalogs
App Director
vC
en
ter
Orc
he
str
ato
r
VC
AC
Se
rvin
gs
Ca
talo
g M
an
ag
er
Scripts/Tasks
Application
Blueprints
Deployment
Tasks
Cloud Provider
Access Control
Provisioning Policies
Users
Catalogs
SDLC Instance
Config
Management
3rd Party
LB
IPAM
Analytics
Async
call
Audits
Postgres DB
Message
Handlers
IPAM
Client LB
Client CM
Client
Extension Module
20 20
Policy Driven Business Workloads De-Provisioning
Self-serviced Portal for extension
of lease term.
Fully automated de-provisioning
once the lease term expires.
Reclamation of resources will
help reduce the future CAPEX
investments.
Rabbit MQ Server MQ
Spring Module
Lotus LDAP
Server vShield Edge
CM
Client
Cloud Automation & Management
Access Control
Provisioning Policies
Users
Catalogs
App Director
vC
en
ter
Orc
he
str
ato
r
VC
AC
Se
rvin
gs
Ca
talo
g M
an
ag
er
Scripts/Tasks
Application
Blueprints
Deployment
Tasks
Cloud Provider
Access Control
Provisioning Policies
Users
Catalogs
SDLC Instance
Config
Management
3rd Party
LB
IPAM
Analytics
Async
call
Purge
Audits
Postgres DB
IPAM
Client LB
Client LDAP
Client
Purge Records
MQ
Purge Message Handlers
Purge Controller Modules
Rea
d V
M P
rofile
s
Extension Module
21 21
Provisioning with VMware Application Director
Cloud Provider (vCAC or vCD)
VM1
VM2 VM3
Guest Cust
Copy Files
Install Web
Start Web
Guest Cust
Copy Files
Install App
Start App
Guest Cust
Copy Files
Install DB
Config DB
Start DB Config App
Config Web
OneCloud IaaS Software Defined Data Center
22 22
Policy Driven Management
Policy Driven Storage Provisioning Leveraging vCloud Stack
In this diagram depending on the Service Profile requested, storage will be provisioned to appropriate storage profile.
Reduced OPEX with automation and reduced CAPEX with workloads provisioned with right resources
Setup different service profile policies driven by the cost and performance needs of the enterprises like Platinum, Gold, Silver etc.
Service Profile = Storage + Network + Monitor + Existing Services
Setup approval and access policies
vSphere
VMware Application Director
Access Control
Provisioning Policies
Users
Catalogs
vCenter Organization
pVDC Tier3 pVDC Tier 2 pVDC Tier 1
Application Blueprints
Cloud Providers
Deployment Profiles
Workflows Policies
vC
en
ter
Orc
hestr
ato
r
VC
AC
Serv
ice C
ata
log
Man
ag
er
vApps provisioned
in Tier 1, Tier 2
and Tier 3 Org
vDCs
Organization vDCs
Organization Network
vApp Network
External Networks
Resource Pools
Datastores
Port Groups or
dvPort Groups
Cloud
System
Admin
23 23
Integrated Monitoring
Monitor Dev, Test, Load or
Production environment
Workload Monitoring
Monitor application layers e.g.
Portal, SOA, EBS
Layer Level Monitoring
View metrics from vCops, Log
Insight in single Graph
Metrics Correlation
Real time monitoring of Key
performance Indicators
Real time view
Log Insight Hyperic vCenter Operations
Management Suite
Allows user to create new
dashboard for the required
metrics
Create New Dashboards
Drill down to resource level from
aggregated view
Drill down
3rd Party
24 24
Results
Phase 1 Phase 2
Cycle Time Hours per dev/test instance
172
36
Today
Phase 1 Phase 2
VMs Transitioned # of Non-production VMs
Phase 1 Phase 2
AppOpps team # of Engineers
Goal – 4000
2,800
2,200
Goal - 5
Start - 27 22
Reduced provision time
95% (4 weeks to 36 hours)
Improved productivity
of 600 developers
20%
Reduced
IT operations costs
$1.5M /year
Able to say
“yes” to developer requests
Reduced the cost of
a VM/month
80% ($133 to $20)
Reduced
infrastructure costs
$4.5M/year
Start – 4 weeks
Goal – 24 hours
Today Today
25 25
VMware IT: Lessons Learned
Build a business case with clear goals and planned ROI and get buy-in from the
CIO/VP level management.
Develop a phased approach with clear scope on the Services to be provided
either in Infrastructure or Applications for the successful implementation.
Develop Architecture using more cohesive integrated stack like vCloud Stack to
build Automation and Mgmt. platform to achieve all the capabilities at lower
costs and time to market.
Define the cost model for the services to serve different cost & performance
needs like Dev, UAT, Mission Critical etc.
Separate Operationalization team from the platform implementation team to
optimal implementation.
Have time dedicated for the Operationalization; new self-service processes,
retuning of skills and also ensure to have QA automation done at the end of
automated provisioning to get maximum ROI and efficiency.
26 26
Key Take-Aways
• Proven vCloud technology. Used by top customers for public and private
clouds.
• Automate, Manage and Monitor your Cloud effectively using vCloud Mgmt.
Stack to reduce provisioning time, costs and improve agility
• Hard to find a better ROI project to reduce OpEx and CapEx
In Summary
What’s Next
• Network virtualization with NSX
• Next level of optimization of Automation and Infrastructure & Application
resource usage
• More visibility into the provisioned environments to monitor, manage & self-
heal.
Other VMworld sessions & booths
27 27
Questions?
28 28
Other VMware Activities Related to This Session
Group Discussions:
VSVC1006-GD
vCloud Suite and SDDC with Tom Stephens
VSVC4948
THANK YOU
Moving Enterprise Application Dev/Test to VMware’s
Internal Private Cloud – Architecture,
Implementation and Integration
Thirumalesh Reddy, VMware
Padmaja Vrudhula, VMware
VSVC4948
#VSVC4948