vmworld 2013: automated management of tier-1 applications on vmware

Automated Management of Tier-1 Applications

on VMware

Jeremy Kuhnash, VMware

Scott Salyer, VMware

VAPP5473

#VAPP5473

2 2

Agenda

Introduction

MS SQL Server Rolling Patch Upgrades with Automation

Migrating Oracle Databases from Bare Metal to VMware

SAP Performance Remediation with VMware Automation

3 3

Introduction

4 4

vCenter Site Recovery Manager vCloud Networking and Security

vCenter Orchestrator

vCloud Automation Center

vCenter Operations Manager

Hyperic

Tier-1 App Virtualization Domains

Monitoring and Operations

• Data Collection Methods (Adapters)

• App-specific Dashboards

• Alerts and Triggers

Automation and Provisioning

• Patching

• Migration

• Remediation

• Hybrid Cloud

Disaster Recovery

• App-specific site resiliency strategies

• In-app features that complement SRM

• Fully automated site recovery

Network Security

• Edge and App Functionality

• Elastic network scaling

• Management and Reporting

Virtualization of Tier 1 Apps

- Best Practices

- Design/Sizing

- Performance

- Availability

5 5

Benefits of Automation

Rapidly execute predictable or repetitive tasks

• Patching

• Mass migrations

• Trigger-based remediation

• Hybrid Cloud deployment

Work around software limitations to address unique use cases

• Pull deployment information from a configuration database

• Integrate multi-part approval workflows

Avoid human error

• Complex deployments such as RAC

• Deployments that have multiple steps/systems to update (e.g. DNS, AD)

Delegate administrative tasks; hide complexity from end users

• Self-service provisioning and maintenance for app owners or developers

6 6

Automation Toolset

Product Description Best Used For

PowerCLI

(or other OS/App scripts)

• Scripting tool built on Windows

PowerShell

• Other OS’s or Apps may have

other options

• Simple automation scripts that

can manage both vSphere and

Guest OS

vCenter Orchestrator • IT Process Automation engine

• Develop complex automation

tasks

• Launch workflows directly from

the vSphere Client or via various

triggering mechanisms.

• Complex, multi-step workflows;

glue many scripts together

• Multiple systems to update (DB,

AD, DNS, etc.)

• Human workflow elements

required (e.g. email notifications)

vCloud Automation Center • Customized, self-service

provisioning and automation

based on business policies

• Secure portal where authorized

users can request new / manage

existing resources from

predefined menus

• Delegation of workflow execution

(e.g. App Admins)

• “Industrializing” extremely useful

or popular workflows

• Adding a layer of security to limit

potential mishaps and audit

workflow execution

7 7

Automation Toolset Positioning

OS/App Scripting (e.g. PowerCLI)

VMware vCenter Orchestrator

VMware vCloud Automation Center

Com

ple

xity o

f S

olu

tion

Organizational Impact

Maturity of Solution

Security

/ D

ele

gation

8 8

Automation Examples


• Completely automated solution for patching non-clustered SQL Server VMs

• Less than 1 minute of downtime (15 seconds for small DB VMs)

• Can replace clustering in cases where clustering is only needed for patching

• Can be securely delegated to app admins with vCloud Automation Center


• Completely automated solution for migrating Oracle DBs from a single physical

server with multiple DBs to many VMs, each with one DB

• Utilizes Oracle DB scripts to migrate databases…coordinates with VM creation

processes to split DBs amongst the appropriate number of VMs


• Completely automated solution for provisioning a pre-configured SAP front

end-server VM based on a triggered performance threshold from vCenter

Operations Manager

• Notifies SAP Admin of the provisioning action

9 9


10 10

Patching MS SQL Server at XYZ Corp

Necessary, but Expensive

• Long maintenance windows

• Productivity costs through downtime

• High Administrator costs in managing patches

and maintenance windows

• Reproducibility is a challenge

11 11

Option 1: Clustering MS SQL at XYZ Corp

Clustering

• Provides high availability (HA)

• Patching features

• Higher License Fees than Standalone

12 12

Option 2: MS SQL Server Rolling Patch Upgrades Using VMware

Virtualization allows use of Secondary / Passive Server

• Dramatically shortens time required for maintenance windows

• Standard templates can be used for revving versions

Still a Manual Process

• Time Consuming

• Many steps

• Still must manage outage windows

13 13

VMware Automation for Tier 1 Applications

VMware

PowerCLI

Scripting Automation

on VMware

Final Role Switch Configure Standby VM

to IP Address of Primary

VM, observe Application

connections resume.

Add Resource Associate storage with

Secondary VM in

vSphere, bring OS

volume online.

Remove Resource Hot remove storage from

Primary VM in vSphere.

Deploy Standby VM Create Standby VM clone,

upgrade to desired SQL

Server level.

Network Disconnect Disconnect public NIC

device on Primary VM

SQL Rolling Upgrade Process

14 14

Scripted MS SQL Server Rolling Patch Upgrades

VMWare PowerCLI and Powershell provide a reproducible result

What about…

Audit trail / log of execution?

Which roles participate in managing upgrade and how?

VMware ESX VMware ESXi

15 15

Use vCenter Orchestrator and vCloud Automation Center to Enhance Rolling Patch Upgrades

Automation Execution and Status

• Workflows provide a powerful means for process flow and

control

• Creates a standard definition of infrastructure processes

• Execution status available in realtime

Integrates with Scripting and Systems

• Managed Powershell execution

Self Service

• Self Service Portal

• Initiated by assigned user Roles

• Delegated Approvals

16 16

vCAC Self Service Portal

17 17

vCAC Self Service Portal

18 18

VMware vCO

vCenter Orchestrator and vCAC + Scripted Execution


VMware vCAC

Automation Infrastructure

PowerShell Host

20 20

Steps to Building Out Self Service Automation

Develop and Configure vCAC

Extension in Visual Studio

1

Develop vCO Workflow for

Execution of Powershell /

PowerCLI

2

Develop Powershell /

PowerCLI and Configure vCO

Powershell plugin and

PowerShell Host

3

21 21


22 22

Oracle Database Migration - Physical to Virtual

From Physical to Virtual machines

• License and Maintenance Savings (i.e. RISC to x86)

• Return On Investment

“Deconsolidation” of Multiple Databases on a single host

• Workload Measurement

• Quality of Service (QoS)

• Avoids Single Point of Failure (SPoF)

Completely Automated with Provisioning

• Oracle data pump utility

• vCloud Orchestrator

23 23

VMware Automation for Tier 1 Applications: Oracle

VMware

Orchestrator

Process Automation

Engine

Finalize Migration Configure applications for

new database, observe

connections resume.

Create Database Execute ‘create_db.sh’

script on target. Copy Data File Using common storage

between source and target,

copy exported database

dump file.

Provision Oracle VM Prepare and create VM

clone with target Oracle

binaries installed.

Export Database Execute ‘exp_db.sh’ script

on source database host.

This exports the database to

a dump file.

Oracle Migration Process

Import Database Execute ‘imp_db.sh’ script

on target database VM. This

imports the database from

the dump file.

Get DBs on Oracle

Server Get database instances on

Source Server

24 24

vCO Workflow: Oracle Migrate Databases from Host to VMs

25 25


SSH into Oracle server

Parses /etc/oratab

Extracts databases instances

Returns as a vCO array of database

names

databaseNames: ‘orcl’, ‘orcl2’

26 26


Creates a VM Clone from Template

that has Oracle binaries preinstalled.

VM Name uses Database name

Starts VM, waits for boot

Returns VM as Workflow Attribute,

used as target for Import steps


targetVMName: ‘ORAVM-orcl’

currentDatabaseName: ‘orcl’

27 27



targetVMName: ‘ORAVM-orcl’

currentDatabaseName: ‘orcl’

28 28

vCO Workflow: Migrate Oracle Data in Detail

export_db.sh: Export Database to

dump file using ‘expdp’

(Oracle Data Pump)

create_db.sh: Create Database

as currentDatabaseName

import_db.sh: import database

from database dump file using

‘impdp’ (Oracle Data Pump)

29 29

vCloud Orchestrator Automated Oracle Database Migration

Oracle Database Server

VMware vCO


30 30


31 31

Variability in SAP Workloads Require More Compute Nodes

Scaling out to meet ad-hoc spikes in demand as required by the

business

• May be unforeseen

Scaling out to meet extra demand at month end

• Predictable spike in demand

In this example, 4 applications servers required for month end

• Beginning of the month, 2 application servers are sufficient and up and running

• 2 extra application server VMs available but powered down

• High CPU usage > 80%, custom threshold violation in vCloud Operations

Manager generates an SNMP trap

• vCloud Orchestrator detects this trap and starts remediation workflow to start

another application server

32 32

VMware Automation for Tier 1 Applications

VMware

vC Ops and

vCO

Notify Admin Send email notification

that remediation has been

handled.

Workflow Starts

Standby Server vCO workflow manages

startup of powered off

SAP app server.

vCOPs Creates

SNMP Trap Create alert for SAP servers

based on criteria.

Handle with vCO

SNMP Trap Policy Handle and process alert as

a vCO policy.

SAP Performance Remediation Process

Execute vCO

Workflow Start workflow which

remediates issue.

33 33

vCenter Orchestrator SNMP Integration Detail

vCO Policy listens for trap from SNMP Device

Policy executes script

Script calls workflow based on trap data

vCenter Ops

Manager

SNMP

Device Script Workflow

Trap

Trap Data from

SNMP Service

Analytics VM IP

Trap Port

Start

34 34

Configure vCO Policy for SNMP Traps

vCO Supports scripting as a response to an SNMP Trap allowing

filtering and custom handling of values

Workflows can also be configured for handling a trap directly

35 35

Correlating Trap OIDs and Values

Common criteria can be used within the script for filtering based

on values of the trap OIDs

vCO Workflow ID

OID Keys

Expected Values For OIDs

36 36

Custom Scripting for vCO Policies

Scripts use filtering logic to start the workflow when

criteria matches

If conditions are met, Run the

Workflow.

37 37

vCO SAP Remediation Workflow

Once started, the workflow will start an available or specified SAP

app server VM that has been configured but is powered off

vCenter Orchestrator also

supports full user interaction

for those environments that

require Administrator

response before remediation.

38 38

vCOPs and vCO for SAP Remediation

Desktop

Virtual

SAP Servers

Enterprise Storage

VMware

vC Ops

VMware

vCO

39 39

Summary

40 40

Key Takeaways

Use the most appropriate tool for the job

Need better management of script execution? Start by using vCO

Common, recurring processes can be presented to Users or

Admins as self service within vCAC…use vCAC to “industrialize”

your most commonly used workflows

Manage security and auditing of delegating to users within vCAC

Utilize notifications to keep stakeholders informed; keeps admins

in the know

41 41

Questions?

THANK YOU

Automated Management of Tier-1 Applications

on VMware

Jeremy Kuhnash, VMware

Scott Salyer, VMware

VAPP5473

#VAPP5473

45 45

Appendix

46 46

Implementation Notes

Process takes around 15 minutes per small database

• Actual time will vary based on provisioning time, resources and actual size of

database

vCO processing of SSH commands will fail if the result code or

stderr streams indicate failure.

• Not all DB objects in the dump file will succeed on import, but this does not

necessarily indicate a real error. Some objects may already correctly exist from

loading the catalog.

• Advise review of import logs as part of a sanity check.

Additional details in Oracle Sessions #xxxx and at the Tier1 Apps

Booth

vmworld 2013: automated management of tier-1 applications on vmware

Technology

vmware automation

triggers automation

benefits of automation

complex automation tasks

vmware virtualization

patching ms sql server

vmware vapp5473

process automation engine