Oracle Exadata and Oracle Enterprise

Manager 12c: Extreme Consolidation in Cloud

Nilanjay Bhattacharjee

Assistant Vice President IT

Nilanjay.Bhattacharjee@hdfcbank.com

About HDFC Bank

Housing Finance

Banking

Asset Managem

ent Insurance

Brokering

Others

HDFC

Group

HDFC Ltd. Gruh Finance Ltd.

HDFC Bank

HDFC Asset Management HDFC Standard Life Insurance HDFC Ergo General Insurance

HDFC Securities Ltd

HDB Financial Services Ltd. Credila Financial Services Pvt. Ltd.

About HDFC Bank

71 93 121 158 207 289

408 536

714

939

1223

0

200

400

600

800

1000

1200

1400

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Indian GAAP figures. Fiscal year ended 31st March

•10 year Compounded Annual Growth Rate

• Figures in Million USD

Million

USD

Dreaming of Cloud

• IT Strategy 2015

• Role of Cloud in HDFC IT

• Consolidation Planning

• Cloud Initiatives in Bank

• Database-as-a-Service on Exadata

• Schema-as-a-Service Implementation

• Secure Subset Self-Service Provisioning

• Extending to Middleware and Apps-as-a-Service

• Taking Private Cloud to next level

• Summary/Q&A

Agenda

• CapEx & OpEx Reduction

• Best in class customer services

• Maximize business opportunities

• Joint targets with business

• Process Improvement

IT Strategy 2015

Why Cloud

• Average 3 days to provision UAT Database for lending segment

• Silo’ed IT environment with Average 30% utilization

• Compliance requirement consume testing resources

• SI expenses for provisioning databases manually

• Overhead in managing configuration drift between production and test

environments.

• Rollout impact/delay on new business initiatives

• Challenges in managing exponential growth of data

Consolidation Options

Server

consolidation Database

consolidation Schema

consolidation PDB

consolidation

Implementation Easy Easy Difficult Easy

Application suitability Some All Some Some

ROI Low high extremely high highest

Time to market long long short short

Isolation highest high limited high

Availability High highest highest highest

Scalability limited excellent excellent excellent

Consolidation density low high highest highest

Application Classification

System classification Application Classification

Tier 1 Top 15 in terms of systemic risk Customer facing mission critical systems

Tier 2 Top end & middle tier High visibility & enterprise wide usage

Tier 3 Bottom end of middle tier

Smaller user base but critical business

function

Tier 4 Low tier

Surround Systems - Satellite system build

around core system

Classify application based on system availability requirement/criticality

Consolidation Advisory

• Applications with different workload profiles

• Multiple resources need to be analyzed

– CPU

– Memory

– Storage

– Network

Automatic Mapping of Source Servers onto Target Servers

Manual Mapping can be used if existing servers are part of the

consolidation exercise

Helps administrator determines candidates for consolidation

Maximizes server density

Helps maintain performance commitment

Satisfies business, compliance, and technical constraints

Database Consolidation on Exadata

Primary Database

LMS – 4

LOS – 2

WebCollection – 2

UAT Database

LMS UAT – 6

LOS UAT – 6

WC UAT – 4

LMS/LOS Archive – 2

LMS/LOS Bunker – 2

BCP Database

LMS – 4

LOS – 2

WebCollection – 2

Applications for DB Consolidation

Target consumer for Exadata UAT cloud

• Test system for Application already on Exadata.

• Application which need to certify or benchmark application on Exadata

Target consumer for AIX UAT cloud

• Application planning to migrate to 11g version from 10g

• Application for Load testing with near production load without need to

buy large test system

Application profile

• LMS - Loan Management System batch intensive DB size ~2.5 TB

• LOS - Loan Origination System pure OLTP DB size ~250 GB

• Webcollection - Web Collection module for Loan DB size ~250 GB

Enterprise Archival System using SCaaS

Business Challenges

• Long term Data availability requirement to meet regulatory compliance

• Data Availability requirement by forensic and analytics teams

• Read-only databases

• Applications lacking purging policy or budget for separate archival system

• Sunset applications which are on older hardware and version

• Applications where archival system have grown to unmanageable size

Enterprise Archival System using SCaaS

Solution: Schema As A Service

• Self Service Schema Provisioning/Schema-as-a-Service(SCAAS)

• Schema is allocated out of a large database which eliminated need to

creating and managing multiple database instances

• Separate Schema ensures isolation and security across multiple schema

objects and required isolation

• Utilize Advanced Compression and Exadata/ZFS Hybrid Columnar

Compression(HCC) capabilities as core building blocks for Archival Use

case in Cloud

• Used for Historical Reporting/Forensic/Archiving purpose

• Review database character set during migration

• Under score parameter set on current database should be reviewed

• Compatibility and optimizer parameter revision will impact migration

• SQL plan if fixed for any specific query need to be retained

• Keep schema name unique to avoid conflict issues

• Keep redo log on a faster file system

• Size undo retention to avoid snapshot too old errors

SCaaS - Pre-migration Check List

• No direct access to database server OS to avoid configuration change by Apps

team

• Bulk loading and maintenance if any should be done during low workload

period

• Access to database sys/system user should be controlled

• Full backup need to be retained by application team during initial migration of

data

• Each schema credentials are with schema owner

• NO OS dependent operation to be scheduled on schema host

• Set session timeout to weed out inactive sessions eating DB resources

• Ensure use of private synonyms and avoid public synonyms

• Enforce security policy and control the access of OS / Database sys user

SCaaS - Operational Controls

• Number of concurrent users accessing the schema

• Size database server memory appropriately keeping headroom for growth

• Size storage for the required IOPS as consolidation will bring concurrency

and performance bottleneck

• Schema with high workload should be controlled by profile and set

appropriate quota

• Size of indexes should be included while considering database sizing

• Check usage of parallelisms on tables

• Check existing system usage to gauge workload for compute requirement

• Co-host database of same flavor together

• Check benefit of compression for sizing storage

• Test compression with advisory to get actual benefits DBMS_COMP_ADVISOR

• Exadata /ZFS HCC compression. COMPRESS FOR QUERY LOW , ARCHIVE LOW /HIGH

SCaaS – Schema Cloud Sizing Tips

SCaaS – Challenges/Solution

• Challenges

• SCaaS creates Tablespace for each schema with single datafile

• Location of datafile can not be configured in EM12c

• Auto selects File system for existing dbf files

• Single listener for all schemas hosed on particular database

• Ignores tablespace level definition from source to destination

• Workarounds/Solution

• Use ASM for I/O rebalancing even for non RAC environments

• After schema is provisioned change the tablespace attributes as required

• Alter table definition for compression in staging database

DBaaS Solution Architecture

DBaaS Building blocks

• Hardware for UAT and Archival system zones

• Exadata X2-2 Half rack High capacity

• Sun X3 -2B with ZFS for HCC on smaller system

• AIX hardware with storage and Oracle software pre-installed

• Hardware for OMS and Repository

• 2 node OMS server

• 2 node RAC on Exadata

• Hardware load balancer for OMS load balancing

• Software

• EM12c (DB Cloud and Lifecycle Management pack)

• OEL 5.7 for OMS install

• Other packs for DB management

• Oracle Advanced Compression

• Project Planning

• Scoping implementation

• Engaging End Users and Stakeholders early on from Scoping till Go live

• Oracle as trusted advisor to Project

EM 12c Deployment Architecture

• Intel servers for OMS

• Citrix Netscaler for Load

Balancer

• EM12c Cloud, Lifecycle

Management, Diagnostics

and Tuning Packs

Details

EM12c

Repository OMS Server

Server Hardware Oracle Exadata IBM BLADE

Database /OMS

Version 11.2.0.3 Weblogic 10.3

Operating System OEL 5.7 OEL 5.7

CPU Allocation 2 CPU/node

12 Cores /

server

High Availability 2 node RAC 2 server

Memory Allocation 8 GB /node 32 GB /server

• Setup Cloud Management infrastructure

• Deploy EM 12c site

• Deploy Oracle Database, Cloud, Storage Management Framework and

Virtualization plug-ins

• Define Roles

• Roles for each application on Exadata, AIX and Sun zones

• Setup S/W Library

• Setup Zones/Pools

• Setup PaaS Infrastructure Zones

• Exadata, AIX and Sun Zones

• Setup Database Pools

• DB UAT and Archival Pools

• Setup Service Catalog

• Configure Profile

• Create Service Templates

• Setup Governance Rules and Policies

• Configure Request Settings

• Configure Quotas

• Configure Chargeback

Cloud Administration Setup

Cloud Administration Setup - Pools

Cloud Administration Setup – Profiles/Service Templates

Self-Service DB Provisioning from SSA Portal

• Self Service Database Provisioning with Content

• Early Adopter for RMAN restoration integration feature with DBaaS

• Level 0 RMAN Backups of Production Databases from Standby[Bunker Copies]

• Service Templates created for each of Level 0 Backups

• Process to Provision Databases through RMAN is same as Empty Databases

• Used for UAT/Testing purpose

SCaaS – Self Service Portal

Secure Subset Provisioning in Cloud

• Business Drivers

• UAT databases sometimes cannot be created due to large DB size,

security considerations, storage space & time constraints

• Impacting technical/functional testing

• Delayed rollout of bug fixes to production

• Existing UAT setups in use for prior testing mandating the setup to

remain untouched

• Objectives

• Creation of seed database containing sample data

• Reducing the storage footprint

• Ensuring sensitive data masked in test systems

• On demand generation & deployment

• Enable parallel execution of technical / functional UATs

010010110010101001001001001001001

001001001001000100101010010010010

011100100100100100100100001001001

011100100101010010010101010011010

100101010010

Test Database Masked Data

Pump File

Production/Standby

One-step Subset and Mask

RMAN Copies

Self-Service

Cloud

Secure Subset Provisioning Implementation

Workflow driven Create Application Data Model

Define Table relationships

Subset and Masking Definition

Define Rule parameters

Define Table Rules

Generate Subset

Import Subset

Prepare secure test/reference database

Self-Service Cloud

Secure Subset Provisioning – ADM

• Create Application Data Model(ADM) as knowledge base for metadata,

referential relationships, sensitive columns

• ADM setup as first step for both Data Subsetting and Data Masking

• Review Table relationships and sensitive columns

• TEST application data subset

successfully created for a month's

data.

• Subset includes master tables and

selective data from transaction

tables.

• Process tuned for completion from

an initial period of 18 hours to

approximately 2 hours.

• Time duration further reduced by 14

minutes, with usage of invisible

indexes on date parameter fields

• Subset released for UAT and signed

off for rollout

Secure Subset Provisioning – Case study

DWMS UAT Regular

Database

Subset

Database

Database

Size 430 GB 102 GB

Export Dump

Size 185 GB 6 GB

Import Time 6 – 8 hrs

30 mins (May

very as per

subset size)

Secure Subset Provisioning - Comparative Analysis

Sr No. Activities for Regular DB restoration

Estimated

Time

(collective)

Sr No. Activities for DB Subset Estimated Time

1 IT Team logs a call for TEST restoration

3 – Man

days

1 One Time ADM & Subset

definition

Will vary depending on

the quantum of tables

to be parametrized

2

Restore team identifies TEST, based on

date specified by IT Team and informs

Librarian for restoration.

2

Specify parameter values

such as start date, end date

for subset generation.

2 mins

3

Librarian will get the TEST to SITE for

duplication, if data has been backed up

from SITE master server.

3 Generate Subset

2 hrs (taken for this

TEST subset for one

month's data)

4 TEST Duplication 4 Import Subset

30 mins (taken for this

TEST subset for one

month's data)

5 Duplicate TEST is couriered to SITE2.

6 Maintain free Storage space for dump

restoration

7 Execute TEST dump restoration

8 Import Dump

Operational Efficiency – Application teams

• Automated Self-service provisioning eliminated error prone manual

activities

• Ability to test new releases quickly due to rapid provisioning

• Ability to create secure reduced size production like environments

• Application teams can manage backup/restore from SSA portal

• New application POCs can utilize high-end infrastructure

• Application teams can self-manage their archival system

• Best Practices in place for managing historical

reporting/forensic/archive data

Operational Efficiency - Cloud Admins

• Planning done upfront by Infrastructure team and no brokering of

individual requests

• DBAs managing fewer databases rather than several silos

• Sensitive data masked in test systems without manual intervention

• Storage for projected growth already available due to use of

compression technologies and subsetting in cloud

• Standardized templates ensure adherence to baseline and

compliance

• Ability to perform configuration drift analysis between test and

production

• Centralized dashboard to manage overall cloud infrastructure and

resource usage

• EM12c driven MW/App Cloud allows Administrators to

– Pool resources

– Standardize and automate deployment processes

– Publish established templates to service catalog

– Setup role-based access and privileges

– Set quotas to limit over-consumption

– Establish policies for scale-up, scale-down and retirement

– Enable metering and optional chargeback.

• For Consumers

– Request and Provision Middleware Services

– Monitor performance of provisioned middleware service

– Control availability of provisioned WLS through simple push buttons

– Scale Up / Scale down provisioned middleware instance

– Deploy/Undeploy/Redeploy Java EE Applications

– Create Data Sources

– Monitor deployed J2EE Applications

– Deletion of middleware service instances that will no longer be used

Extending to Middleware and Apps-as-a-Service

• New business initiatives are easily launched due to rapid provisioning

• Effective usage of infrastructure leading to better ROI

• Empowered Application teams to provision Apps/DBs/Schemas using Self

Service

• Hardware hosting sunset applications phased out of DC reducing power

and cooling requirement

• Cut down on Support/SI costs for DBA activities

• 10X compression achieved with Enterprise Archival System in Cloud

leading to lower CapEx requirement

Benefits

• Snap Clone

– Thin Cloning on ZFS, Netapp, Hitachi (once available)

– “Time Travel” Capability enhances Self Service benefits

• DB 12c and PDBaaS

– DB 12c Beta Customer and Early Adopter

– Schema v/s PDB Consolidation

– PDB Management and PDBaaS

• Enterprise Cloud on SuperCluster

– EM Ops Center and EM CC managing SSC

– EM OC automating Solaris zones provisioning

– DBaaS on Exadata

– MWaaS/Apps-as-a-Service

– Snap Clone on SSC

– Cloud Blueprints

Taking Private Cloud to next level

• HDFC Bank’s Private Cloud Solution

• Banking on a Private Cloud

“HDFC Bank maximizes Oracle Enterprise Manager 12c to reduce costs and launch

new business initiatives. “

• HDFC at OOW ‘13

– Zero to Cloud: Real Customers, Real-World Success Stories [CON9585]

– Best Practices for Mission-Critical Applications on Oracle SuperCluster

[CON9263]

– Oracle Exadata and Oracle Enterprise Manager 12c: Extreme Consolidation

in the Cloud [CON2734]

References

Q&A