martin lambert business development manager – oracle australia next generation database management...

Martin LambertBusiness Development Manager – Oracle Australia

Next Generation Database Management with Oracle 11g

Oracle Enterprise Manager:Empowering IT to Drive Business Value

Oracle Enterprise Manager:Empowering IT to Drive Business Value

<Insert Picture Here>

Agenda

• The “Big 4” 11g features• Real Application Testing• Advanced Compression• Active DataGuard• Total Recall

• Enterprise Manager 11G and the DBA

• “It’s not just about the database”

Oracle Database 11g Highlights• High Availability

• Logminer, Physical Standby, Streams support for XMLType, TDE

• Fast Start Failover with Async Propagation• Faster Logical Standby Apply• Flashback Transaction• Data Repair Advisor

• Integration• Faster Gateways• Gateways to Legacy Databases• Streams Synchronous Capture• Streams Faster Apply

• Performance and Scalability• RAC Scalability Enhancements• Faster PL/SQL Triggers

• Security• Tablespace and LOB Encryption• Secure By Default• Enterprise Manager support for TDE, OLS,

VPD, EUS• TDE works with LOBS, log miner

• Windows• VSS Writer for Point In Time Copies

• Manageability and Diagnosability• Unified Automatic Memory Management• Scheduler supports remote jobs• Automatic Health Monitoring• Faster Database Upgrades

• ASM• Variable extent sizes, Fast resync, Preferred reads

• RAC• Highly optimized protocols• ADDM for RAC

• Content Management Infrastructure• Binary XML storage• Enhanced XML indexing• Combined Text and Relational Indexes• DICOM Medical Imaging Support• Storage & Query of Semantic Content (RDF, OWL)

• Business Intelligence• Next generation OLAP• Supermodels for Data Mining• Enterprise Manager is DW aware• PIVOT/UNPIVOT operators

The Big 4 Features in 11g

• Real Application Testing• Advanced Compression• Active Data Guard• Total Recall

Real Application Testing – Database Replay

• Recreate actual production database workload in test environment

• Identify, analyze and fix potential instabilities before making changes to production

• Capture Workload in Production• Capture full production workload with real load &

concurrency info• Move the captured workload to test system

• Replay Workload in Test• Make the desired changes in test system• Replay workload with production load & concurrency• Honor commit ordering

• Analyze & Report• Errors• Data divergence • Performance divergence

Analysis & Reporting

Database Replay Workflow

Pre- Process

TransferASM

…

Test ( 11.1)

Replay Analysis &

Reporting

Storage

Replay Driver *

Linux

Production ( 9.2.0.8)

Mid-Tier

Clients…

Capture

Storage

Windows

File System

Real Application Testing

• SQL Performance Analyzer• Predicts impact of change on

SQL response time• Assess change by executing

query part of SQL in isolation without concurrency

• Unit testing of SQL, e.g., optimizer stats/config, drop index, etc.

• Can use for upgrades from • 9.2/10.1 10.2 or 11g• 10.2 10.2 or 11g

• SQL Performance Analyzer• Predicts impact of change on

SQL response time• Assess change by executing

query part of SQL in isolation without concurrency

• Unit testing of SQL, e.g., optimizer stats/config, drop index, etc.

• Can use for upgrades from • 9.2/10.1 10.2 or 11g• 10.2 10.2 or 11g

• Database Replay• Predicts impact of change on

workload throughput

• Assess change by replaying workload with concurrency

• Comprehensive testing of DB stack, e.g, memory, RAC, concurrency related

• Can use for upgrades from

• 9.2 or 10.2 11g

• Database Replay• Predicts impact of change on

workload throughput

• Assess change by replaying workload with concurrency

• Comprehensive testing of DB stack, e.g, memory, RAC, concurrency related

• Can use for upgrades from

• 9.2 or 10.2 11g

Real Application Testing = SPA + Database Replay SPA + Database Replay are complementary Together, provide comprehensive & flexible testing solution

Snapshot Standby – Leverage Standby Database for Testing

• Convert Physical Standby to Snapshot Standby and open for writes by testing applications• ALTER DATABASE CONVERT TO SNAPSHOT STANDBY;

• Discard testing writes and catch-up to primary by applying logs • ALTER DATABASE CONVERT TO PHYSICAL STANDBY;

• Preserves zero data loss • But no real time query or fast failover

• No idle resources• Similar to storage snapshots, but:

• Provides DR at the same time• Uses single copy of storage

Physical Standby Apply Logs

Snapshot Standby Perform Testing

Open Database

Back out Changes

Continuous Redo Shipping

The Big 4 New Features in 11g


Advanced Compressionin Oracle Database 11g

• Comprehensive set of data compression capabilities• Structured ( inline ), unstructured ( zlib ), rman ( zlib ),

network transport ( zlib ), datapump ( bzip2 )

• Reduces storage consumption by 2 – 3 times• Savings cascades into test, dev, QA, backup, DR

environments

• Enhances memory and network efficiency• Complete application transparency• Benefits diverse application workloads

• Transaction Processing• Data Warehousing• Content Management

Up To

3XCompression

OLTP Table Compression

1•John•Doe 2•Jane• Doe 3•John•Smith 4• Jane • Doe 5•Jack •Smith Free Space

Header

Uncompressed Block

John=|Doe=|Jane=|Smith=Header

1•John•Doe 2•Jane• Doe 3•John•Smith 4• Jane • Doe

Free Space

1•• 2•• 3•• 4 •

• 5•Jack•

Free Space

Compressed Block

Local Symbol Table

Local Symbol Table More Data

Per BlockMore Data Per Block

Data Guard vs Active Data Guard

Data Guard 11g

• Stop redo apply at 8am• Open read-only for queries

Active Data Guard Option

• Redo apply is always on

• Always open read only

• By 4pm, data is 8 hours old • Queries and reports always see latest data

• Any failover will be delayed due to backlog of data that must be applied

• Failover is immediate when needed, standby database always up-to-date

Active Data Guard Reader Farm

Active Data Guard 11g Scale-out Query Performance to Web-Scale*

ProductionDatabase

updates

StandbyDatabases

Or Single Nodequeries

queries

queries

queries

queries

DR included *

Managing Data History – Current Approaches

• Application or mid-tier level• Combines business logic and archive policies

• Increases complexity

• No centralized management

• Data integrity issues if underlying data is updated directly

• Database level• Enabled using Triggers

• Significant performance and maintenance overhead

• External or Third-party• History stored in separate database

• Cannot seamlessly query OLTP and history data

Introducing Flashback Data Archive

•Seamless access to archived historical data• Using “AS OF” SQL construct

select * from product_information AS OF TIMESTAMP

'02-MAY-05 12.00 AM‘ where product_id = 3060

• History is stored in automatically created history tables inside the archive

• History is captured asynchronously by new background process (fbda)


Agenda


• Enterprise Manager 11G and the DBA


Top DBA Challenges

Performance Management

Change Management

Ongoing Administration

Top DBA Challenges


Sustain Optimal Performance

Performance Diagnostics

Application Tuning

Change Management Preserve Order Amid Change

Ongoing Administration Manage More with Less

Boost Administrator Productivity

Boost Administrator Productivity

Eliminate Performance Bottlenecks

Eliminate Performance Bottlenecks

Lower IT Management Costs

Lower IT Management Costs

Deliver Higher Quality of Service

Deliver Higher Quality of Service

Automatic Performance DiagnosticsMaximize Performance With Self-Managing Database

• Sustain maximum performance with self-managing database• Top-down analysis using Automatic Workload Repository snapshots• Real-time performance analysis with Automatic Database Diagnostic Monitor• Resolve performance issues faster with drill-down root-cause analysis• Classification tree based on Oracle performance tuning expertise• Performance expert; now a RAC specialist too in Oracle Database 11g

Automatic Workload

Repository Snapshots

Performance & Management

Advisors

HighLoadSQL

RACIssues

IO / CPUIssues

Self-Diagnostic Engine

Pre-PackagedApps

Pre-PackagedApps

CompositeApps

CompositeApps

CustomApps

Middleware

Database

Physical & Virtual Servers

StorageA

pp

lica

tion

-to-

Dis

k

eCommerce Apps • Enhanced ADDM

• NEW: SQL impacting performance findings highlighted

• Global AWR report for easier RAC diagnostics

• Backward compatibility: Can analyze snapshots from earlier versions

• Active Session History (ASH)• NEW: ASH for Standby: Facilitates

performance analysis of Standby DB• Global ASH reports for RAC systems

Automatic Performance DiagnosticsNew in Oracle Enterprise Manager 11g

Performance DiagnosticsNEW: Active Reports

• Interactive reports for off-line analysis• Offer same level of interactivity with drill-downs for detailed analysis• Can be emailed to appropriate personnel for analysis • Does not require Oracle Enterprise Manager installation

• NEW: Monitoring and management of Clusterware

• NEW: Grid Plug-and-Play support for policy-managed RAC databases

• Addition and deletion of RAC nodes

• ADDM shows blocking sessions across RAC instances

RAC Database ManageabilityNew in Oracle Enterprise Manager 11g

• Configuration-wide summary view with charts for RAC/HA setup• Provides recommendations for MAA implementation

• NEW: Minimum downtime automation for single instance to MAA• Show Availability related events across the configuration• NEW: Manage role-based standby DBs, e.g. convert Physical

Standby to Active Data Guard

High AvailabilityNew in Oracle Enterprise Manager 11g

Top DBA Challenges


Sustain Optimal Performance

Performance Diagnostics

Application Tuning



Application TuningAutomatic SQL Tuning

• Automatic SQL Tuning• Captures high-load SQL• Tunes SQL using SQL Profiles• Implements greatly improved SQL plans (optional)

• Performance benefit of advice provided• SQL Profiling tunes execution plan without changing SQL text• Enables transparent tuning for packaged applications

ApplicationsApplications

SQL

High-Load

Packaged Apps +SQL Profile

Customizable Apps + SQL Advice

Customizable Apps + Indexes & MVs +

Partitions

Well-Tuned SQL

Automatic Tuning Optimizer

SQL Tuning Advisor New in Oracle Enterprise Manager 11g

• SQL Tuning Advisor • NEW: Identifies alternative execution plans using real-time and historical

performance data• NEW: Recommends appropriate degree of parallelism to improve SQL

performance

SQL Profiling

Statistics Analysis

Access Path Analysis

SQL Restructure Analysis

Alternative Plan Analysis

Parallel Query Analysis

Automatic Tuning Optimizer

AdministratorAdministrator

SQL

ComprehensiveSQL Tuning

Recommendations

Gather Missing or Stale Statistics

Create a SQL Profile

Add Missing Access Structures

Modify SQL Constructs

Adopt AlternativeExecution Plan

Create Parallel SQL Profile

SQL

SQL Tuning Advisor

• Automatically monitors long running SQL• Enabled out-of-the-box with no

performance impact• Monitors each SQL execution • Exposes monitoring statistics

• Global execution level• Plan operation level• Parallel Execution level

• Guides tuning efforts• NEW: Available as Active Reports in

Grid Control 11.1• NEW: Enhanced to monitor PL/SQL

Real-Time SQL MonitoringLooking Inside SQL Execution

Top DBA Challenges

Performance Management Sustain Optimal Performance

Change Management

Preserve Order Amid Change Change Detection

Change Testing Change Automation


Security

Monitoring

Policies

Reports

Compare

Detecting Configuration and Application Changes

Configuration Changes• Ensures system compliance with company

standards

• Detects unauthorized (malicious) changes

• Over 300 out-of-box policies per Oracle “best practices”

• Monitor compliance continuously

Application Changes• NEW: Real-time schema change detection• Schema compare capability help identify

• Malicious SQL• Unauthorized access control changes• Changes to tables, procedures, triggers,

functions• New/dropped indexes, etc.

Top DBA Challenges


Change Management

Preserve Order Amid Change

Change Detection Change Testing

Change Automation


Secure Test System Deployment Oracle Data Masking

• Deploy secure test system by masking sensitive data• Sensitive data never leaves the database• Extensible template library and policies for automation• Sophisticated masking: Condition-based, compound, deterministic• Integrated masking and cloning• Leverage masking templates for common data types• NEW: Command line (EMCLI) support for data masking actions

LAST_NAME SSN SALARY

SMITH 111—23-1111 60,000

MILLER 222-34-1345 40,000

LAST_NAME SSN SALARY

AGUILAR 203-33-3234 40,000

BENSON 323-22-2943 60,000

Production Test

Change TestingOracle Real Application Testing

• SQL Performance Analyzer (SPA)

• Unit testing of SQL response time

• Database Replay

• Concurrent load testing using Production workloads

Replay

Workload

Capture

Workload

Deploy Replay

Clients

Create Test System

End-to-End Workflow

• Test and predict impact of system changes on SQL query performance• Analyze performance changes for improvements or regression• Comprehensive performance analysis and reporting• Re-execute SQL queries in test environment• End-to-end solution: STS, SQL Plan Baselines, & SQL Tuning Advisor

Oracle Real Application TestingSQL Performance Analyzer

SQL Workload

STS

SQL Plans + Stats

Pre-Change Trial

SQL Plans + Stats

Post-Change Trial

Compare SQL Performance

Analysis Report

SPA Enhancements

• Analyze performance of two workloads thru STS comparison

• Allows comparing two different STS and generate SPA report

• Better SPA trial accuracy through multiple test execution

• SPA provides “Alternate Plans” recommendation for individual regressed SQL in a SPA trial

• SPA Active Reports

• Reduce costs and risks of database changes

• Fully automated workflow

• Change Assurance for• Database and O/S upgrades,

e.g. migration to Exadata V2

• Configuration changes, e.g., conversion from SI to RAC

• Storage changes

• O/S migrations, e.g., Windows to Linux

• Supports upgrades from Oracle 9iR2 to 10gR2 and 11g!

TestProductionReplay DriverClients

Storage Storage

Oracle Real Application Testing Database Replay

Top DBA Challenges


Change Management

Preserve Order Amid Change

Change Detection Change Testing

Change Automation


Testing AutomationReduce Time and Effort of Database Testing

• Automate production to test

workflow of Database Replay

• Capture production workload,

performance data

• Copy workload, performance data to

staging

• Provision software, create test

database

• Deploy replay clients and

orchestrate replay

• Automate production to test

workflow of Database Replay

• Capture production workload,

performance data

• Copy workload, performance data to

staging

• Provision software, create test

database

• Deploy replay clients and

orchestrate replay

• Gold image based provisioning• Simplified RAC provisioning with automation• Single click RAC scale-out and scale-back• Provision full RAC and Clusterware systems • ASM Support

Deployment Automation

Create Reference RAC System

1

Stage Gold Image

2

Scale-Out RAC

4

Create Production RAC System

3

Database Change AutomationSchema Changes

Change Management• Schema change propagation

• Capture development changes in baseline version

• Preview changes before confirming

• Propagate planned changes from baseline to multiple databases

Corporate DMZ

Software Change Automation: Patching

•My Oracle Support connection

•Proactive advisories•CPUs•Feature based

•Patch downloads

•Patch Database•Patch/CPU/Patchset

•Zero downtime patching for RAC

•Lights-out patching for Linux•Automatic download from Oracle ULN

My Oracle Suppor

t

Top DBA Challenges




Manage More with Less Space and Resource

Management Automated

Self-Management

Space and Resource Management• Space Management

• NEW: Compression Advisor (Oracle Database 11.2)• Analyzes objects to give estimate of space savings for different

compressions methods• Helps in deciding correct compression level for application• Recommends various strategies for compression

• Resource Manager• NEW: Instance Caging (Oracle Database 11.2)• More flexible alternative to server partitioning• Wide platform support than operating system resource managers• Lower administration overhead than virtualization• I/O Resource Management for Exadata

• ASM Cluster File System support• NEW in Oracle Enterprise Manager 11g• Comprehensive administration• Proactive Monitoring• Security

Top DBA Challenges




Manage More with Less Space and Resource

Management Automated

Self-Management

Automated Self-Management

Oracle Database ManageabilityEmpowering DBAs To Drive Business Value

Enhance Service Quality By Maximizing Application-to-Disk Performance

Improved IT Agility with Risk-free Change Management

Lower IT Costs Through Self-Managing Automation


Agenda


• Enterprise Manager 11G Update


Oracle Application Operations ManagementOracle Application Operations Management

Complete and Integrated Solution

User Experience

Management

User Experience

Management

System Monitoring & Diagnostic

System Monitoring & Diagnostic

Configuration Management

Configuration Management

Oracle Fusion Middleware ManagementOracle Fusion Middleware Management

Oracle Database ManagementOracle Database Management

Oracle Solaris & Oracle Enterprise Linux ManagementOracle Solaris & Oracle Enterprise Linux Management

Oracle VM ManagementOracle VM Management

Oracle Sun Server ManagementOracle Sun Server Management

3rd Party Technology Management3rd Party Technology Management

Oracle Enterprise Manager

Lifecycle AutomationLifecycle

Automation

UI, Security, Integration, Data Management, My Oracle Support IntegrationUI, Security, Integration, Data Management, My Oracle Support Integration

Architecture: Network Protocol Analysis

InternetInternet

DB DB

Firewall

Switch

Webserver

Webserver

Webserver

Sniffing

Netw

ork

Resp

onse

Tim

e•Delivered?•Aborted?•Network timeout?

Pagerequest

Serv

er

Resp

onse

Tim

e

•Response?•Server error, web site

error, Hit OK•Content of page (i.e order

details, functional errors)

Collected data:

•Client-ip, Server-ip, Gateway-ip•Requested URL (Page,Object)•Cookie/Session-id•Referrer•GET & POST requests• User agents (phone type)

Real User Experience Insight key highlights:

- Replay of poor performance

- Executive dashboards

- Extensive KPI and SLA monitoring

- Full alerting capabilities

- Quick bottle-neck analyses

- Transaction performance analyses

- Customized reporting

- Trend analyses

- Full data integration via XML

KPI Monitoring

Transaction Performance Executive dashboard

Product OverviewOptimizing end-user experience through real end-user monitoring

Data Export

Business Reporting

53

Tracking Transactions

• Real-time tracking of each transaction• Follows transactions across all infrastructure and applications

• App Servers, Applications, ESB's, BPM's, Etc.

• Leverages message content – the business value flowing through the application

• Provides this without modifying or tagging the messages• Message Fingerprinting

• Non-invasive; doesn’t break applications

53

Process Engine Service Bus

End-to-End, Application Level View

Performance Analysis

• View Service topology

• Drill into functional implementation

• View performance for each step (BPEL activity, service bus pipeline, portal page, region, etc.)

• Drill into Java EE implementation of functional step

• Even drill to SQL performance for any service (including dehydration SQL)

Business-Driven IT Management

martin lambert business development manager – oracle australia next generation database management...

Documents

database slide

testing open database

oracle database

database replay spa

production workload

generation database

test system replay workload

g oracle enterprise