tools and techniques - amazon s3 · oracle enterprise manager cloud control 13c database express...

©2018 OneNeck IT Solutions LLC. All rights reserved. All other trademarks are the property of their respective owners.

Tools and Techniques to Address

Performance Problems

Biju Thomas

@biju_thomas


Biju Thomas

3

Principal Solutions Architect with OneNeck IT Solutions Over 20 years of Oracle Database development and administration expertise Over 10 years of Oracle E-Business Suite Architecture & Tuning expertise First book published in September 2000, seventh in 2015 DBA blog since 1997 – www.bijoos.com Daily Oracle Tidbits #oratidbit Oracle ACE Director

http://www.bijoos.com/

https://twitter.com/hashtag/oratidbit


OneNeck IT Solutions at a Glance

Hybrid & Multi-Cloud Solutions

Managed Services, Advisory Services

Coast to Coast Data Centers

4

©2018 OneNeck IT Solutions LLC. All rights reserved. All other trademarks are the property of their respective owners.5

User opens ticket – performance issue

Check Database

Single user/program issue or database wide issue

What changed

Collect information

How to reproduce the issue

Review information

Remediation

Root Cause Analysis

Prevention or Alert

Performance Troubleshooting CycleTypical – Reactive Performance Management


Optimizer

Inefficient plan

Application

Coding

Literal usage

Resource

CPU, Memory, I/O Contention

Locks

Why SQL statements perform slow?


SQL Scripts Oracle Enterprise Manager

Cloud Control 13c Database Express 12c

SQL Developer v18.1 Third party products Enterprise Edition Options

Diagnostic & Tuning Pack Automatic Workload Repository (AWR) Active Sessions History (ASH) Automatic Database Diagnostics Monitor (ADDM) Real-time SQL and Database Monitoring SQL Tuning Advisor

Which diagnostics tool?


Database Quick Check – OEM Performance Home


Database Quick Check: OEM Top Activity


SQL Developer: Instance Viewer


Database wide issue

Resource Contention? Disk

CPU

Memory

Oracle tools OEM

OS Watcher (301137.1)

Oratop (1500864.1)

ADDM & AWR

Performance Hub

QuestionsNew program?

Code change?

Parameters valid?

Problem start time?

Statistics current?

New statistics?

Wait event?

… … …

Database Wide or Single Program?Most performance issues are single program or single session

Single program or session

Oracle tools OEM

SQL Monitoring

ADDM & AWR

Trace Session

Solution Fix Code

Better Statistics

SQL Profile

SQL Plan Baseline

Bug?


OEM Database Express or SQL*Plus

The Performance Hub shows all the performance data available for a specified time period.

Single view for ADDM, SQL Tuning, Real-time monitoring, ASH

Real-time and historical

$ORACLE_HOME/rdbms/admin/perfhubrpt.sql

DBMS_PERF Package REPORT_PERFHUB ( ) function – entire database

REPORT_SESSION ( ) function – specific database session

REPORT_SQL ( ) function – specific SQL

Performance HubOEM Database Express / SQL*Plus

SummaryActivity

WorkloadRAC

Monitored SQLADDM

Current ADDM FindingsDatabase Time

ResourcesSystem Statistics

Basic

Typical

All


Performance Hub Report


STATISTICS_LEVEL

Retention – Default 8 days

Snapshot Frequency

Ways to generate AWR reports

Types of AWR data

Baselines

Comparison reports

Using *\_HIST\_* views

Automatic Workload Repository (AWR)

=TYPICAL (BASIC turns AWR off. ALL is problem)

Change to 30 or 60 or more days for meaningful history.

Default 60. OK for normal systems

Write powerful scripts to analyze performance

Compare with baselines or other snapshots

Periodic baselines help to compare performance metrics

OEM, SQL Developer, SQL

Non-CDB, CDB, PDB, Standby, RAC


SELECT

statistics_name,activation_level

FROM v$statistics_level

ORDER BY activation_level;

ALL: only at session level, to debug specific issues.

Huge WRH$_LATCH_CHILDREN

STATISTICS_LEVELDefault is TYPICAL


Objective is to reduce DB Time (CPU, IO & Non-idle waits)

AWR Reports Standard: awrrpt.sql / awrrtpi.sql Compare (difference): awrddrpt.sql /

awrddrpi.sql Global (RAC): awrgrpt.sql / awrgrpti.sql Global difference: awrgrdpt.sql Single SQL Statement: awrsqrpt.sql /

awrsqrpi.sql General info: awrinfo.sql

ADDM Report addmrpt.sql ASH Report ashrpt.sql

Generate AWR ReportsUse SQL*Plus, OEM or SQL Developer


Top-down approach unless looking for specific issues

Duration of AWR report

Number of sessions at the beginning and end

Load profile & Instance efficiency

Timed foreground events

Instance CPU

SQL Sections

Tablespace I/O

SGA & PGA

RAC related waits

What to look for in AWRWhen to check ADDM report

Always check corresponding ADDM report for actionable recommendations.

ADDM report also shows what is not a problem.


AWR Report – Quick Review


ADDM Report – Quick Review


Real-Time ADDM


OEM

Top Activity

SQL Details

Search Sessions

SQL Developer

Session Monitor

SQL*Plus

V$ & GV$ views

Finding the Problem Session & Offending SQL


@$ORACLE_HOME/rdbms/admin/ashrpt.sql ASH information for a specified duration

@$ORACLE_HOME/rdbms/admin/ashrpti.sql Instance # or “ALL” for all RAC instances

Filter by Session ID, SQL ID, Wait Class, Service Hash, Module, Action, Client ID

Active Session History (ASH)OEM / SQL Developer / SQL*Plus / ASH Viewer


OEM ASH Analytics & ASH Report


Real-time SQL Monitoring (SQLDev)


(Real-Time) SQL Monitoring (OEM)


OEM SQL DetailsStatistics / Activity / Plan / Plan Control / Tuning History / SQL Monitoring


Tracing SessionsOEM / SQLDev / SQL / DBMS_MONITOR

DBMS_MONITOR.SESSION_TRACE_ENABLE(

session_id IN BINARY_INTEGER DEFAULT NULL,

serial_num IN BINARY_INTEGER DEFAULT NULL,

waits IN BOOLEAN DEFAULT TRUE,

binds IN BOOLEAN DEFAULT FALSE,

plan_stat IN VARCHAR2 DEFAULT NULL);

If serial_num is NULL but session_id is specified, a session with a given session_id is traced irrespective of its serial number.

If both session_id and serial_num are NULL, the current user session is traced.


Trace

Analyzer

Name of

Control

File

Control

File

Text Report

HTML Report

Log Files

Event

10046

Trace

Event

10046

Trace

Event

10046

Trace

Event

10046

Trace

Event

10046

Trace

Reading Trace FileTKPROF or Use Trace Analyzer – TKPROF on Steroids! (MOS 224270.1)

Trace

Analyzer

Trace

File

Name

Event

10046

Trace

Text Report

HTML Report

Log Files


OEM SQL DetailsUse Search SQL from Cursor Cache and AWR Reports


SQL Tuning AdvisorAutomatic Tuning Optimizer is the central brain used by SQL Tuning Advisor.


Create a tuning task DBMS_SQLTUNE.CREATE_TUNING_TASK

(task_name=>'&&Task_name', sql_id=>'&sql_id‘)

Execute and report DBMS_SQLTUNE.EXECUTE_TUNING_TASK

DBMS_SQLTUNE.REPORT_TUNING_TASK

Accept Recommendations DBMS_SQLTUNE.ACCEPT_TUNING_TASK

Accept SQL Profile DBMS_SQLTUNE.ACCEPT_SQL_PROFILE

SQL Tuning Set (STS)http://kerryosborne.oracle-guy.com/2008/09/23/


@find_sql Find the SQL ID for the statement

@dplan.sql Use dbms_xplan.display_cursor to see the plan

@unstable_plans.sql Statements that have experienced significant variances in execution time.

@awr_plan_change.sql Shows plan_hash_value changes and execution time.

@find_sql_stats.sql Execution stats

Kerry’s Scriptshttp://kerryosborne.oracle-guy.com/


Proper Statistics

Histograms

Multi-column stats

SQL Profiles

SQL Plan Baselines

Directing the Optimizer


Plan Capture Creation of SQL plan baselines that store accepted execution plans for all relevant SQL statements.

Plan Selection Ensures only accepted execution plans are used for statements with a SQL plan baseline and

records any new execution plans found for a statement as unaccepted plans.

Plan Evolution Evaluate all unaccepted execution plans for a given statement, with only plans that show a

performance improvement becoming accepted plans.

SQL Plan Baselines

http://www.oracle.com/technetwork/database/bi-datawarehousing/twp-sql-plan-mgmt-12c-1963237.pdf


Automatic Capture OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES

Manual Capture From SQL Tuning Set

Use DBMS_SPM.LOAD_PLANS_FROM_SQLSET

Plans loaded are automatically accepted.

From Cursor Cache Use DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE

Filter SQL by text, module, parsing schema, SQLID

From AWR Repository (12cR2) Use DBMS_SPM.LOAD_PLANS_FROM_AWR

Specify range of snapshots

Plan Capture

DBMS_SPM.PACK_STGTAB_BASELINEExport / Import Staging TableDBMS_SPM.UNPACK_STGTAB_BASELINE


OPTIMIZER_USE_SQL_PLAN_BASELINES (default TRUE)

Plan Selection


Query DBA_SQL_PLAN_BASELINES An enabled plan is a plan that is eligible for use by the optimizer. An accepted plan is a plan that is in a SQL plan baseline for a SQL statement and thus

available for use by the optimizer. A fixed plan is an accepted plan that is marked as preferred, so that the optimizer

considers only the fixed plans in the baseline. Unused plans are purged after 53 weeks by default.

DBMS_SPM.CONFIGURE('plan_retention_weeks',n); Explain Plan on SQL shows if a baseline is used.

SQL Plan Baselines – Things to know


Copying Optimizer PlansUsing OEM – SQL Profiles & SQL Plan Baselines


Download SQLT (sqlt_latest.zip) Extract files Scripts under sqlt/utl directory Run coe_xfr_sql_profile.sql with SQL_ID as

parameter Shows each plan hash value and average

elapsed_time/executions Creates one file for each plan hash value of

SQL_ID found in cursor cache and AWR. Scripts:

coe_xfr_sql_profile_<sql_id>._<PHV>.sql

Run the required profile create script in desired database.

Copying Optimizer Plans – SQL ProfilesUsing Oracle COE Scripts – SQLT (MOS 215187.1)


Scripts to copy SQL Plan Baselines under sqlt/utl/spm directory.

Option 1: Create SQL Plan Baseline (SPB) in Source

From a Cursor; or From AWR

Package & Export SPB from Source Import & Restore SPB into Target

Option 2: Create SQL Tuning Set (STS) in Source

From a Cursor; or From AWR

Package & Export STS from Source Import & Restore STS into Target Create SPB from STS in Target

Copying Optimizer Plans – SQL Plan BaselinesUsing Oracle COE Scripts – SQLT (MOS 215187.1)


The Proactive Part…


Gather Optimizer Statistics Dictionary Stats

Fixed Object Stats

Database/Schema Stats

Capture Periodic AWR Baselines

Periodic review: Database Health

Resource utilization

Top SQL

Optimizer plan stability

Alert/terminate blocking sessions

Proactive ActionsSaves you from the fire drill!

(frequency?)


The Optimizer Statistics Advisor runs a task called AUTO_STATS_ADVISOR_TASK during the maintenance window

Can be personalized on what rules to check (v$stats_advisor_rules) DBMS_STATS.CONFIGURE_ADVISOR_OBJ_FILTER DBMS_STATS.CONFIGURE_ADVISOR_RULE_FILTER

Report recommendations dbms_stats.report_advisor_task

Remediation script DBMS_STATS.SCRIPT_ADVISOR_TASK

Manually execute the advisor DBMS_STATS.CREATE_ADVISOR_TASK DBMS_STATS.EXECUTE_ADVISOR_TASK

Implement all recommendations DBMS_STATS.IMPLEMENT_ADVISOR_TASK

Optimizer Statistics AdvisorDatabase 12c - 12.2 Feature


Optimizer Statistics Advisor Taskshttps://docs.oracle.com/en/database/oracle/oracle-database/12.2/tgsql/optimizer-statistics-advisor.html


Develop a strategy If nobody complains, define scope by yourself.

Baseline metrics & configuration

Performance thresholds

Report/review frequency

Tools ORAchk

DBSAT

OSWatcher

Health Monitor (DBMS_HM)

Third party tools (SolarWinds, Foglight, APM, etc)

Database and System Health checks


Review OEM Performance Home, Top Activity and AWR reports periodically, and know the typical performance profile. When problem happens you can easily point out what stands out.

Review reports from normal periods and acquaint with the “norm”.

Understand the load on the system and busy times, especially large batch jobs.

And, read “Oracle Database Performance Tuning Guide”

Familiarize with the environment


Oracle Database 12c Features for Developers!

1:30pm - 2:30pm @ Standley 2

Essential (free) Tools for DBA

2:45pm - 3:45pm @ Cotton Creek 2

RMOUG TD18 SessionsThursday, February 22 (tomorrow)

Essential (free) tools for EBS Administrator

Thank you!

Daily #oratidbit on Facebook and Twitter. Follow me!

Tweets: @biju_thomasFacebook: facebook.com/oraclenotes

Google+: +bijoosoraclenotesBlog: bijoos.com/oraclenotes

https://www.facebook.com/hashtag/oratidbit

https://twitter.com/hashtag/oratidbit

https://twitter.com/biju_thomas

https://www.facebook.com/oraclenotes

https://plus.google.com/+BijoosOracleNotes

http://bijoos.com/oraclenotes/


Thank you!

Please Complete Session Evaluation

@biju_thomas @oraclenotes

tools and techniques - amazon s3 · oracle enterprise manager cloud control 13c database express...

Documents