1 j2ee: black box in the oracle world don bergal, confio software august 17, 2006
TRANSCRIPT
![Page 1: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/1.jpg)
1
J2EE: Black Box in the Oracle World
Don Bergal, Confio Software
August 17, 2006
![Page 2: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/2.jpg)
Objective: Answer a Question
“Why does a DBA need visibility upstream in the application in order to manage the database?”
![Page 3: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/3.jpg)
An alternate title for today:
“How to be a better DBA by watching something other than the Oracle database”
![Page 4: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/4.jpg)
Agenda
J2EE Black Box in an Oracle World
a) The Black Box Application Problem
b) The DBA Hot Seat
c) System Statistics versus Wait Time Diagnostics
d) A Better J2EE End-To-End Performance Solution
e) Best Practices Response Time Diagnostics Methodology
f) Q & A
![Page 5: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/5.jpg)
The Black Box Application Problem
![Page 6: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/6.jpg)
J2EE: Black Box in Oracle World
How can you manage here?
If you cannot see here!
Web customer
Internal user
Batch job
Billing
Financial System
Web store
Black Box:
Applications and End Users
![Page 7: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/7.jpg)
See the applications that impact database performance
Web customer
Internal user
Batch job
Billing
Financial System
Web store
Database
App Servers Web Users
![Page 8: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/8.jpg)
Result: Disconnect Between IT Groups?
DBA’s
Databases• Database Management
• Database Development
• SQL Development
• SQL Tuning & Optimization Tools
• Ratio Based Monitoring & Tuning
J2EE Web & App Tiers
Java Development
• Client Web/Applet/Application
• JSP/Servlet Development
• EJB Container Development
• JDBC Development
• Uses Desktop J2EE Profilers
No Visibility into the Database Tier
No Visibility into the J2EE Tier
![Page 9: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/9.jpg)
Effect: Black Boxes leave DBA defenseless
‘Silo-based, Single System’ tools only shows database system perspective (even though they are blamed for problem in any system). Can’t tell when to hand-off problem to another group (Dev/SysAdmin/etc…)
Blamed for all layers, but can only see the database
Black Box
Black Box
![Page 10: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/10.jpg)
The DBA Hot Seat
![Page 11: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/11.jpg)
Finger Pointing Happens When You Can’t See the Problem
What’s wrong with your *&!@$# database?
It’s a database issue!
Java/Web DeveloperManagement DBA
Guilty until proven innocent
It’s a configuration
issue!
Software Vendor
It’s a database issue!
Network Engineers DBA Hot Seat
![Page 12: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/12.jpg)
Do Customers See the Problem Before You?
Java Developer
DBA
Not my code…
What performance problems ?!?
SLA ViolationU
ser
Wait
Tim
eWarning!
Not my Network,
Servers, Load Balancer, Firewall…
System Admin
![Page 13: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/13.jpg)
Hot Seat comes from UNCLEAR Ownership
Need evidence to show where the problem exists, avoid “Finger Pointing” to the database
It’s an Application issue!
It’s a Database issue!
Java App OwnerDBA
![Page 14: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/14.jpg)
14
DBA
NetworkingDeveloper
SystemAdmin
Performance Suspects Line Up…
![Page 15: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/15.jpg)
Lack of Visibility is Source of Problem
J2EE layer
App Layer Oracle eBusiness
Oracle 9iAS Peoplesoft
SiebelJD Edwards
SAPCustom apps
Oracle databaseWeb Server
Web page requestApache
WebLogicWebSphere
IIS
Service Levels Measured Here…
…Blame assigned Here
Typical Multi-Tiered J2EE System Leaves the DBA Blind
![Page 16: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/16.jpg)
Why Does the DBA Need J2EE Visibility?
You can’t control what you can’t seeYou can’t show/demonstrate it’s not a DB IssueYou can’t manage your database if you are blind to the
source of the requests
![Page 17: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/17.jpg)
Traditional Performance Tools versus Wait-Time Diagnostics
![Page 18: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/18.jpg)
Current Tools – Never Cross the Silo Boundaries
Tune inside each “Silo” to improve end-end performance
“System Health” or ratio-based approach to performance…
Statistics…% Utilization…Event Counts… No direct relationship to End-User experience or SLA Weak linkage between tiers limits collaboration
CPU
Memory
Sessions
DatabaseJ2EE ServerWeb Server
CPU
Heap Size
Threads
CPU
Reads
Processes
![Page 19: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/19.jpg)
Standard J2EE App Performance Monitoring: How Does This Help Identify User Response?
Idle threads
Number of requests
Throughput
Waiting Requests
Memory (Heap) Usage
![Page 20: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/20.jpg)
J2EE Profilers: High Load, Test Data Only
![Page 21: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/21.jpg)
OS Level System Monitoring (rstatd/perfmon)
![Page 22: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/22.jpg)
Network System Monitoring
![Page 23: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/23.jpg)
Traditional Tools vs. Wait-Time Methodology
Ratio-Base traditional tools show “Oracle system health” but no link to end user service
Use Wait Time to correlate user requests to SQLs and performance impact
![Page 24: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/24.jpg)
A Better J2EE End-To-End Performance Solution
![Page 25: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/25.jpg)
Requirements for Oracle & J2EE Monitoring
Use Oracle Wait-Time Based Monitoring Visibility across Java and Oracle layers Lightweight JVM Method Tracker Agnostic JVM server support Fast installation and data collection
techniques Cost Effective EASE OF USE – MAKE IT OBVIOUS!
![Page 26: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/26.jpg)
Wait-Time Based Performance Analysis
Tune by determining where client request time is spent
Determine where Oracle resources are spent during the execution of SQL execution
Emerging best-practice for database tuning• “You can’t tell how long something took by counting how many
times it happened. … If you’re only measuring event counts, then you’re not measuring what the users care about.”
—Optimizing Oracle Performance, O’Reilly Press
Oracle is building up Wait-based tuning capabilities with each database release…10g has over 800 wait-time based performance counters
![Page 27: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/27.jpg)
Oracle Wait Interface Book
Free book at www.confio.com
![Page 28: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/28.jpg)
A Wait-Time J2EE Performance Methodology: 3 Key Requirements
1. Follow every transaction individually2. Identify every “step” along the path3. Measure the “wait-time” at each step
Result: Minimize time to resolution and division of work by clearly identifying which step is performing poorly.
![Page 29: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/29.jpg)
Alternative Monitoring Architectures
![Page 30: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/30.jpg)
Best Practices Wait Time Diagnostics Methodology(Day in the life on the performance team)
![Page 31: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/31.jpg)
4 Steps for J2EE to Oracle Visibility
1. Capture Every User Request (end-to-end transaction)2. Capture Every Step Inside Each Tier3. Measure Response Time of Each Step 4. Identify Possible Causes Based on Wait-Times
Since every step is visible, you can always see the problem
![Page 32: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/32.jpg)
4 Steps for J2EE to Oracle Visibility
1. Capture Every User Request (end-to-end transaction)2. Capture Every Step Inside Each Tier3. Measure Response Time of Each Step 4. Identify Possible Causes Based on Wait-Times
Since every step is visible, you can always see the problem
![Page 33: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/33.jpg)
1. Capture Every User Request
Identify the slowest performing end-to-end Requests
Slowest performing can be:
• Longest average request-response time
• Largest accumulated Wait-Time in application (across all sessions)
![Page 34: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/34.jpg)
1. Capture Every User Request
PlaceOrder.jsp Request has most serious delays
![Page 35: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/35.jpg)
2. Capture Every Step Inside Each Layer
Identify the slowest steps within the slowest Request
![Page 36: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/36.jpg)
Key Requirement: Correlate Application Response to SQL Response Times
Question: Which SQL statements are causing Wait Time for my application?
![Page 37: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/37.jpg)
2. Capture Every Step, Identify Slowest
UpdateOrder Total (SQL) is slowest in PlaceOrder.jsp(Most Wait-Time accumulated)
![Page 38: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/38.jpg)
3. Measure Response Time of Each Step
Focus on steps that consume the most time
![Page 39: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/39.jpg)
Detailed Response Time for the Critical Step
Update Order Total• ‘Undo Segment Extension’ Wait-Event is bottleneck
![Page 40: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/40.jpg)
4. Possible Causes Based on Wait Times
Diagnose the step with the greatest Wait Time
![Page 41: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/41.jpg)
4. Possible Causes Based on Wait Times
Execution path and time spent on each ‘Hot Spot”
![Page 42: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/42.jpg)
Identify Chain of Calls
• URL Request -> Method Call –> SQL
• Allocate Wait Time to each Method or SQL Call
• Profiler data Without the System Impact
![Page 43: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/43.jpg)
Recall 3 Key Requirements to Provide the Analysis
2. Identify Every step – Java method
or Wait-Event
3. Measure Wait time for Every Step
1. Identify individual Request
![Page 44: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/44.jpg)
Customer Example
Retail transaction system Online media fulfillment Customer requests running slowly,
unknown reason Ignite for Oracle and Ignite for Java used
to examine situation
![Page 45: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/45.jpg)
Wait Time for “Authorize Transaction” Identifies Problem Steps
![Page 46: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/46.jpg)
Associates Request Wait-Time with Single Procedure
Same query multiple times in single procedure Same row retrieved 7 times Change to select once, cut Wait-Time to less
than half
![Page 47: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/47.jpg)
Summary: Look For…
Continuous monitoring 24/7 – not session/period specific
Production data – not synthetic “test” transactions
Correlate Application & Oracle – where most of the time is spent
Individual transactions, not system-wide statistics
![Page 48: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/48.jpg)
Visibility for the DBA
DBA has a need to see the J2EE Application Solve database problems by understanding the
other systems Look for Wait Time – that’s where the action is Achieve collaboration between Developers,
Architects and DBA’s
DBA can be the leader, not the one to take the blame!
![Page 49: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/49.jpg)
About Confio Software
Wait-Time based performance tools for continuous, low impact monitoring
Ignite for Oracle – Deep, detailed visibility into SQLs and Wait-Events
Ignite for Java – End-to-End Application through Oracle visibility, correlates Java and SQL Wait-Time
Ignite for SQL Server – Coming soon, customer testing
Free Trial at www.confio.com
![Page 50: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/50.jpg)
About the Author
Don Bergal, Confio Software Responsible for product management,
among other things… 18 years software/telecom industry
experience [email protected]
![Page 51: 1 J2EE: Black Box in the Oracle World Don Bergal, Confio Software August 17, 2006](https://reader036.vdocument.in/reader036/viewer/2022062718/56649e955503460f94b99495/html5/thumbnails/51.jpg)
Q & A