optimizing your websphere applications with optim and purequery
Post on 12-Jan-2016
50 Views
Preview:
DESCRIPTION
TRANSCRIPT
Optimizing Your WebSphere Applications with Optim and pureQuery
Curt Cotner
IBM Fellow, CTO for IBM Database Servers
Development Tools for Your Java Database Applications
1. Select table
Java Data Access in 5 Simple Steps
2. Name bean & select styles 3. Generate test code
4. Map table to bean 5. Select template SQL CRUD
4
What happened?Generate code from table
The code is generated for Bean (aka Java Object) representing the table
Interface with associated SQL or Sample Application with Inline SQL
Implementation of Interface
Optionally a JUnit test case.
5
Generated Test Cases for Your SQL
• Jump start your migrations!
6
SQL analysisSQL executionSQL validation
Developing with pureQueryUnleash SQL from Java
SQL content assist
7
SQL content assist SQL analysisSQL execution
Developing with pureQueryUnleash SQL from Java
SQL validation
8
SQL validationSQL content assist SQL analysis
Developing with pureQueryUnleash SQL from Java
SQL execution
Run SQL with parameters at design time in the Java program WITHOUT writing a test application
9
SQL executionSQL validationSQL content assist
Developing with pureQueryUnleash SQL from Java
SQL analysis
View explain plans for SQL inside you Java programs
10
pureQuery OutlineSpeed up problem isolation for developers – even when using frameworks
• Capture application-SQL-data object correlation (with or without the source code)
• Trace SQL statements to Java source code for faster problem isolation
• Enhance impact analysis by identifying specific application code impacted by a database changes
• Answer “Where used” questions like “Where is this column used within the application?”
• Use with modern Java frameworks e.g. Hibernate, Spring, iBatis, OpenJPA
11
pureQuery Outline
pureQuery Outline View’s 3 categorizations
• How do you look at the relationship between SQL and Java?
Optim Development Studio
• Problem determination and isolation
– with pureQuery outline go to the source of the problematic SQL
• Improve Hibernate data access calls
– Re-write HQL– Use better
performing native SQL
• With performance metrics Identify the hot spots in your hibernate application
Source code correlation
Database object correlation
Identify and change the HQL
Analyzing Performance
Analyze Use of Sensitive Data
See queries accessing sensitive data
Optim Development Studio
Icon identifies sensitive access
Filter to see use of sensitive data
View only SQL accessing
sensitive data
Filter SQL by action
IBM Data Studio
pureQuery For DBAs
and Applicatio
n Developer
s (v1.2) - Part 1
1515
pureQuery ToolsSQL templates and customizations
Create your own SQL templates
Use templates to write SQL that is frequently reused
Use tabs to change the variable names after inserting SQL statement from the template through SQL context assist
Use newly created merge template in your java code using SQL assist
1616
Java/SQL Development on SteroidsWhy the 10x Productivity Improvement
Optimizing Your WebSphere Applications
DB2 ConnectRecommended Deployment Options
Desktop PCsDesktop PCs
Application serversApplication servers
Web application serversWeb application servers
Several options: Personal Edition or DB2 Connect Server or file server
Several options: Personal Edition or DB2 Connect Server or file server
Co-locate DB2 Connecton the application server
Co-locate DB2 Connecton the application server
Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops
Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops
Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring
Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring
Open Source Persistence
Engine
JPA Persistence Engine
JDBC API
Hibernate, iBATIS, EclipseLink,...
JPA for WebSphere, Apache OpenJPA
pureQueryAPI
Data Studio pureQueryPlain JDBC
Data Web Services, Project Zero, sMash
On-ramps to pureQuery
JCC driver
JDBC
.Net Applications
.Net applications
pureQuery
JPA APIOpen Source
Persistence API Web API
ADO .Net
DB2, Informix, and Oracle now…more coming
Client OptimizationImprove Java data access performance– without changing code
• Captures SQL for Java applications– Custom-developed, framework-based, or packaged applications
• Bind the SQL for static execution without changing a line of code– New bind tooling included
• Delivers static SQL execution value to existing DB2 applications– Making response time predictable and stable by locking in the SQL access path
pre-execution, rather than re-computing at access time– Limiting user access to tables by granting execute privileges on the query
packages rather than access privileges on the table– Aiding forecasting accuracy and capacity planning by capturing additional
workload information based on package statistics – Drive down CPU cycles to increase overall capability
• Choose between dynamic or static execution at deployment time, rather than development time
Optim pureQuery Runtime for z/OS
• In-house testing shows double-digit reduction in CPU costs over dynamic JDBC
•
• IRWW – an OLTP workload, Type 4 driver• Cache hit ratio between 70 and 85%• 15% - 25% reduction on CPU per txn over dynamic JDBC
274
360420 446
485524
0
100
200
300
400
500
No
rmalized
Th
rou
gh
pu
t (I
TR
)
Normalized Throughput by API for JDBC Type 4 Driver
-35%
-14%
6%15%
25%
-50%
% in
cre
ase/r
ed
ucti
on
in
CP
U p
er
tran
sn
co
mp
are
d t
o J
DB
C
% increase/reduction in CPU per transaction compared to JDBC using Type 4 driver
Throughput Increase with .NET• Same IRWW OLTP application used for the Java tests but in .NET
• Application access DB2 for z/OS via Windows Application Server (IIS)
• Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio
pureQuery -- More Visibility, Productivity, and Control of Application SQL
• Capture SQL• Share, review, and optimize SQL• Revise/optimize SQL and validate equivalency
without changing the application• Bind for static execution to lock in service level
or run dynamically • Restrict SQL to eliminate SQL injection
Capture Review Optimize Revise Restrict
Visualize execution metrics
Execute, tune, share, trace, explore SQL
Replace SQL without changing the application
Position in Database Explorer
Visualize application SQL
Have You Heard of SQL Injection?
pureQuery – Stripping Literals from SQL
JDBC app
INSERT INTO T1 VALUES(‘ABC’,2,’DEF’) INSERT INTO T1
VALUES(:h1,:h2,:h3)
pureQueryRuntimeconversion
• pureQuery can identify statements that use no parameter markers, and strip the literals out at runtime• significant performance gains:
• less CPU cost at PREPARE• better use of dynamic statement cache
What Is Heterogeneous Batching?
Data Server
Table 2, operation 2
Table 1, operation 1
Table 1, operation 2
Table 1, operation 3
Table 2, operation 1
Table 3, operation 1
Heterogenous Batching – multiple operations across different tables all execute as one batch
Table 1, operation 4
JDBC Batching vs pureQuery Batching• JDBC batching used by
Hibernate Batcher is currently limited
– Cannot batch entities that map to multiple tables
• Primary and Secondary tables.• Inheritance Join and Table per
class strategies– Cannot batch different operations
against same table• Field level updates• Insert, update
– Cannot batch different entities• pureQuery heterogeneous
batching plug-in– Can batch entities that map to
multiple tables– Can batch different operations
against the same table– Can batch different entities into a
single batch– Combines insert, deletes, updates
into single batch
The advantage of Heterogenous Batching
050
100150200250300350400450500
0 20 40 60 80 100
# of operations per transaction
Elap
sed
Tim
e (m
Sec)
HeteroBatching
No Batching
JDBC Batching
* Preliminary findings based on validation with a test designed to demonstrate heterogeneous batching differences. This is not intended to be a formal benchmark.
OpenJPA and Hibernate --SQL Query Generation
JPA Query
Select emp_obj(), dept_obj()
SQL
Select * from EMP WHERE …Select * from DEPT WHERE …
JPAquery transform
• Hibernate and OpenJPA often rewrite queries• No database statistics are used – entirely heuristic!!!• Can often result in poorly performing queries
Monitoring of Java Applications
Optim Database Performance Manager future directions -- Associate SQL with Java SourceHeat
ChartDashboard
Alerts
SLAs
In-flight analysis
Database: Accounting
Statement text schema E2E elapsed occurrences sort time phys. I/O
SELECT TIME FROM UNIVERSE SAP3 132.13 1323 123.32 1.303
SELECT SALARY FROM PAYMENT … SYSIBM 323.4 221 11.3 32.1
DELETE FROM ACCOUNT WHERE AID = 3…
PROC 23.3 435 32322.3 32.1
TOP 3 currently running SQL Statements
TOP by DS elapsed DS CPU time Physical I/O Sort time
- +
- +
- +
SELECT TIME FROM UNIVERSEStmt text Analyze
Application DS user ID KARN Client IP addr / hostname TPKARN.de.ibm.com Client user ID KARN Client workstation name TPKARN Client application name Jawaw.exe Client accounting N/A application name Online banking application contact hkarn@de.bm.com package West.OLBank class Account method Transfer() source line 314
Time distribution
Force applicationStop SQL
sorting
Resource usageQuery cost estimates 18.456Buffer Pools Data – hit ratio (%) 43.4% Data – physical reads / min 4323 Index – hit ratio (%) 54.2% Index – physical reads / min 3214
Statement information X
DS Proc
USER CPU SYSTEM CPU
Unacc wait
DS sorting
Statement elapsed timeCurrent 132.13 sec last day 239.40 seclast week 15.60 sec
Toughest issue for Web applications – Problem diagnosis and resolution
Web BrowserUsers Web
Server
Application Server
DB2Server
Business Logic
Data A
ccess Logic
Persistence Layer
DB
2 Java Driver
JDBCPackage
EJB
Query Language
What’s so Great About DB2 Accounting for CICS Apps?
z/OS LPARCICS AOR1Txn1 - Pgm1 - Pgm2
CICS AOR2TxnA - PgmX - PgmY
DB2PROD
CICS AOR3Txn1 - Pgm1 - Pgm2
App CPU PLANTxn1 2.1 TN1PLNTxnA 8.3 TNAPLN
DB2 Accounting for CICS apps allows you to study performance data from many perspectives:• By transaction (PLAN name)• By program (package level accounting)• By address space (AOR name)• By end user ID (CICS thread reuse)
This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.
JDBC Performance Reporting and Problem
Determination – Before pureQuery
Application ServerDB2 or IDS
A1
A2
A5
A3
A6
A4
USER1
USER1
USER1
User CPU PACKAGEUSER1 2.1 JDBCUSER1 8.3 JDBCUSER1 22.0 JDBC
What is visible to the DBA? - IP address of WAS app server - Connection pooling userid for WAS - app is running JDBC or CLI
What is not known by the DBA? - which app is running? - which developer wrote the app? - what other SQL does this app issue? - when was the app last changed? - how has CPU changed over time? - etc.
Data A
ccess Logic
Persistence Layer
DB
2 Java Driver
EJB
Query Language
What’s so Great About Optim pureQuery Accounting for WebSphere Applications?
z/OS LPAR
CICS AOR2TxnA (PLANA) - PgmX - PgmY
App CPU TxnA 2.1 TxnB 8.3
Data Studio and pureQuery provide the same granularity for reporting WebSphere’s DB2 resources that we have with CICS:• By transaction (Set Client Application name )• By class name (program - package level accounting)• By address space (IP address)• By end user ID (DB2 trusted context and DB2 Roles)
This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.
Unix or Windows
WAS 21.22.3.4TxnA (Set Client App=TxnA) - ClassX - ClassY
Simplifying Problem Determination Scenario
Application Developer
• Available for each db access
–SQL text generated
–Access path
–Cost estimates
–Estimated response time
–Elapsed & CPU time
–Data transfer (getpages)
–Tuning advice
Database Administrator
• Available for each SQL
–Application name
–Java class name
–Java method name
–Java object name
–Source code line number
–Source code context
–J-LinQ transaction name
–Last compile timestamp
Java
Profiling
pureQuery
DRDA Extentions
Using pureQuery to Foster Collaboration and Produce Enterprise-ready Apps
Application Server
Catalog data for SQL
ApplicationMeta data
DB2 or IDSProd
A4
A1
A1
A6
A6
A2
A2
A3
A3
A4
A4
A5
A5
A1
A4
A5
PerformanceData
Warehouse
Application Developer Database
Administrator
A1
A6A2 A3
A4 A5
Quickly compare unit test
perf results to production
Use pureQuery app metadataas a way to communicate in terms familiar to both DBA and developer
ApplicationMeta data
DB2 or IDSDev System
A1
A6
A2 A3
A4 A5A1
A4
A5
Customer Job Roles – A Barrier to a “Holistic View”
Application Server
DBServer
Data A
ccess Logic
Persistence Layer
DB
Java Driver
JDBCPackage
EJB
Query Language
WebS
phereC
onnectionP
ool
Busines
sLogic
13
54
2
ApplicationDeveloper
SystemProgrammer DBA
NetworkAdmin
DB2 Performance Expert and Extended Insight• Enables early and rapid problem detection to
prevent impact to production systems– Identify resource shortages across CPU, memory,
and file systems, locking conflicts and deadlocks, and data skew
• Increase ability to meet service level agreements
– Provides optimization and tuning recommendations
• Supports trend analysis and growth planning– Maintains and analyzes performance warehouse
• Supports a variety of database workloads – Includes online transaction processing, data
warehouse, and enterprise content management– Monitors Workload Management environment
• Visualizes the end-to-end response time – See where Java database workloads,
transactions, and SQL requests are spending their time across the database client, application server, and network
Provides in-depth database monitoring, problem isolation, and trend analysis for DB2 for
Linux, UNIX, and Windows databases. Add Extended
Insight Feature for visibility into where Java database
workloads, transactions, and SQL requests are spending their
time
Scenario
It seems that the first application server has a problem. Double-click to drill-down.
In this situation, all applications are equally affected, and the problem seems not to be in the data server.
Scenario - continued
Double-click to drill-down and display detail information.
Most of the time is spent for „WAS connection pool wait“ time.
Scenario – continued
5 second wait time indicates that the maximum number of allowed connections is not sufficient…
… which becomes also evident when comparing the parameters and metrics of this client with other clients.
Where to get IBM Data Studio ?
• IBM Data Studio –www.ibm.com/software/data/studio
• FAQs / Tutorials
• Downloads
• Forum / Blogs
• Join the IBM Data Studio user community
top related