high performance database applications with pure query and ibm data studio.bacvanski
TRANSCRIPT
1801
Developing High Performance Database Applications with pureQuery and IBM Data Studio
Dr. Vladimir Bacvanski, SciSpike [email protected]
0
1801
Show of Hands: What Data Access Technology Have You Used?
• What’s most important to you? o Productivity o Performance o Security o Portability
1
JDBC
iBatis/MyBatis
SQLJ
Hibernate EJB Entity
Beans
JPA
EJB: Enterprise Java Beans JPA: Java Persistence API
1801
mashup http Stored
Procedures
JSON
QoS goals JSP
XML
JDBC
Runstats
Response Time!
SQL
Spring
REORG
Partition strategy
Application Developer
SQLJ
JDBC
JPA
MyBatis, . . .
Spring Why does this query take so long?
I can’t believe I got called out last week. I wish I could see how these queries will run in production.
Writing Java code is so easy with this eclipse environment. I wish it was that easy to get the SQL right.
This ORM doesn’t allow me to leverage all my database’s SQL.
Static SQL? Sounds like another delay to getting my program deployed
Sometimes I need POJOs, sometime JSON, sometimes XML, what should I use?
Java Data Access – Two Views of the World
Database Developer & Administrator
Another runaway query! Where are these coming from? JDBC? Hmmm…
Inconsistent response time? How long will it take me to find the offending application sending bad SQL this time?
These ad-hoc queries are dangerous. We need a library of tested SQL interfaces.
Can I examine the SQL “before” the application is deployed?
Another GRANT request? This security administration is out of control.
1801
Data Mapping Approaches • Application-Centric
o Top-Down o Object Domain Model o Object Relational Mapping o Well supported in dynamic
languages and frameworks
• Hybrid
o Meet in the middle o Can be challenging w/o comprising
• Data-Centric o Bottom-UP o Start with Relational Data Model o Not well supported in dynamic
languages and frameworks
3
Persistence Layer
Top Down
Bottom Up
Meet in the Middle
1801 4 4
JPA, Hibernate, EJB vs. The Database
! DBA and SQL developer chasm Where is the SQL coming from? What is it? Where is it? How do we tune it? How de we manage it?
! Performance Concerns: Some vendors claim (unsurprisingly) that
Managed objects performs fine. There are many user claims of bad
Managed object performance is bad on the web.
! As always, the truth is in the middle.
And will depend on your app server, application, database, etc ..
1801
Introducing pureQuery
pureQuery Components: • Simple and intuitive API
o Enables SQL access to databases or in-memory Java objects o Facilitates best practices
• IBM Data Studio (integrates with Rational Application Developer /Rational Software Architect) o Integrated development environment with Java and SQL support o Improve problem isolation and impact analysis
• Optim pureQuery Runtime o Flexible static SQL deployment for DB2
A high-performance, data access platform to simplify developing, managing, securing, and optimizing data access.
1801
pureQuery Balances Productivity and Control
Managed objects Object-relational mapping
Spring templates
Full SQL control
Code all your SQL
Use SQL templates, inline only
Complex OR mapping and persistence management, but loss of controls
Adds container management option
JDBC / SQLJ
MyBATIS
Hibernate
OpenJPA
Add basic OR mapping and annotated-method style pureQuery
1801
Code Development Productivity • Code Generation, Content Assist • Database aware, Java SQL Editor
Design Phase pureQuery close-up
SQL Performance Metrics
• Find and sort query elapsed time from Java
Java to SQL Integration • Categorize by Java, SQL, Database ,
Packages, track back to line of code
SQL Injection Prevention • Lock down SQL for Dynamic
Static SQL • Lock in Access plans, Improve Security,
Consistent Performance
Problem Determination • Monitor WebSphere Connection
Pool, JDBC Driver, Network • Track back to SQL and line of
code in the application
SQL Replacement • Replace Query w/o changing source
Existing JDBC to Static • Reroute Dynamic Queries to Static
Jump Start Application Design • Generate SQL and Code from Database Objects • Setup basic DAO Pattern
Oracle Support • Replace Query w/o changing
source
1801
Code Example: JDBC Table
Column Type
EMP NAME CHAR(64)
EMP ADDRESS CHAR(128)
EMP PHONE_NUM CHAR(10)
class%Employee%{%%%%%public%String%name;%
%%%%public%String%homeAddress;%%%%%public%String%homePhone;%
%%%%…%}%
java.sql.PreparedStatement%ps%=%con.prepareStatement(%
%%%%%%%%"SELECT%NAME,%ADDRESS,%
%%%%%%%%%PHONE_NUM%FROM%EMP%%
%%%%%%%%%WHERE%NAME=?");%
ps.setString(1,%name);%
java.sql.ResultSet%rs=%ps.executeQuery();%
names.next();%
Employee%myEmp%=%new%Employee();%
myEmp%.setName(rs.getString(1));%
myEmp%.setHomeAddress(rs.getString(2));%
myEmp%.setHomePhone(rs.getString(3));%
names.close();%
1801
Code Example: pureQuery
9
Employee%myEmp%=%db.queryFirst(%%%%%%"SELECT%NAME,%ADDRESS,%PHONE_NUM%FROM%EMP%%%%%%%WHERE%NAME=?",%Employee.class,%name);%
%
Even simpler, if we have a method getEmployee with a Java annotation or XML file with SQL for the query: Employee%myEmp%=%getEmployee(name);%
1801
Develop
Design
Deploy
Optimize
Operate
Models Policies
Metadata
IBM Optim pureQuery • Reduce costs
o Increase system throughput o Improve developer productivity o Move workload to zIIP and zAAP
• Improve quality of service for new and existing Java applications o Improve performance o Lock in access plans o Speed up problem resolution
• Reduce development time for new Java applications o Bridge Java and data o Balance productivity and control o Enhance developer and DBA collaboration
• Enhance security o Limit user access o Minimize SQL injection risk o Improve audit readiness
Develop Code Debug Test
Tune, Package
Tester
Developer
1801
Why should DBAs care ? • DBAs have little to no visibility of application SQL before deployment,
no opportunity for review and optimization
• Problem isolation takes days with contemporary environments such as Java, PHP, .NET, etc. due to inability to trace SQL to Java application and source code
• Constantly increasing Java application workload taxes existing systems – need to fit more work into existing systems
• SQL injection represents an increasing risk to data security
1801
Why should Developers care ?
• Get data access right the first time!
• Get it done faster - Improved productivity
• Single environment that spans Java application and database development
• Improved problem isolation and resolution
1801
How well does it work? – Java applications • In-house testing shows significant performance improvements
• IRWW – an OLTP workload, Type 4 driver
• Cache hit ratio between 70 and 85%
• 23 % improvement in throughput using pureQuery over dynamic JDBC
• 15% - 25% reduction on CPU per transaction over dynamic JDBC
13
-35%
-14%
6%15%
25%
-50%
% in
crea
se/r
edu
ctio
n in
CP
U p
er
tran
sn c
om
par
ed t
o J
DB
C
EJB
2
JPA
pQ M
etho
d Dy
nam
icCl
ient
Opt
. Sta
ticpQ
Met
hod
Stat
ic
% increase/reduction in CPU per transaction compared to JDBC using Type 4 driver
274
360420 446
485524
0
100
200
300
400
500
Norm
alize
d Th
roug
hput
(ITR
)
EJB 2
JPA
JDBC
pQ M
ethod
Dyn
amic
Client
Opt
imizn
Stati
cpQ
Meth
od S
tatic
Normalized Throughput by API for JDBC Type 4 Driver
1801
How well does it work? - .Net applications
• Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio
14
*Any performance data contained in this document were determined in various controlled laboratory environments and are for reference purposes only. Customers should not adapt these performance numbers to their own environments as system performance standards. The results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.
! IRWW – OLTP application
! Application accesses DB2 for z/OS
1801
Optim pureQuery Runtime • Control performance
o Decide at deployment time how the SQL is executed o Understand and lock down the access plan for SQL o Replace suboptimal SQL without changing the application
• Control security o Prevent SQL injection o Prevent execution of unauthorized SQL o Better manage database security
• See inside applications that are driving your database o Understand where SQL comes from o Understand when frameworks and ORM’s are getting in the way
• Simplify problem determination and troubleshooting o Correlate problem SQL with applications, ORM’s and frameworks
15
1801
How do I start with pureQuery? • Existing applications
o Optimize existing JDBC (and .NET) applications o No code changes needed o Have to go through the client optimization process to get to static
SQL • New applications
o Use the pureQuery API o Development codes using one API regardless of whether it is
deployed dynamically or statically o DBA deploys statically o No need to go through client optimization process
• Other o JPA, MyBatis, Hibernate
16
1801
pureQuery Facilitates Best Practices • Supports both inline SQL and Java annotations (method)
• Intuitive interfaces for common data retrieval and manipulation scenarios hides JDBC complexity
o Query First
o Homogeneous Batch
• Reduce network trips to the database
o Query Over Java Collections
o Heterogeneous Batch
• Use custom result handlers to map results to POJO’s, XML, JSON, …
17
1801 18
Data Studio SQL Capture
! Capture – Determine SQL coming from application
! Review – Review SQL to identify issues – Optimize
• Replace SQL with enhanced to SQL – Restrict
• SQL approved list
! Configure – Flag SQL to be in a package and configure
driver properties – Bind
• Create packages according to configure
! Execute – Switch to run with new configuration
Capture
Review Optimize Restrict
Configure Bind
Execute
1801
A Typical Application Architecture with pureQuery
19
Presentation Layer
Business Service Layer
Data Access Layer
pureQuery
Database
Using the pure-query API to access the
database. Provides a technology
neutral API to the data used by the business
services
Never use the pureQuery API directly. Gets data from the Data Access
Layer
Implements the U/I or network protocols using the business services
pureQuery makes this layer easy, fast,
consistent and traceable
1801
RAD or RSA / IBM Data Studio Data Centric Development Scenario
Presentation
• JSF2, Ajax …
Application
• Business Logic
Objects
• Access to data
Tables
• Data
Write in Java Using RAD/RSA/Eclipse
Write in Java with pureQuery Using Data Studio in RAD
Access generated Java data objects from code developed in RAD
1801 21
1801
How to get pureQuery?
• Bundled Offerings
o Part of DB2 LUW Advanced Enterprise Server Edition 10.1 and higher • Can be deployed on any number of applications connecting to that DB2
server
o In DB2 Connect Application Server Advanced Edition
o Comes with DB2 Connect Unlimited Advanced Edition for System z
• Stand-alone Offerings o InfoSphere Optim pureQuery Runtime for LUW
o InfoSphere Optim pureQuery Runtime for z/OS
1801
Conclusion: pureQuery Advantages
• Excellent performance o Static and dynamic SQL is captured during test and optimized
before deployment o Enables lock-in of access path
• Great productivity o Excellent tool support through IBM Data Studio
• Shell share with Rational tools
o Mapping from SQL to Java captured and traceable o Facilitates collaboration between DBA’s and developers
• Performance tuning, impact analysis
• Better security o Limits SQL injection o Controlled database access
23
1801
Questions?
24
1801
Getting in Touch
• Dr. Vladimir Bacvanski o Email: [email protected] o Blog: http://www.OnBuildingSoftware.com/ o Twitter: http://twitter.com/OnSoftware o LinkedIn: http://www.linkedin.com/in/VladimirBacvanski