purequery deep dive part 2: data access...

pureQuery Deep Dive Part 2: Data Access Development

pureQuery Deep Dive Part 2: Data Access Development

Dan GalvinGalvin Consulting, Inc.

Dan GalvinGalvin Consulting, Inc.

© 2010 Galvin Consulting, Inc.

AgendaAgenda

• The Problem…

• Data Access in Java

• What is pureQuery?

• How Could pureQuery Help within My Data Access Architecture?

• Performance Improvements with pureQuery

• Best Practices In Java Data Access Development


What Is the Problem?What Is the Problem?• Our Efforts Want to Use Java to

Develop Business Critical Applications

• Access to Operational Data Stores

• High Volume of Real-Time Transactions

• The Great Divide – Data vs. Application

• Is Data Access My Responsibility?


What Are the Goals?What Are the Goals?• Increase Productivity

• Developer, Data Specialists, Performance Experts

• Connect the Dots in Problem Resolution Faster

• Close the Gap!

• Deliver More Useful Solutions

• Improve Performance• Significantly Reduce CPU Consumption

• Improve Access Path Reliability

• Reduce Costs• Reduction in Capacity Requirements

• Improved Productivity

• Reduction in Time to Market

HOW DO WE ACCESS DATA TODAY?

HOW DO WE ACCESS DATA TODAY?


Java Data Access MethodsJava Data Access Methods• JDBC Frameworks• Entity Beans• Java Data Objects• O/RMs – Mappers and Wrappers

• Hibernate• Java Persistence API• Toplink• iBATIS


JDBC and Frameworks Pros and ConsJDBC and Frameworks Pros and Cons• JDBC is mature• Competent resources are available in the

workforce• Frameworks can help to eliminate much of the

tedious coding• Tooling exists to help with coding• Straight JDBC is tedious and time-consuming• Industry Frameworks • JDBC is executed as dynamic SQL only, even

for SQL that is static in nature.


Static vs. Dynamic SQL ExecutionStatic vs. Dynamic SQL Execution


ORM Pros and ConsORM Pros and ConsAllows Data Model and Object Model to be developed and maintained separatelyInherent capability to reduce calls to the database with lazy fetch, managed objects and intelligent updatesSQL is Generated at runtime by the implementationRequires developers to learn a new language –JPQL, HQLJDBC implementation. Dynamic SQL only. To maintain model independence benefit, you have to use “meet in the middle” approach…which is extremely complex

© 2010 Galvin Consulting, Inc.12

• Application-Centric• Top-Down• Start with Object Domain

Model• ORM Mapping• Well supported in dynamic

languages and frameworks

• Hybrid• Meet in the middle• Can be challenging w/o

comprising

• Data-Centric• Bottom-UP• Start with Relational Data

Model• Not well supported in

dynamic languages and frameworks

Persistence Layer

TopDown

BottomUp

Meet in theMiddle

Data Mapping ApproachesData Mapping Approaches


• I don’t need to know Java…just show me the SQL

• Here is a long running query…who owns it?

• Do they think we just have unlimited capacity?

• Who is going to prevent SQL injection in the application code?

• What is J2EE? Isn’t this a Java Application?

• Why don’t developers know how to tune?

Difference of ConcernsDifference of Concerns

DeveloperDBA

I need to inflate and persist objects.

I know SQL…well enough at least.

Why do I need to understand how the data is stored? Isn’t that the DBA’s job?

What is an access path?

I need to fulfill the business requirements on time. We can tune the data access later.

mashuphttp

JSON

JSPXML

JDBC

SQL

SpringStoredProcedures

QoS goalsRunstats

Response Time! REORG

Partition strategy

DATA ACCESS METADATADATA ACCESS METADATA


Introducing pureQuery Introducing pureQuery

pureQuery Components:

• Optim Development Studio– Integrated development environment with Java and SQL support – Improve problem isolation and impact analysis

• Simple and intuitive API– Facilitates JDBC best practices – Enables heterogeneous batching– Enables SQL access to databases or in-memory Java objects

• Optim pureQuery Runtime– Flexible static SQL deployment for DB2

A high-performance, data access platform to simplify developing, managing, securing, and optimizing data access

for new and existing applications.


pureQuery APIpureQuery API• Annotated Method

• Interface with Annotations• Generated Implementations• Provides Option for Static Packages

• In-Line• Used for Dynamic Natured SQL• Coded Directly in the DAO Class


pureQuery AnnotationspureQuery Annotations@Select – Specifies that the method executes a SELECT

or SELECT FROM INSERT SQL statement.

@Update – Specifies that the method runs an UPDATE, INSERT, or DELETE SQL statement.

@Call – Specifies that the method executes a CALL SQL statement.

@Handler – Specifies specific handler classes to be used for input and output

@Sql – Specifies that the String contains a SQL statement to be validated (any Java class).


Demo 1: pureQuery Productivity Using Optim Development Studio

Demo 1: pureQuery Productivity Using Optim Development Studio

Layered ArchitectureLayered Architecture• Simplify development, but …

– Complex Problem Resolution– Performance– Obscure impact analysis– Impede capacity planning

• Layers obscure linkages– Where is the problem?– Which application is running?– Which application does this

SQL belong to?

Database LayerApplication Layer

Data A

ccess Logic

Persistence Layer

DB

Java Driver

EJB Query Language

Connection

Pool

BusinessLogic

1

2

3

Network

A

A

A


Review SQL coming from applications

Metadata Provided by pureQueryMetadata Provided by pureQuery

• Consolidate SQL by replacing literals with parameter markers at runtime• Optimize database resources (e.g. dynamic statement cache) usage


Demo 2: Data Access Metadata in Problem

Resolution

Demo 2: Data Access Metadata in Problem

Resolution


Performance with pureQueryPerformance with pureQuery

• Static SQL Execution• Access Path Stability• SQL Operations – Cursors versus

SELECT INTO• Heterogeneous Batching


• IRWW – an OLTP workload, Type 4 driver• Cache hit ratio between 70 and 85%• 23 % improvement in throughput using pureQuery over dynamic JDBC• 15% - 25% reduction on CPU per transaction over dynamic JDBC

How well does pureQuery work on z/OS?How well does pureQuery work on z/OS?

-35%

-14%

6%15%

25%

-50%

% in

crea

se/re

duct

ion

in C

PU p

ertr

ansn

com

pare

d to

JD

BC

EJB

2

JPA

pQ M

ethod

Dyn

amic

Clien

t Opt

. Sta

ticpQ

Meth

od S

tatic

% increase/reduction in CPU per transaction compared to JDBC using Type 4 driver

274360

420 446485

524

0

100

200

300

400

500

Nor

mal

ized

Thr

ough

put (

ITR

)

EJB

2

JPA

JDBC

pQ M

etho

d Dy

nam

icCl

ient

Opt

imiz

n St

atic

pQ M

etho

d St

atic

Normalized Throughput by API for JDBC Type 4 Driver


pureQuery Increased Throughput on DB2 LUWpureQuery Increased Throughput on DB2 LUW

On identical hardware, pureQuery API Static

(Method Style) more than doubled the transaction

throughput.

* See speaker notes


Best PracticesBest Practices

• Configuration• Data Access Architecture• Scope and Granularity• Static SQL Execution• Batching• Transactions in Data Access

Configuration with spring-pdqConfiguration with spring-pdq<beans>

<bean id="CustomerDAO" class="com.inferdata.db.dao.CustomerDAO">

<constructor-arg ref="custData" /></bean><bean id="custDataClass" class="java.lang.Class“ factory-

method="forName"><constructor-arg value="com.inferdata.db.dao.pdq.CustData"

/></bean><bean id="custData“

class="org.springmodules.purequery.PdqAnnotatedMethodFactory“ factory-method="getData">

<constructor-arg ref="custDataClass" /><constructor-arg ref="dataSource" /><constructor-arg ref="pdqProperties" />

</bean>

</beans>


DAO Pattern ImplementationDAO Pattern Implementation• Insulate the Service Object from the specific

Data Access implementation• Provide the SO with a consistent API• Group together operations on associated

data structures which occur within one Unit of Work (UOW) operation

• Implement the DAO Pattern with a concrete class that sits in front of the pureQuery Interfaces.


Data Access ArchitectureData Access Architecture


Scope and GranularityScope and Granularity• DAO Objects should encapsulate data

access logic for one SO call.• One SO may invoke multiple DAOs• One DAO may invoke multiple pureQuery

Interface methods across one or more interfaces

• Interfaces should be generally organized by major data objects

• Consider limited refactoring if one SO call invokes an excessive number of DB2 packages


Static SQL ExecutionStatic SQL Execution• If the statement is static in nature – execute

it statically• Even if a statement is dynamic in nature, if

it is executed frequently enough, look for small numbers of patterns which are static

• Do Not Use Literals in SQL, unless they really never change.

• Regardless of your Data Access Methodology, consider using Client Optimization to obtain static execution

• Use wsdb2gen with O/RMs to bind statically your access and allow for easier SQL review


BatchingBatching• Homogeneous Batching allows for one line-

turn to the DBMS with the same statement• Heterogeneous Batching can be used with

multiple statements in one line-turn• The larger your batches, the more savings

you will realize• Heterogeneous Batches can contain 0 to

many Homogeneous Batches• Code Batch Control in the DAO• Returns UpdateManyException if there is

an error in the batch. Statements are handled as non-Atomic


Transactions in Data AccessTransactions in Data Access• As a general rule, always use an explicitly

defined transaction when invoking a DBMS• …Even for Read Only statements• …Even if you think you are only going to

make one call in the transaction• With DB2, an implicit transaction will be

created for each call not contained within an existing transaction

• With Implicit Transactions, you will pick up the default transaction isolation


Resources Resources • Web page: IBM pureQuery platform:

ibm.com/software/data/optim/purequery-platform/

• Community page: Integrated Data Management (links to downloads, forums, articles, and more): ibm.com/developerworks/spaces/optim

• Redbook: Using Integrated Data Management To Meet Service LevelObjectives: www.redbooks.ibm.com/abstracts/sg247769.html?Open

• Tutorial: Optimize your existing JDBC applications with pureQuery ibm.com/developerworks/edu/dm-dw-dm-0808titzler-i.html

• Demo: pureQuery for DB2 for z/OS (Improving ROI for existing applications):ibm.com/developerworks/offers/lp/demos/summary/im-purequery4zos.html

• Demo: Java acceleration solution demo: ibm.com/developerworks/offers/lp/demos/summary/im-optimsolutionsforjava.html

http://www.ibm.com/software/data/optim/purequery-platform/

http://www.ibm.com/developerworks/spaces/optim

http://www.redbooks.ibm.com/abstracts/sg247769.html?Open




Resources, continues Resources, continues • Article: What’s new and cool in Optim Development

Studio 2.2ibm.com/developerworks/data/library/techarticle/dm-0906optimdeveloper/

• See the related video series on ChannelDB2http://www.channeldb2.com/video/whats-new-in-optim-development-1

• Video tutorial series: Develop a Java application using pureQuery annotated methodsibm.com/developerworks/views/data/libraryview.jsp?type_by=Demos&search_by=pureQuery+annotated+methods

• Product documentation:http://publib.boulder.ibm.com/infocenter/idm/v2r2/index.jsp

http://www.ibm.com/developerworks/data/library/techarticle/dm-0906optimdeveloper/

http://www.ibm.com/developerworks/data/library/techarticle/dm-0906optimdeveloper/

http://www.channeldb2.com/video/whats-new-in-optim-development-1

http://www.channeldb2.com/video/whats-new-in-optim-development-1

http://www.ibm.com/developerworks/views/data/libraryview.jsp?type_by=Demos&search_by=pureQuery+annotated+methods

http://www.ibm.com/developerworks/views/data/libraryview.jsp?type_by=Demos&search_by=pureQuery+annotated+methods

http://publib.boulder.ibm.com/infocenter/idm/v2r2/index.jsp


Contact InformationContact Information

• Email: [email protected]

• Web: www.galvinconsulting.com

• Blog: http://www.channeldb2.com/profile/DanGalvin

• Twitter: @Dan_Galvinhttp://twitter.com/Dan_Galvin

• LinkedIn: http://www.linkedin.com/in/dangalvingci

mailto:[email protected]

http://www.galvinconsulting.com/

http://www.channeldb2.com/profile/DanGalvin

http://twitter.com/Dan_Galvin

http://www.linkedin.com/in/dangalvingci


This session will be held on April 29, 2010.Registration available soon at

ibm.com/developerworks/data/events/idmbriefings.html

Coming next: What’s new in Optim®Performance Management Tooling for DB2®for Linux, UNIX, and Windows

Coming next: What’s new in Optim®Performance Management Tooling for DB2®for Linux, UNIX, and Windows

Get the latest and greatest news about the IBM Optim performancemanagement solution for DB2 for Linux, UNIX, and Windows to helpyou resolve emergent performance problems before they affect thebusiness.

Don’t leave yet. Please fill out the survey!April 29, 2010

http://www.ibm.com/developerworks/data/events/idmbriefings.html

www.galvinconsulting.com

purequery deep dive part 2: data access...

Documents