my jdbc performance sucks and what i did about it!

©2010 PROS Holdings, Inc. All rights reserved.

My JDBC Performance Sucks and What I Did About It!

©2010 PROS Holdings, Inc. All rights reserved. 2

Who am I? Peter Tran Senior Performance Technical Lead Over 12 years with PROS 8 years working on server-side performance 6 years Oracle tuning 3 years SQL Server tuning [email protected]


Topics Assumptions Transaction Isolation Levels Connection Batching Queries Parsing Misc


Assumptions Will not compare different driver type

- Recommendations should be driver type agnostic Examples in presentation uses Type IV Thin Driver Examples uses SQL Server 2008 and Oracle 10g

databases Will not cover SQL optimization Will not cover JDBC resource leak


Transaction Isolation Levels Big impact on performance Critical for correct business logic java.sql.Connection

Transaction Isolation Level Dirty-Reads Non-Repeatable Phantom Performance

TRANSACTION_NONEN/A N/A N/A Fastest

TRANSACTION_READ_UNCOMMITEDYes Yes Yes Fast

TRANSACTION_READ_COMMITTEDNo Yes Yes Medium

TRANSACTON_REPEATABLE_READNo No Yes Slow

TRANSACTION_SERIALIZABLENo No No Slowest


Connection java.sql.Connection Use connection pooling

- Big performance penalty to create new physical connection to database

1 10000

1

2

3

4

5

6

7

Time to Create Physical Connection

# of Physical Database Connections

Seco

nds


Batching Batch UPDATE and INSERT SQLs Use

- java.sql.(Prepared/Callable)Statement- Added benefit of preventing SQL Injection (security)

Do not use- java.sql.Statement

Remember to setAutoCommit to FALSE SQL statement reuse Reduces parse


Batch Insert Example – Wrong! Wrong way to do batch update.//turn off autocommit

con.setAutoCommit(false);

Statement stmt = con.createStatement();

stmt.addBatch("INSERT INTO employees VALUES (1000, 'Joe Jones')");

stmt.addBatch("INSERT INTO departments VALUES (260, 'Shoe')");

stmt.addBatch("INSERT INTO emp_dept VALUES (1000, 260)");

// submit a batch of update commands for execution

int[] updateCounts = stmt.executeBatch();


Batch Insert Example Right way to do batch update using PreparedStatement// turn off autocommit


PreparedStatement stmt = con.prepareStatement( "INSERT INTO employees VALUES (?, ?)");

stmt.setInt(1, 2000);

stmt.setString(2, "Kelly Kaufmann");

stmt.addBatch();


stmt.setString(2, "Bill Barnes");

stmt.addBatch();

// submit the batch for execution

int[] updateCounts = stmt.executeBatch();


Batch Query Example Right way to do batch query using CallableStatement

// turn off autocommit


CallableStatement stmt = con.prepareCall("sp_GetEmployees (?, ?)");


stmt.setString(2, "Kelly Kaufmann");

ResultSet rs = stmt.executeQuery()


Batch Insert Test Results

Test 1: Statement – No Batch Test 2: PreparedStatement – No batch/no reuse Test 3: PreparedStatement – No batch/reuse Test 4: PreparedStatement – With batch/reuse

1 2 3 40

1

2

3

4

5

6

7

Batch Tests

Test #

Seco

nds


Batch Pitfalls Forgetting to set autocommit to FALSE Excessively large batch size

- Writer blocks readers- Use up more memory on database server- ORA-01555 snapshot too old


Query Performance Set fetch size

- Settable java.sql.ResultSet java.sql.Statement java.sql.(Prepared/Callable)Statement

- Oracle 11.2g driver defaults to 10- SQL Server v3.0 driver defaults to 128- Reduces Physical/Logical I/O- Uses more memory in client to hold data


Query Performance Fetch only data that is needed

- SQL Optimization – Physical/Logical IOs- Reduce Network

Use appropriate getXXX() method for retrieving data- Minimize data conversion guessing- Eliminates nasty conversion bug


Microsoft Query Set Fetch Size Test Results

Test 1: Query 1M rows – fetch size = 10 Test 2: Query 1M rows – fetch size = 128 (default) Test 3: Query 1M rows – fetch size = 1M

1 2 30

0.05

0.1

0.15

0.2

0.25

0.3Microsoft Query Fetch Size Tests

Test #

Seco

nds


Oracle Query Set Fetch Size Test Results

Test 1: Query 1M rows – fetch size = 10 (default) Test 2: Query 1M rows – fetch size = 500 Test 3: Query 1M rows – fetch size = 1M

1 2 30

0.05

0.1

0.15

0.2

0.25

Oracle Query Fetch Size Tests

Test #

Seco

nds


Parsing Parsing is expensive!

- Scalability killer for high volume OLTP applications Not as big of concern for OLAP applications Consumes CPU intensive

- Database engine not doing real-work Increase latch contention Increase memory usage SQL statement cache is a singleton

- Oracle: Shared Pool- SQL Server: Plan Cache


Parsing Best place to fix is in the application code

- Use PreparedStatement or CallableStatement No Parse – Predicates uses bind variables and

Statement reused Soft Parse – Predicate uses bind variables but

Statement not reused Hard Parse – Predicate with literal values

Parse Type PerformanceParse Once BestSoft Parse GoodHard Parse Bad


Parse Fix - Hack Do not have access to application code May select a suboptimal query plan Buggy


Oracle Parsing Example Oracle 11.2g JDBC Driver

Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once

1 2 302468

1012

Parse Test

Test #

Seco

nds

http://download.oracle.com/docs/cd/E11882_01/java.112/e10589/toc.htm


Microsoft Parsing Example Microsoft JDBC Driver version 3.0


1 2 301234567

Parse Test

Test #

Seco

nds

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=a737000d-68d0-4531-b65d-da0f2a735707&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+MicrosoftDownloadCenter+(Microsoft+Download+Center)&utm_content=LocalHost


jTDS Parsing Example for SQL Server jTDS JDBC Driver version 1.2.5


1 2 30

0.51

1.52

2.53

3.54

4.5

Parse Test

Test #

Seco

nds

http://jtds.sourceforge.net/


Parse Fix - Hack SQL Server

- Use PARAMETERIZATION = FORCED- Set at database level (Database Properties -> Options)- Restrictive: Check Books-On-Line (BOL)- YMMV – 5% improvements


Parse Fix - Hack Oracle

- CURSOR_SHARING = SIMILAR | FORCED Instance level or session level Use trigger to hack session level for each login

CREATE OR REPLACE TRIGGER after_usr_logon

AFTER LOGON ON <USER>.SCHEMA

BEGIN EXECUTE IMMEDIATE 'ALTER SESSION SET CURSOR_SHARING=FORCE';

END;

Super buggy Use SIMILAR – mixed work-load OLTP/OLAP Use FORCE – pure OLTP Use EXACT (default) – DSS/DW OLAP


Driver Specific Improvements Requires casting to drive specific implementation Your Performance May Vary (YPMV) Microsoft JDBC Driver version 3.0

- Adaptive Buffer – reduces memory footprint in client to minimize Out of Memory but requires more round-trip to database (default=adaptive as of version 2.0)

Oracle 11.2g JDBC Driver- Statement and ResultSet Caching- Connection Caching

jTDS JDBC Driver version 1.2.5- Implicitly caches PreparedStatement – not a new

feature

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=a737000d-68d0-4531-b65d-da0f2a735707&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+MicrosoftDownloadCenter+(Microsoft+Download+Center)&utm_content=LocalHost

http://download.oracle.com/docs/cd/E11882_01/java.112/e10589/toc.htm

http://jtds.sourceforge.net/


Database JDBC Driver Tuning Parameters

SQL Server- Implicit Unicode conversion

Use setStringParametersAsUnicode Bypass index and use table scan

http://www.sql-server-performance.com/articles/per/java_drivers_p1.aspx


My JDBC Performance Workout Plan Maximize the usage of batching Set appropriate fetch size for queries Set the appropriate isolation level Reduce the hard/soft parses

Before After


Q&A

my jdbc performance sucks and what i did about it!

Documents