top 10 sql performance tips & tricks for java …...title top 10 sql performance tips &...

Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|

Top10SQLPerformancetips&tricksforDevelopers

GeraldVenzlMasterProductManager

ServerTechnologiesOracleDevelopment

@GeraldVenzl

geraldonit.com

Copyright©2018,Oracleand/oritsaffiliates.Allrightsreserved.|

Gerald’s SafeHarborStatementNothingsaidhereisaguaranteeforsolvingyourparticularproblem.Whilethetechniquespresentedherecanhaveahugepositiveimpactonperformanceitisnotguaranteedthattheywillfixyourparticularproblem.

Performanceisrelative. Itdependsonahugeamountofdifferentfactorsandcan’tbecompared1:1.Thenumbersinthisdeckreflectatiny2GBDockerenvironmentwithonlyonevirtualCPU,keepthatinmind.

2


Whatisperformance?

3


Whatisperformance

Performance=?

Asimpleformula

4


Whatisperformance

Performance=latencyxthroughput

Asimpleformula

5


Whatisperformance

• Latency:Thetimetoprocessaunitofwork• Throughput:Theamountofunitsofworkthatcanbeprocessedinparallel• Aunitofworkiseitherprocessed(CPU) orhastowait(I/O)• Goalsare:– Processaunitofworkasefficientlyaspossible– Reducewaittimeasmuchaspossible

– Avoidunnecessaryresourceconsumption(CPU&I/O)

6


Whatisperformance


10,000workunits/s=?

Asimpleformula

7


Whatisperformance


10,000workunits/s=1msx10wu

Asimpleformula

8


Whatisperformance


10,000workunits/s=10msx100wu

Asimpleformula

9


DatabaseWorkingStates

10


Thedatabaseasa

11


Thedatabaseasablackbox

12


Adatabaseiseither

13


Adatabaseiseitherworking

14


Adatabaseiseitherworkingoridle

15


Theworkingstatesofthedatabase

16



17

Busy



18



19



20

Waiting



21



22



23


TrackingtheworkingstatesoftheOracle Database

24


TrackingtheworkingstatesoftheOracle Database

25

=Statistics


TrackingtheworkingstatesoftheOracleDatabase

26

=Statistics

=WaitClasses


OracleDatabasestatistics• 2011statisticsintotalinOracleDB18c• Query:v$sysstat,v$mystat,v$sesstat (,v$statname)• Forexample:SELECT n.name, s.valueFROM v$mystat s, v$statname nWHERE s.statistic#=n.statistic#AND value > 0ORDER BY n.name;

27


OracleDatabasewaitclasses• Administrative:WaitsresultingfromDBAcommandsthatcauseuserstowait(indexrebuild,etc.)

• Application:Waitsresultingfromuserapplicationcode(lockwaitscausedbyrowlevellockingorexplicitlockcommands)

• Cluster:WaitsrelatedtoOracleRealApplicationClustersresources(globalcacheresourcessuchas'gc cr blockbusy')

• Commit:Thiswaitclassonlycomprisesonewaitevent- waitforredologwriteconfirmationafteracommit(thatis,'logfilesync')

• Concurrency:Waitsforinternaldatabaseresources(latches,cursorpins,etc.)

• Configuration:Waitscausedbyinadequateconfigurationofdatabaseorinstanceresources(undersizedlogfiles,sharedpool,etc.)

• Idle:Waitsthatsignifythesessionisinactive,waitingforwork('SQL*Netmessagefromclient')

• Network:Waitsrelatedtonetworkmessaging('SQL*Netmoredatatodblink’)

• Other:Waitswhichshouldnottypicallyoccuronasystem(forexample,'waitforEMONtospawn')

• Queueing:Delaysinobtainingadditionaldatainapipelinedenvironment. (parallelqueries,orDBMS_PIPEPL/SQLpackages)

• Scheduler:ResourceManagerrelatedwaits('resmgr:becomeactive’,etc.)

• SystemI/O:WaitsforbackgroundprocessI/O(forexample,DBWRwaitfor'db fileparallelwrite')

• UserI/O:WaitsforuserI/O(forexample'db filesequentialread')

28


OracleDatabasewaitclasses• 1884 waiteventsintotalinOracleDB18c• Query:v$system_event,v$session_event,v$session_wait (,v$event_name)• Forexample:• SELECT sid, event, p1, p2, p3

FROM v$session_waitORDER BY sid, event;

29


Tip#1:Roundtrips

30

Copyright©2018,Oracleand/oritsaffiliates.Allrightsreserved.| 31

Roundtrips


Roundtrips

• Avoidunnecessaryroundtripstothedatabase• Everyroundtripincludestimethatyouspendonthenetwork• Thatdoesnotmeantoavoidprocessinganythingatthedatabase

32


Tip#2:(Auto)Commit

33


Commits– saviorofyourdataHowdoestheDBmakesureyourdataisstoredondisk?

34

INSERTINTOpurchase(…)

VALUES(:1,:2,:3,…);COMMIT

BufferCache REDOlogs


Commits– saviorofyourdataWhathappensonaCOMMIT?

35

GenerateSCN



36

GenerateSCN

LGWRwritestoREDOlogs



37

GenerateSCN


Releaselocksontables



38

GenerateSCN



Removesafe-points



39

GenerateSCN



Removesafe-points

Performcommitcleanout



40

GenerateSCN



Removesafe-points


MarkTXN

complete



41

GenerateSCN



Removesafe-points


MarkTXN

complete


Autocommit

• ThedrivercommitseveryDMLonyourbehalf,implicitly• Thatis,forevery singleINSERT,UPDATE,DELETE,MERGEwillbeaCOMMIT• Impliesasecondroundtrip tothedatabasesforevery DMLoperation• Forcesdatabasetowritetodisk,perhapsunnecessarily• MeansyoucanneverROLLBACKyourchanges• Itisturnedonbydefault– connection.setAutoCommit(false)

WhatistheAutocommit?

42

Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.| 43

PreparedStatement stmt = conn.prepareStatement("INSERT INTO TEST (id, text, created_tms, last_upd_tms) VALUES (?,?,SYSDATE,SYSDATE)");

long start = System.currentTimeMillis();

for(int i=1;i<=rows;i++) {stmt.setInt(1, i);stmt.setString(2, "This is the row with the value of " + i);stmt.executeUpdate();conn.commit();

}

long end = System.currentTimeMillis();System.out.println("Elapsed time(ms) for commit after every row: " + (end-start));

Commits– saviorofyourdata




for(int i=1;i<=rows;i++) {stmt.setInt(1, i);stmt.setString(2, "This is the row with the value of " + i);stmt.executeUpdate();

}conn.commit();

long end = System.currentTimeMillis();System.out.println("Elapsed time(ms) for commit at the end: " + (end-start));



DEMO


0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

Testrun1 Testrun2 Testrun3

Milliseconds

Commiteveryrow

Commitatend

• Threetestrunsinserting10krows

• Darkblueaxisshowselapsedtimewhencommittingaftereveryrow

• Lightblueaxisshowselapsedtimewhencommittingonlyonceafteralldataisloaded

46



Tip#3:Bulkprocessing

47


Bulkprocessing• Whatwouldyouratherdoonatablewith10millionrows:

48


Bulkprocessing• Whatwouldyouratherdoonatablewith10millionrows:PreparedStatement stmt = conn.prepareStatement(

"SELECT value FROM PURCHASE WHERE tms > '2016-09-01'");

ResultSet rslt = stmt.executeQuery();

while (rslt.next()) {val += rslt.getInt(1);

}

49






}

PreparedStatement stmt = conn.prepareStatement("SELECT SUM(value) FROM PURCHASE WHERE tms > '2016-09-01'");

ResultSet rslt = stmt.executeQuery();rslt.next();val = rslt.getInt(1);

50






}

PreparedStatement stmt = conn.prepareStatement("SELECT SUM(value) FROM PURCHASE WHERE tms > '2016-09-01'");

ResultSet rslt = stmt.executeQuery();rslt.next();val = rslt.getInt(1);

51




for(int i=1;i<=rows;i++) {stmt.setInt(1, i);stmt.setString(2, "This is the row with the value of " + i);stmt.executeUpdate();

}conn.commit();

long end = System.currentTimeMillis();System.out.println("Elapsed time(ms) for row by row insert: " + (end-start));

Bulkprocessing




for(int i=1;i<=rows;i++) {stmt.setInt(1, i);stmt.setString(2, "This is the row with the value of " + i);stmt.addBatch();

}stmt.executeBatch();conn.commit();

long end = System.currentTimeMillis();System.out.println("Elapsed time(ms) for set based insert: " + (end-start));

Bulkprocessing


DEMO


4229 4055 4118

76 84 720

500

1000

1500

2000

2500

3000

3500

4000

4500


Milliseconds

Inserteveryrow

Setbasedinsert

• Threetestrunsinserting10krows

• Darkblueaxisshowselapsedtimeofindividualinserts

• Lightblueaxisshowselapsedtimeofsetbasedinserts

55

Bulkprocessing


Bulkprocessing

• Databasesworkbestwithsetbasedprocessing• Processasmuchaspossibleonthesidewhereitmakesmostsense• Generally,asingleSQLstatementexecutionisthefastest• Avoidsunnecessaryroundtrips

56


Tip#4:Bindvariables

57


Bindvariables

SELECTtextFROMTEST

WHEREid=453;

SQLID:5mwwhtqv204ba

SQLID:06jc0z1kcuu6b

SELECTtextFROMTEST

WHEREid=879;

SELECTtextFROMTEST

WHEREid=?;

SELECTtextFROMTEST

WHEREid=?;

SQLID:cknumntjbx8u3


for(int i = 1; i <= rows; i++) {PreparedStatement stmt = conn.prepareStatement(

"SELECT text FROM TEST WHERE id = " + i);ResultSet rslt = stmt.executeQuery();rslt.next(); rslt.getString(1);rslt.close();stmt.close();

}

Bindvariables


PreparedStatement stmt = conn.prepareStatement("SELECT text FROM TEST WHERE id = ?");

for(int i = 1; i <= rows; i++) {stmt.setInt(1, i);ResultSet rslt = stmt.executeQuery();rslt.next();rslt.getString(1);rslt.close();

}stmt.close();

Bindvariables


0

5000

10000

15000

20000

25000

30000


Milliseconds

Selectusing literals

Selectusingbindvar

• Threetestrunsselecting1krows

• DarkblueaxisshowselapsedtimeofSELECTusingliterals

• LightblueaxisshowselapsedtimeofSELECTusingbindvariable

61

Bindvariables


Bindvariables

• SQLstatementsarestrings• ButdatamanipulatedviaSQLmaybeofadifferenttypes(Number,Date,…)• Abindvariableallowsyoutobinddatatoanexplicitdatetype–Withouthavingtoconvertthedatatypefromstring– AndwithouthavingtoparsetheSQLstatementagain

• BindvariablesalsoavoidSQLinjections,i.e.tightersecurity

Whatarebindvariables

62


Tip#5:Fetchsize

63


Fetchsize

• Driverfetchesmorethanonerowfromthedatabaseatonce• Avoidsunnecessaryroundtrips• Greatifyouknowyouwillfetchmanyrows• Defaultissetto10• statement.setFetchSize(int rows);

Implicitandproactivefetching

64


Tip#6:Savepoints

65


INSERT INTO…UPDATE…INSERT INTO…

SAVEPOINT gerald1;

UPDATE…UPDATE…

SAVEPOINT gerald2;

DELETE FROM…

ROLLBACK TO SAVEPOINT gerald1;

INSERT INTO…COMMIT;

Savepoints


Savepoints

• Savetheintermediatestate(s)ofalongrunningtransaction• Ifworkloadfailsyoudon’thavetostartfromscratch• Avoidsredoingworkthathasalreadyhappened

Safetheprogressofyourtransaction

67


Tip#7:KISSKeepitsimpleSQL

68


KISS

• ThesimplertheSQL,theeasiertomaintain• ThesimplertheSQL,thefastertooptimize• ThesimplertheSQL,thelesslikelyfortheoptimizertogetitwrong

KeepitsimpleSQL

69


SELECT (CASEWHEN v.code = ‘ABC’ THEN 1WHEN v.code = ‘DEF’ THEN 2WHEN v.code = ‘GHI’ THEN 3WHEN v.code = ‘YYZ’ THEN 4ELSE 5

END) AS code,v.fk_column AS fk_column,v.fk_list AS fk_list,v.grp_id AS grp_id

FROM (SELECT S1.grp_id,S1.code,MIN (S1.fk_column) AS fk_column,LTRIM (MAX (SYS_CONNECT_BY_PATH (S1.fk_column, ‘;’))

KEEP (DENSE_RANK LAST ORDER BY S1.curr),‘;’) AS fk_list

FROM (SELECT grp_id,code,fk_column,ROW_NUMBER ()

OVER (PARTITION BY grp_id, codeORDER BY fk_column) AS curr,

ROW_NUMBER ()OVER (PARTITION BY grp_id, code

ORDER BY fk_column) – 1 AS prevFROM quite_a_big_table

WHERE grp_id > 0) S1GROUP BY S1.grp_id, S1.codeCONNECT BY S1.prev = PRIOR S1.curr

AND S1.grp_id = PRIOR S1.grp_idSTART WITH curr = 1) v

ORDER BY grp_id, code

KISS


SELECT (CASEWHEN v.code = ‘ABC’ THEN 1WHEN v.code = ‘DEF’ THEN 2WHEN v.code = ‘GHI’ THEN 3WHEN v.code = ‘YYZ’ THEN 4ELSE 5

END) AS code,v.fk_column AS fk_column,v.fk_list AS fk_list,v.grp_id AS grp_id

FROM (SELECT grp_id,code,MIN (fk_column) AS fk_column,LISTAGG(fk_column, ‘;’)

WITHIN GROUP (ORDER BY fk_column) AS fk_listFROM quite_a_big_table

WHERE grp_id > 0GROUP BY grp_id, code) v

ORDER BY grp_id, code

KISS


KISS- Commontableexpression

• SpecifiesasubqueryasacommontableinaSELECTstatement– Atableorsubquerythatisreadmorethanoncebythesamestatement

• CachesresultsofsubqueryinmemoryonDBserver• Avoidsreadingthesametablesmorethanonce,hencereducingI/O• BetterreadabilityforyourSQL

Ahandyandfasthelperforcommonsubqueries

72


WITH active_users (id, http_session, last_active) AS(

SELECT id, http_session_id, last_active_tmsFROM all_active_usersWHERE …

)SELECT a.id, a.http_session, p.first_name, p.last_name, p.email_addressFROM active_users a LEFT JOIN personal_accounts p ON p.email_address = a.idLEFT JOIN business_accounts b ON b.business_id = a.id;



WITH cte (n) AS(

SELECT 1 FROM dualUNION ALLSELECT n + 1 FROM cte WHERE n < 5

)SELECT * FROM cte;

N----------

12345



Tip#8:Parallelqueries

75


SELECT id, create_date, first_name, last_name, email_address, http_session_id,last_active, avatar, twitter_handle, github_profile, stars

FROM super_big_tableINNER JOIN another_super_big_table

ON (a.id = http_session_id)WHERE id IN (SELECT id FROM active_logons_past_24_hours)

AND stars > 25AND last_active > SYSDATE – 356;

SELECT /*+ PARALLEL */ id, create_date, first_name, last_name, email_address,http_session_id, last_active, avatar, twitter_handle, github_profile, stars

FROM super_big_tableINNER JOIN another_super_big_table

ON (a.id = http_session_id)WHERE id IN (SELECT id FROM active_logons_past_24_hours)

AND stars > 25AND last_active > SYSDATE – 356;

Parallelqueries


Parallelqueries

• Queriesbydefaultrunsinglethreaded–Oneprocess/thread– RunningononeCPU– DoingalltheI/Obyitself

• Longrunningand/orI/Ointensivequeriesmightbenefitfromparallelization• Easytoenable• BUT notauniversalperformancebooster!

Usethefullpowerofthedatabaseserver

77


Tip#9:Explainplans

78


EXPLAIN PLAN FOR<your SQL statement here>

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());

Explainplans


EXPLAIN PLAN FORWITH active_users (id, http_session, last_active) AS(

SELECT id, http_session_id, last_active_tmsFROM all_active_usersWHERE …

)SELECT a.id, a.http_session, p.first_name, p.last_name, p.email_addressFROM active_users a LEFT JOIN personal_accounts p ON p.email_address = a.idLEFT JOIN business_accounts b ON b.business_id = a.id;

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());

Explainplans


Plan hash value: 1707074151-------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 100M| 39G| 1169 (100)| 00:00:01 ||* 1 | HASH JOIN RIGHT OUTER | | 100M| 39G| 1169 (100)| 00:00:01 || 2 | INDEX FULL SCAN | BA_IDX01 | 1 | 13 | 0 (0)| 00:00:01 ||* 3 | HASH JOIN RIGHT OUTER| | 100M| 38G| 854 (100)| 00:00:01 || 4 | TABLE ACCESS FULL | PERSONAL_ACCOUNTS | 1 | 385 | 2 (0)| 00:00:01 || 5 | TABLE ACCESS FULL | ALL_ACTIVE_USERS | 100M| 2479M| 537 (100)| 00:00:01 |-------------------------------------------------------------------------------------------

Explainplans


Explainplans

• ExplainplansgiveyouthestepsthatareexecutedbytheDB• Itallowsyoutospotoutliersthatmightneedfurthertuning• Generatingexplainplansiseasy

LookbeyondyourSQLstatement

82


Tip#10:SQLwaitevents

83


SQL> SELECT sid, event, wait_classFROM v$session_waitWHERE wait_class != 'Idle';

SID EVENT WAIT_CLASS------- --------------------- ----------

265 direct path read User I/O269 enq: TM - contention Application274 direct path read User I/O291 direct path read User I/O

293 enq: TM - contention Application 294 enq: TM – contention Application 296 enq: TM – contention Application 302 enq: TM - contention Application

SQLwaitevents


SQLwaitevents

• EverySQLstatementcanbetracedonitsactivity• EverySQLstatementrecordsonwhatit’swaitingfor• Youcanusuallyjustquerythatinformation

LookingbeyondtheSQLstatement

85


Usefulresources

• PerformanceTuningGuide• SQLTuningGuide• DatabaseDevelopmentGuide• JDBCDeveloper’sGuide• UniversalConnectionPoolforJDBCDeveloper’sGuide

86


Whereisthesource?

github.com/gvenzl/Oracle-JavaOneSuperchargeCodeOptimalDBPerf

87

top 10 sql performance tips & tricks for java …...title top 10 sql performance tips &...

Documents