thinking through enterprise performance - javaone 2012
DESCRIPTION
Taking the high road with Enterprise Java Performance - for a helicopter view Performance of Enterprise Java Applications is a requirement and usually a challenge. Business requirements on systems can be stiff, successful systems can easily be overloaded and complex application architectures can add a burden too. Improving performance by tuning the application after it has been built seldomly renders huge improvements. By taking a step back - or even two - and regarding the application and the performance from a distance, it becomes possible to really design and architect for performance according to the ISYITF-method: it is staring you in the face. Order of magnitude improvements are attainable through logical reasoning and careful application of multi-tier architecture principles and JEE platform facilities.TRANSCRIPT
THINKING THROUGH JAVA ENTERPRISE PERFORMANCE
Lucas Jellema (AMIS, The Netherlands)
JavaOne 2012, San Francisco
OVERVIEW
• What is performance?• Where is performance established?• Advanced tuning methods• ISYITF Method for Performance Improvement:
– Do not do it …• Architecting Enterprise Java applications for
improved performance
PERFORMANCE DEGRADATION
65 %
PERFORMANCE DEGRADATION
Response time
+ 65 %
PERFORMANCE
Measure objectively
Business Objectives
SLAs
What is start and what is end of action
Response timeMeaningful response
Disappearance Hourglass
Expectations
Process Duration
Business Owner
Wait time
Availability ≈ Performance
Who determines in what way the performance
THE THEATER MANAGER
TYPICAL LAYOUT OF ENTERPRISE JAVA APPLICATIONS
Web Browser
RDBMS
Performance ≈ Wait for Response
JEE Application Server
PERFORMANCE CONTRIBUTORS IN ENTERPRISE JAVA APPLICATIONS
Web Browser
RDBMS
Performance ≈ Wait for Response
Response = Wait + ProcessingWait = Network 1 + Response
AppServer
Response = Wait + ProcessingWait = Network 2 + Response
Database
Response = ProcessingProcessing = internal wait (I/O) + CPU
1
2
JEE Application Server
ADVANCED TUNING METHODS
• Use StringBuffer rather than plain String concatenation• Use SAX for XML parsing instead of DOM• Carefully select the collection class to use
– optimize hashcode() in custom Map implementations• Use profiling tools to identify hotspots in the Java code• Remove Assertions from production code• Find optimal JVM switches through trial-and-error
– Focus on GC, Heap size, thread pools• Pool resources and reuse objects rather than recreate• Leverage concurrency features in Java to
– speed up time-to-completion through parallel execution
– prevent underuse of CPU during I/O operations• Optimize algorithms for sorting, pattern matching,
iteration, serialization, …
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
The fastest way to perform a task:
DO NOT DO IT
PREVENT UNNEEDED PROCESSING
if ( expensiveEvaluation & someFlag) {
...
}
if ( someFlag && expensiveEvaluation) {
...
}
PREVENT UNNEEDED PROCESSING
log.debug ( “Outcome step 2: ” +
resultOfExpensiveProcessing );
if (log.doDebug)
log.debug ( “Outcome step 2: ” +
resultOfExpensiveProcessing );
THE SHOPPING ALGORITHM
THE SHOPPING ALGORITHM
• shopForItem Item ( String itemName) {
driveToShop;
Item item = buyItemAtShop ( itemName);
driveHomeFromShop;
return item;
}
GET THIS WEEK’S GROCERIES
getGroceries Item[] ( String[] shoppingList) {
Item[] items = new Item[ shoppingList.length];
for (int i=0; i < shoppingList.length; i++) {
items[i] = shopForItem (shoppingList[i]);
}
return items;
}
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
AT ALL
DO NOT DO ITMORE OFTEN THAN REQUIRED
AT ALL
SWEET MEMORIES
STOCK MANAGEMENT
STOCK MANAGEMENT
DO NOT DO IT…MORE OFTEN THAN REQUIRED
• If it has been produced before…– Reuse before re-produce!
• If it has been shipped before…– Reuse instead of re-ship
• … provided it is still freshWeb Browser
RDBMS
JEE Application Server
DO NOT DO IT…MORE OFTEN THAN REQUIRED
• Save on network trips, context switches and tiers to cross
• Save on ‘reproducing’ same results
Web Browser
RDBMS
JEE Application Server
- JS data (memory)- Cookies
- HTML 5 db
Edge CacheCache
Cluster Fail-Over(Session State)Result StoreWrite Behind Client Result
Cache
Result Cache
Materialized View
MORE PERFORMANCE REQUIRES PARALLEL
MORE PERFORMANCE REQUIRES PARALLEL
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
MOORE’S LAW REVISED: CORES LAW
DO NOT DO IT…ON YOUR OWN
• Parallel means: multiple resources contributing to a task at the same time
• Leverage multi-core CPU• Achieve scalability and performance with a Cluster• Introduce parallellism into your application
– Java: Concurrency (ThreadPools), WorkManager– Database: parallel query and DML ,
dbms_parallel_execute, dbms_job, parallel table functions
– Multi threading: on every tier and even across tiers• Engage compute grid: multi node processing unit
– For example: make Hadoop make those nodes work for you (divvy up the work and merge the results)
BALANCE RESOURCES TO PREVENT CLOGGING
HILTON UNION SQUARE – LAST FRIDAY
THE POORLY PERFORMING BUSINESS PROCESS LOAN REQUEST
Check Identity with Federal
ServiceFraud Analysis Evaluate
RequestLoan
request
Reject request
Transfer Money
Max 1 day Max 2 days Max 3 days Max 3 mins
98% is OK 99.99% is OK
Max 6 days + 3 mins
THAT DINNER IS TAKING MIGHTY LONG…
Dinner is Served
Max 3 hours + 3 mins
All are satisfied
THE DINNER PROCESS
John eats soup
Mary eats soup
Daisy eats soup
Dinner is Served
Max 3 hours + 3 mins
Marty eats soup
John eats main
Mary eats main
Daisy eats main
Marty eats main
John eats desert
Daisy eats
desert
All are satisfied
THE SAME DINNER PROCESS
John eats soup
Mary eats soup
Daisy eats soup
Dinner is Served
Max 3 hours + 3 mins
Marty eats soup
John eats main
Mary eats main
Daisy eats main
Marty eats main
John eats desert
Daisy eats
desert
All are satisfied
THE OPTIMIZED DINNER PROCESS
Dinner is Served
Max 1 hours + 3 mins
All are satisfied
John eats desert
Daisy eats
desert
John eats main
Mary eats main
Daisy eats main
Marty eats main
John eats soup
Mary eats soup
Daisy eats soup
Marty eats soup
FURTHER OPTIMIZATIONS(IF NOT REFINEMENT)
Dinner is Served
Max 53 mins
All are satisfied
John eats desert
Daisy eats
desert
John eats main
Mary eats main
Daisy eats main
Marty eats main
John eats soup
Mary eats soup
Daisy eats soup
Marty eats soup
THE POORLY PERFORMING BUSINESS PROCESS LOAN REQUEST
Check Identity with Federal
ServiceFraud Analysis Evaluate
RequestLoan
request
Reject request
Transfer Money
Max 1 day Max 2 days Max 3 days Max 3 mins
98% is OK 99.99% is OK
Max 6 days + 3 mins
SPEEDING UP THE BUSINESS PROCESS LOAN REQUEST
Check Identity with Federal
Service
Fraud Analysis
Evaluate Request
Loanrequest
Reject request
Transfer Money
Max 1 day
Max 2 days
Max 3 days
Max 3 mins
Max 3 days + 3 mins
PANCAKE PARTY
BETTER PERFORMING PANCAKE PARTY
PIPELINED PANCAKE PARTY: BEST PERFORMANCE
PIPELINING ACROSS THE TIERS
• Database: – Pipelined Table Functions– Pipes and Queues
• Middle tier:– Multiple threads– Queues (JMS)
• Client tier:– AJAX “channels”– WebSockets
Web Browser
RDBMS
THE PINNACLE OF UN-PERFORMANCE
FIRE AND FORGET
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELY
FIRE AND FORGET IN THE REAL WORLD
DO NOT DO IT…IMMEDIATELY (OR SYNCHRONOUSLY)
• Submit information, file a complaint or request, start a process, trigger interaction– No immediate response is required!
• Asynchronous– Start batch job (db) or worker-thread (java)
• Or fire event
– Write behind (from grid) (NO SQL)– DML Error log
DO NOT DO IT…IN BATCH (UN-IMMEDIATELY)
• Batch jobs can put peak load on a system – choking on line applications– Monthly reporting, quarterly prolongation, yearly
calculation, • Batch jobs are increasingly unwanted in 24/7
– When is the “nightly” batch window?– Data not current (enough) by today’s standards:
“batch has not run yet”• Batch jobs used to be desirable or needed as
a result of technical limitations – that may not apply anymore
• Continuous, near real-time operations – leveraging events, decoupling and integration architectures – are a serious alternative
DON’T CALL US … WE’LL CALL YOU
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELYAS PER REQUEST
DO NOT DO IT…AS PER REQUEST
• Push has two key advantages over poll– Most up to date information– Reduction in network traffic and load on server
• Push is available in several flavors– Middleware to Browser: comet, ADF Active Data
Service, WebLogic Server HTTP Channels, long poll, WebSockets in HTML 5
– Database to Middleware: JMS/AQ, Database Query Result Change Notification, Table Triggers, utl_http push to servlet
– “piggy backing” – adding subscribed-to information in regular requests
• Event driven architecture is based on push (of events) to mediating event handling infrastructure
BITE OFF MORE THAN YOU CAN HAVE TO CHEW
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELYAS PER REQUEST
IN TOO BIG OR TOO SMALL STEPS
DO NOT DO IT…IN TOO BIG STEPS
• Performance perception is often: time until page is displayed and accessible (hourglass disappears)
• Web pages frequently contain much more than is initially visible or even required– Tree nodes, inactive tabs, invisible popups,
unopened dropdown lists • Performance perception can be enhanced by
not initially loading what is not required• Use AJAX based post-loading to (lazy-)fetch
content in subsequent, background round-trips
• /*+ First Rows */ vs. /*+ All Rows */
PENSION FUND – SEPTEMBER 2012
Employer
Participants
Job & Benefits
><
FETCHING THE DATA OF THE PENSION FUND FOR THE WEB APPLICATION
>< select * from employers where id = < 324>
select * from participants where employer_id = < 324>
select * from benefits where participant_id = <#>
1 record
100s records
10s records
REPORTING ON MANY EMPLOYERS
select * from employers
select * from participants where employer_id = <#>
select * from benefits where participant_id = <#>
10k records
100k records
100s records1 query
100s queries
10k queries
SINGLE BULK RETRIEVE REPLACING MULTIPLE QUERIES• Have the database bulk up the data retrieval• Return Ref Cursor, Types and Collections or
JSON/XML
select * from employerswhere id in <some set> select *
from participants where employer_id in <some set>
select b.* from benefits b join participants p on (p.id = b.participant_id)where p.employer_id in <some set>
Benefits Package
DO NOT DO IT…IN TOO BIG OR TOO SMALL STEPS
• Every network round-trip and context-switch adds overhead– Compare dialing the operator for every digit in the
telephone number you want to learn about• Bundling up information to reduce the number
of round trips can be advantageous for performance– Bring all items from the shop in one round trip– Leverage collections and types, XML or JSON to
retrieve complex, structured object graphs from DB
– Zipping up multiple web resources in single archive
– Mashing up icons or images into a single big picture
– Piggy-back information onto requests
THE HARD WAY
A CONVOLUTED WAY
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELYAS PER REQUEST
IN TOO BIG OR TOO SMALL STEPSIN A CONVOLUTED WAY
DO NOT DO IT…IN A CONVOLUTED WAY
• Pragmatism can overcome theoretical purity (or old dogs’ tricks)– With good reason and well documented
• Have client side Java Script directly access Google Maps – by-passing the application server
• Have client directly access Database services• Use RESTful (JSON, CSV) rather than WS* and XML
between browser client and application server• Use POJOs (Entities) throughout the application,
from JPA to Web Tier – rather than copying/transferring
• When that suffices, use simple substring i/o parsing big xml in DOM
• Pass plain CSV/JSON/XML from DB through Java middle tier to Client when that is appropriate
BOTTLENECK / CRITICAL CHAIN
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELYAS PER REQUEST
IN TOO BIG OR TOO SMALL STEPSIN A CONVOLUTED WAY
IN A SUBOPTIMAL PLACE
BOTTLENECK / CRITICAL CHAIN
• Make sure that the bottleneck resource in your enterprise application is not used (at peak times) for work that is not critical or that can be outsourced– Use auxiliary resources – outside critical chain– Engage specialized servers, optimized for
specific tasks– Manage resources in a strict manner
• Resource manager (DB) or Work manager (WLS)
DO NOT DO IT… LIVE EVENT PROCESSINGIN A SUBOPTIMAL PLACE
• The league of real time events– Continuous stream of a multitude of tiny
events with hardly any payload, to analyze & aggregate
– Sent from physical sensors (temperature, pressure, RFID, security gates), process sensors, Twitter, manufacturing equipment, database triggers, web servers, ESBs, stock trade tickers, sport statistics, RSS, network switches, …
DO NOT DO IT… HTML RENDERINGIN A SUBOPTIMAL PLACE
• (X)HTML is not very compact– Information density of HTML is very low
• DHTML, JavaScript &AJAX allow for– Dynamic HTML
rendering in browser– Dynamic, Partial
Page Refresh• Most HTML presented
by application is pre-defined– Dynamic data content
fetched from RDBMSor other services issmall fraction
Web Browser
RDBMS
JEE Application Server
DO NOT DO IT…IN A SUBOPTIMAL PLACE
• Do not perform a task in a resource that is not ideally suited for that task– If it directly contributes to overall performance
DO NOT DO IT…IN A SUBOPTIMAL PLACE
• Leverage database for what it’s good at– Data Integrity – Primary Key /Unique Key /Foreign Key– Aggregation– Sorting– Data Rule enforcement– Bulk DML and Copy data– Analytical Functions, Model clause, Rollup
• Specialized engines for– Imaging and Document Processing– Match and Search– Speech Recognition– Cryptography– 3D – Real time Data Processing– ….
ISYITF METHOD FOR PERFORMANCE IMPROVEMENT
DO NOT DO IT …AT ALL
MORE OFTEN THAN REQUIREDON YOUR OWN
IMMEDIATELYAS PER REQUEST
IN TOO BIG OR TOO SMALL STEPSIN A CONVOLUTED WAY
IN A SUBOPTIMAL PLACE
ARCHITECT FOR PERFORMANCE
Web Browser
RDBMS
JEE Application Server
ARCHITECT(URE) FOR PERFORMANCE
Web Browser
Services ( Google Maps, Translation, Conversion,
Data Feeds
- JS data (memory)- Cookies
- HTML 5 local db
Fire&Forget
Post load(AJAX)
HTML renderingValidation, Calculation, Parsing“Processing” (image, cryption, compression, SETI)
piggy-back
by-pass
JEE Application Server
ARCHITECT(URE) FOR PERFORMANCE
Web Browser
Services ( Google Maps, Translation, Conversion,
Data Feeds
- JS data (memory)- Cookies
- HTML 5 local db
push
Fire&Forget
Post load(AJAX)
HTML renderingValidation, Calculation, Parsing“Processing” (image, cryption, compression, SETI)
piggy-back
by-pass
Edge Cache
Cache
Load balancerSticky ip sessions, Throttling
JEE App
ServerNode
JEE App
ServerNode Node
WorkManagerParallel ThreadsJMS
Cluster Fail-Over(Session State)Result StoreWrite Behind
CEP
CMSCompute
GridCrypto
ImagePrint
Server
Search&
Match
ARCHITECT(URE) FOR PERFORMANCE
RDBMS
Client Result Cache
Edge Cache
Cache
Load balancerSticky ip sessions, Throttling
JEE App
ServerNode
JEE App
ServerNode Node
WorkManagerParallel ThreadsJMS
Cluster Fail-Over(Session State)Result StoreWrite Behind
CEP
CMSCompute
GridCrypto
ImagePrint
Serverpush
Postload
REST API
by-pass
Fire&Forget
Result Cache
Materialized View
AQ/JMSHTTP PushDB QRCN
Resource MgtJobs
PipeliningParallel Processing
AggregationFilter & SortData IntegrityBulk DML
CBO
CEP
CMSCompute
GridCrypto
ImagePrint
Server
Search&
Match
Web Browser
Services ( Google Maps, Translation, Conversion,
Data Feeds
- JS data (memory)
- Cookies- HTML 5 local db
push
Fire&Forget
Post load(AJAX)
HTML renderingValidation, Calculation, Parsing“Processing” (image, cryption, compression, SETI)
piggy-back
by-passEdge
Cache
Cache
Load balancerSticky ip sessions, Throttling
JEE AppServerNo
de
JEE AppServerNo
deNode
WorkManagerParallel ThreadsJMS
Cluster Fail-Over(Session State)Result StoreWrite Behind
CEPCMS
Compute
GridCrypt
oImage
PrintServer
RDBMS
Client Result Cache
push
Postload
REST API
Fire&Forget
Result Cache
Materialized View
AQ/JMSHTTP PushDB QRCN
Resource MgtJobs
PipeliningParallel Processing
AggregationFilter & SortData IntegrityBulk DML
CBO
SUMMARY
• Performance requirements are derived from measurable and meaningful business objectives
• Unavailability equals Zero Performance– Treat Performance and Availability elements in
the same equation• Performance should [also] be addressed in a
top-down approach, across all tiers and constituent parts
• Some ISYITF guidelines:– Do not do it … [AT ALL | MORE OFTEN THAN
REQUIRED | ON YOUR OWN | IMMEDIATELY | AS PER REQUEST | IN TOO BIG OR TOO SMALL STEPS | IN A CONVOLUTED WAY | IN A SUBOPTIMAL PLACE ]