introduction to performance tuning perl web applications
DESCRIPTION
I gave this presentation at YAPC::NA 2012 in Madison, WI.TRANSCRIPT
![Page 1: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/1.jpg)
Introduction to Performance Tuning
Perl Web ApplicationsPerrin Harkins
![Page 2: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/2.jpg)
Find someone to blame
![Page 3: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/3.jpg)
Performance vs scalability
● Not really the same thing● Can look the same● We're mostly going to talk about
performance
![Page 4: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/4.jpg)
Slowness
● Maybe it's the designer's fault!● Chrome Developer Tools
![Page 5: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/5.jpg)
Network
![Page 6: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/6.jpg)
PageSpeed
![Page 7: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/7.jpg)
Narrow it down further
![Page 8: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/8.jpg)
Create a repeatable test
● Measure progress● Metrics
○ Requests/second○ Time for n requests○ Concurrency
![Page 9: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/9.jpg)
ab
ab -c1 -n100 http://127.0.0.1:8080/bugzilla/buglist.cgi
![Page 10: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/10.jpg)
Benchmarking 127.0.0.1 (be patient).....done
Server Software: Apache/2.2.22
Server Hostname: 127.0.0.1
Server Port: 8080
Document Path: /bugzilla/buglist.cgi
Document Length: 14562 bytes
Concurrency Level: 1
Time taken for tests: 33.816 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 1494100 bytes
HTML transferred: 1456200 bytes
Requests per second: 2.96 [#/sec] (mean)
Time per request: 338.163 [ms] (mean)
Time per request: 338.163 [ms] (mean, across all concurrent requests)
Transfer rate: 43.15 [Kbytes/sec] received
![Page 11: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/11.jpg)
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 313 338 20.7 333 448
Waiting: 312 337 20.6 332 448
Total: 313 338 20.7 333 448
Percentage of the requests served within a certain time (ms)
50% 333
66% 339
75% 343
80% 345
90% 357
95% 374
98% 430
99% 448
100% 448 (longest request)
![Page 12: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/12.jpg)
httperf
httperf --wsess=10,5,2 --rate=1 \
--server=localhost --port=8080 \ --uri=/bugzilla/buglist.cgi
![Page 13: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/13.jpg)
Total: connections 50 requests 90 replies 50 test-duration 21.680 s
Connection rate: 2.3 conn/s (433.6 ms/conn, <=11 concurrent connections)
Connection time [ms]: min 321.5 avg 2487.2 max 3884.7 median 2537.5 stddev 1002.1
Connection time [ms]: connect 0.1
Connection length [replies/conn]: 1.000
Request rate: 4.2 req/s (240.9 ms/req)
Request size [B]: 203.0
Reply rate [replies/s]: min 1.6 avg 2.4 max 3.0 stddev 0.6 (4 samples)
Reply time [ms]: response 883.0 transfer 3.4
Reply size [B]: header 388.0 content 14562.0 footer 2.0 (total 14952.0)
Reply status: 1xx=0 2xx=50 3xx=0 4xx=0 5xx=0
Session rate [sess/s]: min 0.00 avg 0.46 max 1.00 stddev 0.49 (10/10)
Session: avg 5.00 connections/session
Session lifetime [s]: 12.4
![Page 14: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/14.jpg)
HTTP::Recorder
● Acts as HTTP proxy● Generates Mechanize script● HTTP::Recorder::Httperf
![Page 15: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/15.jpg)
$agent->get('http://127.0.0.1:8080/bugzilla/');$agent->follow_link(text => 'Search', n => '1');$agent->form_name('queryform');$agent->field('bug_status', '__open__');$agent->field('content', '');$agent->field('product', 'TestProduct');$agent->click();
![Page 16: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/16.jpg)
/bugzilla/ method=GET
/bugzilla/skins/standard/global.css method=GET
/bugzilla/skins/standard/index.css method=GET
/bugzilla/skins/contrib/Dusk/global.css method=GET
/bugzilla/skins/contrib/Dusk/index.css method=GET
/bugzilla/js/yui/yahoo-dom-event/yahoo-dom-event.js method=GET
/bugzilla/js/yui/cookie/cookie-min.js method=GET
/bugzilla/js/global.js method=GET
/bugzilla/skins/standard/index/file-a-bug.png method=GET
/bugzilla/skins/standard/index/search.png method=GET
/bugzilla/skins/standard/index/new-account.png method=GET
/bugzilla/query.cgi method=GET think=4
/bugzilla/skins/standard/search_form.css method=GET
/bugzilla/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=TestProduct&content= method=GET think=6
/bugzilla/js/yui/assets/skins/sam/autocomplete.css method=GET
/bugzilla/js/yui/assets/skins/sam/calendar.css method=GET
/bugzilla/skins/standard/buglist.css method=GET
/bugzilla/skins/contrib/Dusk/buglist.css method=GET
![Page 17: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/17.jpg)
Autobench
![Page 18: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/18.jpg)
Profile to find out where the time is going
● Devel::NYTProf● Wall clock time vs. CPU time● Use your real environment● Multiple runs and warmup avoid
skewed results● Let's look at one...
![Page 19: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/19.jpg)
Ten bucks says it's your database
![Page 20: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/20.jpg)
Sure, every now and then you find
● A bad regex● A string being parsed over and over● Massive object instantiation● Network operations● Disk reads
![Page 21: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/21.jpg)
But mostly it's the database
● What’s slow in modern computers?● What does most of the I/O in a typical
web app?● Either fix your queries or cache them
![Page 22: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/22.jpg)
Profiling DBI
● DBI_PROFILE=2/DBI::ProfileDumper::Apache
● Let's look at one...
![Page 23: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/23.jpg)
Speeding up queries
● EXPLAIN plans○ pt-query-advisor
● SQL generation is not for everything● A little bit of database server tuning
○ pt-variable-advisor
![Page 24: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/24.jpg)
Speeding up DBI
● Cache connections and statements● Manage commits● Use native bulk loading tools
![Page 25: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/25.jpg)
The last resort: caching
● Cache management is a hard problem● Code complexity
○ invalidation calls○ dependency tracking
● Your content creators will hate it● When you do cache, use CHI
![Page 26: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/26.jpg)
A brief word about runtime environments
● Webserver choice has a minimal effect on performance
● Persistent daemon: mod_perl, Plack, FastCGI
● Buffer your output● Size-limiting or auto restarts
![Page 27: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/27.jpg)
Real-world pressures
![Page 28: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/28.jpg)
Flailing
● Changing things based on guesses rather than data
● No QA● Lots of collateral damage● Emergency profiling
![Page 29: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/29.jpg)
Benchmark::Stopwatch
my $stopwatch = Benchmark::Stopwatch->new->start;...$stopwatch->lap('load objects');...$stopwatch->lap('render template');...$stopwatch->lap('send response'); print $stopwatch->stop->summary;
![Page 30: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/30.jpg)
Benchmark::Stopwatch
NAME TIME CUMULATIVE PERCENTAGE load objects 1.000 1.000 14.289% render template 3.000 4.001 42.853% send response 2.000 6.001 28.572% _stop_ 1.000 7.001 14.287%
![Page 31: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/31.jpg)
Doubts about infrastructure
● “You're still using Foobar 1.5?! That’s so slow!”
● Keep your head● Get help: FAQ, Google, mailing list/IRC
![Page 32: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/32.jpg)
Buying hardware
● Good idea!● A boatload of RAM hides a multitude of
sins● Make sure you know what the
bottleneck is
![Page 33: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/33.jpg)
Further reading
● Tim Bunce's Advanced DBI slides on CPAN
● Percona Toolkit
![Page 34: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/34.jpg)
Thank you!
![Page 35: Introduction to performance tuning perl web applications](https://reader034.vdocument.in/reader034/viewer/2022050904/546dc590b4af9f8e2c8b55fa/html5/thumbnails/35.jpg)