devel::nytprof 2009-07 (outdated, see 201008)
DESCRIPTION
The slides of my "State-of-the-art Profiling with Devel::NYTProf" talk at OSCON in July 2009. I'll upload a screencast and give the link in a blog post at http://blog.timbunce.orgTRANSCRIPT
![Page 1: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/1.jpg)
Devel::NYTProfPerl Source Code Profiler
Tim Bunce - July 2009Screencast available at
http://blog.timbunce.org/tag/nytprof/
![Page 2: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/2.jpg)
Devel::DProf
• Oldest Perl Profiler —1995
• Design flaws make it practically uselesson modern systems
• Limited to 0.01 second resolutioneven for realtime measurements!
![Page 3: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/3.jpg)
Devel::DProf Is Broken$ perl -we 'print "sub s$_ { sqrt(42) for 1..100 }; s$_({});\n" for 1..1000' > x.pl
$ perl -d:DProf x.pl
$ dprofpp -rTotal Elapsed Time = 0.108 Seconds Real Time = 0.108 SecondsExclusive Times%Time ExclSec CumulS #Calls sec/call Csec/c Name 9.26 0.010 0.010 1 0.0100 0.0100 main::s76 9.26 0.010 0.010 1 0.0100 0.0100 main::s323 9.26 0.010 0.010 1 0.0100 0.0100 main::s626 9.26 0.010 0.010 1 0.0100 0.0100 main::s936 0.00 - -0.000 1 - - main::s77 0.00 - -0.000 1 - - main::s82
![Page 4: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/4.jpg)
Lots of Perl Profilers• Take your pick...
Devel::DProf | 1995 | Subroutine Devel::SmallProf | 1997 | Line Devel::AutoProfiler | 2002 | Subroutine Devel::Profiler | 2002 | Subroutine Devel::Profile | 2003 | Subroutine Devel::FastProf | 2005 | Line Devel::DProfLB | 2006 | Subroutine Devel::WxProf | 2008 | Subroutine Devel::Profit | 2008 | Line Devel::NYTProf | 2008 | Line & Subroutine
![Page 5: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/5.jpg)
Evolution
Devel::DProf | 1995 | Subroutine Devel::SmallProf | 1997 | Line Devel::AutoProfiler | 2002 | Subroutine Devel::Profiler | 2002 | Subroutine Devel::Profile | 2003 | Subroutine Devel::FastProf | 2005 | Line Devel::DProfLB | 2006 | Subroutine Devel::WxProf | 2008 | Subroutine Devel::Profit | 2008 | Line Devel::NYTProf v1 | 2008 | Line Devel::NYTProf v2 | 2008 | Line & Subroutine
...plus lots of innovations!
![Page 6: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/6.jpg)
CPU Time Real Time
Subroutines
Statements
? ?? ?
What To Measure?
![Page 7: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/7.jpg)
CPU Time vs Real Time
• CPU time- Very poor resolution (0.01s) on many systems
- Not (much) affected by load on system
- Doesn’t include time spent waiting for i/o etc.
• Real time- High resolution: microseconds or better
- Is affected by load on system
- Includes time spent waiting
![Page 8: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/8.jpg)
Sub vs Line
• Subroutine Profiling- Measures time between subroutine entry and exit
- That’s the Inclusive time. Exclusive by subtraction.
- Reasonably fast, reasonably small data files
• Problems- Can be confused by funky control flow
- No insight into where time spent within large subs
- Doesn’t measure code outside of a sub
![Page 9: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/9.jpg)
Sub vs Line
• Line/Statement profiling- Measure time from start of one statement to next
- Exclusive time (except includes built-ins & xsubs)
- Fine grained detail
• Problems- Very expensive in CPU & I/O
- Assigns too much time to some statements
- Too much detail for large subs (want time per sub)
- Hard to get overall subroutine times
![Page 10: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/10.jpg)
Devel::NYTProf
![Page 11: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/11.jpg)
v1 Innovations
• Fork of Devel::FastProf by Adam Kaplan- working at the New York Times
• HTML report borrowed from Devel::Cover
• More accurate: Discounts profiler overheadincluding cost of writing to the file
• Test suite!
![Page 12: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/12.jpg)
v2 Innovations
• Profiles time per block!- Statement times can be aggregated
to enclosing blockand enclosing sub
![Page 13: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/13.jpg)
v2 Innovations
• Dual Profilers!- Is a statement profiler
- and a subroutine profiler
- At the same time!
![Page 14: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/14.jpg)
v2 Innovations
• Subroutine profiler- tracks time per calling location
- even for xsubs
- calculates exclusive time on-the-fly
- discounts overhead of statement profiler
- immune from funky control flow
- in memory, writes to file at end
- extremely fast
![Page 15: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/15.jpg)
v2 Innovations
• Statement profiler gives correct timing after leave ops- unlike previous statement profilers...
- last statement in loops doesn’t accumulate time spent evaluating the condition
- last statement in subs doesn’t accumulate time spent in remainder of calling statement
![Page 16: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/16.jpg)
v2 Other Features
• Profiles compile-time activity
• Profiling can be enabled & disabled on the fly
• Handles forks with no overhead
• Correct timing for mod_perl
• Sub-microsecond resolution
• Multiple clocks, including high-res CPU time
• Can snapshot source code & evals into profile
• Built-in zip compression
![Page 17: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/17.jpg)
![Page 18: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/18.jpg)
Profiling Performance
Time SizePerl
SmallProfFastProfNYTProf+ blocks=0+ stmts=0DProf
x 1 -
x 22 -
x 6.3 42,927KB
x 3.9 11,174KB
x 3.5 9,628KB
x 2.5* 205KB
x 4.9 60,736KB
![Page 19: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/19.jpg)
v3 Features
• Profiles slow opcodes: system calls, regexps, ...
• Subroutine caller name noted, for call-graph
• Handles goto ⊂ e.g. AUTOLOAD
• HTML report includes interactive TreeMaps
• Outputs call-graph in Graphviz dot format
![Page 20: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/20.jpg)
Running NYTProf
perl -d:NYTProf ...
perl -MDevel::NYTProf ...
PERL5OPT=-d:NYTProf
NYTPROF=file=/tmp/nytprof.out:addpid=1:slowops=1
![Page 21: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/21.jpg)
Reporting NYTProf
• CSV - old, limited, dull
$ nytprofcsv
# Format: time,calls,time/call,code0,0,0,sub foo {0.000002,2,0.00001,print "in sub foo\n";0.000004,2,0.00002,bar();0,0,0,}0,0,0,
![Page 22: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/22.jpg)
Reporting NYTProf
• KcacheGrind call graph - new and cool- contributed by C. L. Kao.
- requires KcacheGrind
$ nytprofcg # generates nytprof.callgraph
$ kcachegrind # load the file via the gui
![Page 23: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/23.jpg)
![Page 24: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/24.jpg)
Reporting NYTProf• HTML report
- page per source file, annotated with times and links
- subroutine index table with sortable columns
- interactive Treemaps of subroutine times
- generates Graphviz dot file of call graph
$ nytprofhtml # writes HTML report in ./nytprof/...
$ nytprofhtml --file=/tmp/nytprof.out.793 --open
![Page 25: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/25.jpg)
![Page 26: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/26.jpg)
Summary
Links to annotatedsource code
Timings for perl builtins
Link to sortable tableof all subs
![Page 27: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/27.jpg)
Exclusive vs. Inclusive
• Exclusive Time = Bottom up- Detail of time spent “just here”
- Where the time actually gets spent
- Useful for localized (peephole) optimisation
• Inclusive Time = Top down- Overview of time spent “in and below”
- Useful to prioritize structural optimizations
![Page 28: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/28.jpg)
![Page 29: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/29.jpg)
Timings for each location calling into, or out of, the subroutine
Overall time spent in and below this sub
(in + below)
Color coding based onMedian Average Deviationrelative to rest of this file
![Page 30: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/30.jpg)
![Page 31: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/31.jpg)
Boxes represent subroutinesColors only used to show
packages (and aren’t pretty yet)
Hover over box to see detailsClick to drill-down one level
in package hierarchy
Treemap showing relative proportions of exclusive time
![Page 32: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/32.jpg)
Let’s take a look...
![Page 33: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/33.jpg)
OptimizingHints & Tips
![Page 34: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/34.jpg)
Phase 0 Before you start
![Page 35: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/35.jpg)
DONʼTDO IT!
![Page 36: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/36.jpg)
“The First Rule of Program Optimization: Don't do it.
The Second Rule of Program Optimization (for experts only!): Don't do it yet.”
- Michael A. Jackson
![Page 37: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/37.jpg)
Why not?
![Page 38: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/38.jpg)
“More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity.”
- W.A. Wulf
![Page 39: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/39.jpg)
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.Yet we should not pass up our opportunities in that critical 3%.”
- Donald Knuth
![Page 40: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/40.jpg)
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.Yet we should not pass up our opportunities in that critical 3%.”
- Donald Knuth
![Page 41: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/41.jpg)
How?
![Page 42: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/42.jpg)
“Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you have proven that's where the bottleneck is.”
- Rob Pike
![Page 43: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/43.jpg)
“Measure twice, cut once.”
- Old Proverb
![Page 44: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/44.jpg)
Phase 1Low Hanging Fruit
![Page 45: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/45.jpg)
Low Hanging Fruit1. Profile code running representative workload.
2. Look at Exclusive Time of subroutines.
3. Do they look reasonable?
4. Examine worst offenders.
5. Fix only simple local problems.
6. Profile again.
7. Fast enough? Then STOP!
8. Rinse and repeat once or twice, then move on.
![Page 46: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/46.jpg)
“Simple Local Fixes”
Changes unlikely to introduce bugs
![Page 47: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/47.jpg)
Move invariant expressionsout of loops
![Page 48: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/48.jpg)
Avoid->repeated->chains
->of->accessors(...)
Use a temporary variable
![Page 49: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/49.jpg)
Use faster accessors
Class::Accessor-> Class::Accessor::Fast--> Class::Accessor::Faster---> Class::XSAccessor
![Page 50: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/50.jpg)
Avoid calling subs that don’t do anything!
my $unsed_variable = $self->foo;
my $is_logging = $log->info(...);while (...) { $log->info(...) if $is_logging; ...}
![Page 51: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/51.jpg)
Exit subs and loops earlyDelay initializations
return if not ...a cheap test...;return if not ...a more expensive test...;my $foo = ...initializations...;...body of subroutine...
![Page 52: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/52.jpg)
Fix silly code
- return exists $nav_type{$country}{$key}- ? $nav_type{$country}{$key}- : undef;+ return $nav_type{$country}{$key};
![Page 53: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/53.jpg)
Beware pathological regular expressions
NYTPROF=slowops=2
![Page 54: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/54.jpg)
Avoid unpacking argsin very hot subs sub foo { shift->delegate(@_) }
sub bar { return shift->{bar} unless @_; return $_[0]->{bar} = $_[1]; }
![Page 55: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/55.jpg)
Retest.
Fast enough?
STOP!Put the profiler down and walk away
![Page 56: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/56.jpg)
Phase 2Deeper Changes
![Page 57: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/57.jpg)
Profile with aknown workload
E.g., 1000 identical requests
![Page 58: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/58.jpg)
Check Inclusive Times(especially top-level subs)
Reasonable percentagefor the workload?
![Page 59: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/59.jpg)
Check subroutinecall counts
Reasonablefor the workload?
![Page 60: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/60.jpg)
Add cachingif appropriate
to reduce calls
Remember invalidation
![Page 61: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/61.jpg)
Walk up call chainto find good spots
for caching
Remember invalidation
![Page 62: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/62.jpg)
Creating many objectsthat don’t get used?
Lightweight proxiese.g. DateTimeX::Lite
![Page 63: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/63.jpg)
Retest.
Fast enough?
STOP!Put the profiler down and walk away
![Page 64: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/64.jpg)
Phase 3Structural Changes
![Page 65: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/65.jpg)
Push loops down
- $object->walk($_) for @dogs;
+ $object->walk_these(\@dogs);
![Page 66: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/66.jpg)
Change the data structure
hashes <–> arrays
![Page 67: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/67.jpg)
Change the algorithm
What’s the “Big O”?O(n2) or O(logn) or ...
![Page 68: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/68.jpg)
Rewrite hot-spots in C
Inline::C
![Page 69: Devel::NYTProf 2009-07 (OUTDATED, see 201008)](https://reader034.vdocument.in/reader034/viewer/2022042714/554f84ebb4c905d25b8b4c05/html5/thumbnails/69.jpg)
It all adds up!
“I achieved my fast times by multitudes of 1% reductions”
- Bill Raymond