dynamic feedback: an effective technique for adaptive computing pedro diniz and martin rinard...
TRANSCRIPT
Dynamic Feedback:An Effective Techniquefor Adaptive Computing
Pedro Diniz and Martin Rinard
Department of Computer ScienceUniversity of California, Santa Barbara
http://www.cs.ucsb.edu/~{pedro,martin}
Basic Issue:Efficient Implementation of Atomic
Operations in Object-Based Languages
Approach:Reduce Lock Overhead by
Coarsening Lock Granularity
Problem:Coarsening Lock Granularity
May ReduceAvailable Concurrency
Solution: Dynamic Feedback
• Multiple Lock Coarsening Policies
• Dynamic Feedback• Generate Multiple Versions of Code• Measure Dynamic Overhead of Each Policy• Dynamically Select Best Version
• Context• Parallelizing Compiler
• Irregular Object-Based Programs• Pointer-Based Data Structures
• Commutativity Analysis
Talk Outline
• Lock Coarsening
• Dynamic Feedback
• Experimental Results
• Related Work
• Conclusions
Model of Computation
• Parallel Programs• Serial Phases• Parallel Phases
•Atomic Operations on Shared Objects•Mutual Exclusion Locks•Acquire Constructs•Release Constructs
AtomicOperations
SerialPhase
SerialPhase
ParallelPhase
L.acquire()
L.release()
Mutual ExclusionRegion
Problem: Lock Overhead
L.acquire()
L.release()
L.acquire()
L.release()
Solution: Lock Coarsening
Original After Lock Coarsening
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
Reference: Diniz and Rinard“Synchronization Transformations for Parallel Computing”, POPL97
Lock Coarsening Trade-Off
• Advantage: • Reduces Number of Executed Acquires and Releases• Reduces Acquire and Release Overhead
• Disadvantage: May Introduce False Exclusion• Multiple Processors Attempt to Acquire Same Lock• Processor Holding the Lock is Executing Code that
was Originally in No Mutual Exclusion Region
False Exclusion
Original After Lock Coarsening
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
•••
L.release()
FalseExclusion
Lock Coarsening Policy
Goal: Limit Potential Severity of False Exclusion
Mechanism: Multiple Lock Coarsening Policies
• Original: Never Coarsen Granularity• Bounded: Coarsen Granularity Only Within
Cycle-Free Subgraphs of ICFG
• Aggressive: Always Coarsen Granularity
Choosing Best Policy
• Best Lock Coarsening Policy May Depend On• Topology of Data Structures• Dynamic Schedule Of Computation
• Information Required to Choose Best Policy Unavailable at Compile Time
• Complications• Different Phases May Have Different Best Policy• In Same Phase, Best Policy May Change Over Time
Solution: Dynamic Feedback
• Generated Code Executes• Sampling Phases: Measure Performance of Different Policies• Production Phases : Use Best Policy From Sampling Phase
• Periodically Resample to Discover Best Policy Changes
AggressiveOriginal Bounded
Time
Ove
rhea
d
Sampling Phase Production Phase Sampling Phase
AggressiveCodeVersion Original
Guaranteed Performance Bounds
• Assumptions:• Overhead Changes Bounded by Exponential Decay
Functions
• Worst Case Scenario:• No Useful Work During Sampling Phase• Sampled Overheads Are Same For All Versions• Overhead of Selected Version Increases at Maximum Rate• Overhead of Other Versions Decreases at Maximum Rate
S PS S
Ove
rhea
d
Time
V0
Guaranteed Performance Bound
Definition 1. Policy p is at Most Worse Than Policy p over a Time Interval T if
Work = 0
T
(1 - oi(t))
dt
where
(1 - ) P + (1/) e(-P) Š (- 1) SN + (1/)
Result 1. To Guarantee this Bound
Work - Work Š T T
i
T
jT
i
ji
Definition 2. Dynamic Feedback is at Most Worse Than the Optimal if
Work - Work Š (P+SN) P+SN
opt
P
0 where Work = 1
P+SN
(1 - o1(t))
dt
P+SN
opt
Guaranteed Performance Bounds
(1 - ) P + (1/) e(-P)
(- 1) SN + (1/)
Production Interval P
Con
stra
int V
alue
sFeasibleRegion
Production Interval Too Long:May Execute Suboptimal
Policy for Long Time
Production Interval Too Short:
Unable to Amortize Sampling Overhead
Basic Constraint:Decay Rate () Must be Small Enough
Dynamic Feedback: Implementation
• Code Generation
• Measuring Policy Overhead
• Interval Selection
• Interval Expiration
• Policy Switch
Code Generation
• Statically Generate Different Code Versions for Each Policy• Alternative: Dynamic Code Generation
• Advantages of Static Code Generation:• Simplicity of Implementation• Fast Policy Switching
• Potential Drawback of Static Code Generation• Code Size (In Practice Not a Problem)
Measuring Policy Overhead
• Sources of Overhead• Locking Overhead• Waiting Overhead
• Compute Locking Overhead• Count Number of Executed Acquire/Release Constructs
• Estimate Waiting Overhead• Count Number of Spins on Locks Waiting to be
Released
Sampling TimeSampled Overhead =
Numberof Spins
Number ofAcquire/Release
xx Spin TimeAcquire/ReleaseExecution Time( )+( )
Interval Selection and Expiration
• Fixed Interval Values• Sampling Interval: 10 milliseconds• Production Interval: 10 seconds• Good Results for Wide Range of Interval
Values
• Polling Code for Expiration Detection• Location: Back Edges of Parallel Loop• Advantage: Low Overhead• Disadvantage: Potential Interaction with
Iteration Size
AtomicOperationsPolling
Points
Policy Switch
• Synchronous• Processors Poll Timer to Detect Interval Expiration• Barrier At End of Each Interval
• Advantages:• Consistent Transitions• Clean Overhead Measurements
• Disadvantages:• Need to Synchronize All Processors• Potential Idle Time At Barrier
Experimental Results
• Parallelizing Compiler Based on Commutativity Analysis [PLDI’96]
• Set of Complete Scientific Applications• Barnes-Hut N-Body Solver (1500 lines of C++)• Liquid Water Simulation Code (1850 lines of C++)• Seismic Modeling String Code (2050 lines of C++)
• Different Lock Coarsening Policies
• Dynamic Feedback
• Performance on Stanford DASH Multiprocessor
Code Sizes
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
Barnes-Hut
SerialOriginalDynamic
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
Water
Serial
OriginalDynamic
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
String
Serial
OriginalDynamic
Lock Overhead
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
Barnes-Hut(16K Particles)
Original
Bounded
Aggressive
Percentage of Time that the Single Processor Execution Spends Acquiring and Releasing
Mutual Exclusion Locks
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
Water(512 Molecules)
Original
BoundedAggressive
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
String(Big Well Model)
OriginalAggressive
Contention OverheadC
onte
ntio
n Pe
rcen
tage
Percentage of Time that Processors Spend Waiting to Acquire Locks Held by Other Processors
100
0
25
50
75
0 4 8 12 16Processors
0
25
50
75
100
0 4 8 12 16Processors
0
25
50
75
100
0 4 8 12 16Processors
OriginalBoundedAggressive
Barnes-Hut(16K Particles)
Water(512 Molecules)
String(Big Well Model)
Performance Results: Barnes-Hut
IdealAggressive
Dynamic FeedbackBounded
Original
Barnes-Hut on DASH(16K Particles)
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
Performance Results: Water
Ideal
Bounded
OriginalAggressive
Dynamic Feedback
Water on DASH(512 Molecules)
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
Performance Results: String
String on DASH(Big Well Model)
Ideal
Original
Aggressive
Dynamic Feedback
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
Summary
• Code Size Is Not An Issue
• Lock Coarsening Has Significant Performance Impact
• Best Lock Coarsening Policy Varies With Application
• Dynamic Feedback Delivers Code With Performance Comparable to The Best Static Lock Coarsening Policy
Related Work
• Adaptive Execution Techniques (Saavedra Park:PACT96)
• Dynamic Dispatch Optimizations (Hölzle Ungar:PLDI94)
• Dynamic Code Generation (Engler:PLDI96)
• Profiling (Brewer:PPoPP95)
• Synchronization Optimizations (Plevyak et al:POPL95)
Conclusions
• Dynamic Feedback• Generated Code Adapts to Different Execution
Environments
• Integration with Parallelizing Compiler• Irregular Object-Based Programs• Pointer-Based Linked Data Structures• Commutativity Analysis
• Evaluation with Three Complete Applications• Performance Comparable to Best Hand-Tuned
Optimization
BACKUP SLIDES
0
2
4
6
8
10
12
14
16
Spe
edup
0 2 4 6 8 10 12 14 16Number of Processors
Ideal
Aggressive
Bounded
Original
Barnes-Hut (16K Particles)
Performance Results : Barnes-Hut
Performance Results: Water
Ideal
Aggressive
Bounded
Original
0
2
4
6
8
10
12
14
16
0 2 4 6 8 10 12 14 16
Spe
edup
Number of Processors
Water (512 Molecules)
Performance Results: String
String (Big Well Model)
Spe
edup
Number of Processors
0
2
4
6
8
10
12
14
16
0 2 4 6 8 10 12 14 16
Ideal
Original
Aggressive
Policy Switch
TimerExpires
Policy 1
Policy 2TimerExpires
Motivation
Challenges:• Match Best Implementation to Environment• Heterogeneous and Mobile Systems
Goal: • Develop Mechanisms to Support Code that
Adapts to Environment Characteristics
Technique:• Dynamic Feedback
Overhead for Barnes-Hut
0
0.1
0.2
0.3
0.4
0.5
0 5 10 15 20 25
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Original
Aggressive
Bounded
Barnes-Hut on DASH (8 Processors)FORCES Loop
Data Set - 16K Particles
Overhead for Water
Water on DASH (8 Processors) INTERF Loop
Data Set - 512 Molecules
0
0.1
0.2
0.3
0.4
0.5
0 10 20 30 40 50 60
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Original
Bounded
Overhead for Water
Water on DASH (8 Processors)POTENG Loop
Data Set - 512 Molecules
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Aggressive
Original
Overhead for String
String on DASH (8 Processors)PROJFWD Loop
Data Set -Big Well
0
0.2
0.4
0.6
0.8
1
0 100 200 300 400 500
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Aggressive
Original
Dynamic Feedback
AggressiveOriginalBounded
Time
Ove
rhea
d
Sampling Phase Production Phase Sampling Phase
AggressiveCodeVersion