![Page 1: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/1.jpg)
Dynamic Feedback:An Effective Techniquefor Adaptive Computing
Pedro Diniz and Martin Rinard
Department of Computer ScienceUniversity of California, Santa Barbara
http://www.cs.ucsb.edu/~{pedro,martin}
![Page 2: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/2.jpg)
Basic Issue:Efficient Implementation of Atomic
Operations in Object-Based Languages
Approach:Reduce Lock Overhead by
Coarsening Lock Granularity
Problem:Coarsening Lock Granularity
May ReduceAvailable Concurrency
![Page 3: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/3.jpg)
Solution: Dynamic Feedback
• Multiple Lock Coarsening Policies
• Dynamic Feedback• Generate Multiple Versions of Code• Measure Dynamic Overhead of Each Policy• Dynamically Select Best Version
• Context• Parallelizing Compiler
• Irregular Object-Based Programs• Pointer-Based Data Structures
• Commutativity Analysis
![Page 4: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/4.jpg)
Talk Outline
• Lock Coarsening
• Dynamic Feedback
• Experimental Results
• Related Work
• Conclusions
![Page 5: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/5.jpg)
Model of Computation
• Parallel Programs• Serial Phases• Parallel Phases
•Atomic Operations on Shared Objects•Mutual Exclusion Locks•Acquire Constructs•Release Constructs
AtomicOperations
SerialPhase
SerialPhase
ParallelPhase
L.acquire()
L.release()
Mutual ExclusionRegion
![Page 6: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/6.jpg)
Problem: Lock Overhead
L.acquire()
L.release()
L.acquire()
L.release()
![Page 7: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/7.jpg)
Solution: Lock Coarsening
Original After Lock Coarsening
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
Reference: Diniz and Rinard“Synchronization Transformations for Parallel Computing”, POPL97
![Page 8: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/8.jpg)
Lock Coarsening Trade-Off
• Advantage: • Reduces Number of Executed Acquires and Releases• Reduces Acquire and Release Overhead
• Disadvantage: May Introduce False Exclusion• Multiple Processors Attempt to Acquire Same Lock• Processor Holding the Lock is Executing Code that
was Originally in No Mutual Exclusion Region
![Page 9: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/9.jpg)
False Exclusion
Original After Lock Coarsening
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
L.release()
L.acquire()
•••
L.release()
FalseExclusion
![Page 10: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/10.jpg)
Lock Coarsening Policy
Goal: Limit Potential Severity of False Exclusion
Mechanism: Multiple Lock Coarsening Policies
• Original: Never Coarsen Granularity• Bounded: Coarsen Granularity Only Within
Cycle-Free Subgraphs of ICFG
• Aggressive: Always Coarsen Granularity
![Page 11: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/11.jpg)
Choosing Best Policy
• Best Lock Coarsening Policy May Depend On• Topology of Data Structures• Dynamic Schedule Of Computation
• Information Required to Choose Best Policy Unavailable at Compile Time
• Complications• Different Phases May Have Different Best Policy• In Same Phase, Best Policy May Change Over Time
![Page 12: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/12.jpg)
Solution: Dynamic Feedback
• Generated Code Executes• Sampling Phases: Measure Performance of Different Policies• Production Phases : Use Best Policy From Sampling Phase
• Periodically Resample to Discover Best Policy Changes
AggressiveOriginal Bounded
Time
Ove
rhea
d
Sampling Phase Production Phase Sampling Phase
AggressiveCodeVersion Original
![Page 13: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/13.jpg)
Guaranteed Performance Bounds
• Assumptions:• Overhead Changes Bounded by Exponential Decay
Functions
• Worst Case Scenario:• No Useful Work During Sampling Phase• Sampled Overheads Are Same For All Versions• Overhead of Selected Version Increases at Maximum Rate• Overhead of Other Versions Decreases at Maximum Rate
S PS S
Ove
rhea
d
Time
V0
![Page 14: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/14.jpg)
Guaranteed Performance Bound
Definition 1. Policy p is at Most Worse Than Policy p over a Time Interval T if
Work = 0
T
(1 - oi(t))
dt
where
(1 - ) P + (1/) e(-P) Š (- 1) SN + (1/)
Result 1. To Guarantee this Bound
Work - Work Š T T
i
T
jT
i
ji
Definition 2. Dynamic Feedback is at Most Worse Than the Optimal if
Work - Work Š (P+SN) P+SN
opt
P
0 where Work = 1
P+SN
(1 - o1(t))
dt
P+SN
opt
![Page 15: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/15.jpg)
Guaranteed Performance Bounds
(1 - ) P + (1/) e(-P)
(- 1) SN + (1/)
Production Interval P
Con
stra
int V
alue
sFeasibleRegion
Production Interval Too Long:May Execute Suboptimal
Policy for Long Time
Production Interval Too Short:
Unable to Amortize Sampling Overhead
Basic Constraint:Decay Rate () Must be Small Enough
![Page 16: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/16.jpg)
Dynamic Feedback: Implementation
• Code Generation
• Measuring Policy Overhead
• Interval Selection
• Interval Expiration
• Policy Switch
![Page 17: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/17.jpg)
Code Generation
• Statically Generate Different Code Versions for Each Policy• Alternative: Dynamic Code Generation
• Advantages of Static Code Generation:• Simplicity of Implementation• Fast Policy Switching
• Potential Drawback of Static Code Generation• Code Size (In Practice Not a Problem)
![Page 18: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/18.jpg)
Measuring Policy Overhead
• Sources of Overhead• Locking Overhead• Waiting Overhead
• Compute Locking Overhead• Count Number of Executed Acquire/Release Constructs
• Estimate Waiting Overhead• Count Number of Spins on Locks Waiting to be
Released
Sampling TimeSampled Overhead =
Numberof Spins
Number ofAcquire/Release
xx Spin TimeAcquire/ReleaseExecution Time( )+( )
![Page 19: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/19.jpg)
Interval Selection and Expiration
• Fixed Interval Values• Sampling Interval: 10 milliseconds• Production Interval: 10 seconds• Good Results for Wide Range of Interval
Values
• Polling Code for Expiration Detection• Location: Back Edges of Parallel Loop• Advantage: Low Overhead• Disadvantage: Potential Interaction with
Iteration Size
AtomicOperationsPolling
Points
![Page 20: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/20.jpg)
Policy Switch
• Synchronous• Processors Poll Timer to Detect Interval Expiration• Barrier At End of Each Interval
• Advantages:• Consistent Transitions• Clean Overhead Measurements
• Disadvantages:• Need to Synchronize All Processors• Potential Idle Time At Barrier
![Page 21: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/21.jpg)
Experimental Results
• Parallelizing Compiler Based on Commutativity Analysis [PLDI’96]
• Set of Complete Scientific Applications• Barnes-Hut N-Body Solver (1500 lines of C++)• Liquid Water Simulation Code (1850 lines of C++)• Seismic Modeling String Code (2050 lines of C++)
• Different Lock Coarsening Policies
• Dynamic Feedback
• Performance on Stanford DASH Multiprocessor
![Page 22: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/22.jpg)
Code Sizes
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
Barnes-Hut
SerialOriginalDynamic
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
Water
Serial
OriginalDynamic
0
20
40
60
Size
Tex
t Seg
men
t (K
byte
s)
String
Serial
OriginalDynamic
![Page 23: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/23.jpg)
Lock Overhead
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
Barnes-Hut(16K Particles)
Original
Bounded
Aggressive
Percentage of Time that the Single Processor Execution Spends Acquiring and Releasing
Mutual Exclusion Locks
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
Water(512 Molecules)
Original
BoundedAggressive
0
20
40
60
Perc
enta
ge L
ock
Ove
rhea
d
String(Big Well Model)
OriginalAggressive
![Page 24: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/24.jpg)
Contention OverheadC
onte
ntio
n Pe
rcen
tage
Percentage of Time that Processors Spend Waiting to Acquire Locks Held by Other Processors
100
0
25
50
75
0 4 8 12 16Processors
0
25
50
75
100
0 4 8 12 16Processors
0
25
50
75
100
0 4 8 12 16Processors
OriginalBoundedAggressive
Barnes-Hut(16K Particles)
Water(512 Molecules)
String(Big Well Model)
![Page 25: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/25.jpg)
Performance Results: Barnes-Hut
IdealAggressive
Dynamic FeedbackBounded
Original
Barnes-Hut on DASH(16K Particles)
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
![Page 26: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/26.jpg)
Performance Results: Water
Ideal
Bounded
OriginalAggressive
Dynamic Feedback
Water on DASH(512 Molecules)
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
![Page 27: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/27.jpg)
Performance Results: String
String on DASH(Big Well Model)
Ideal
Original
Aggressive
Dynamic Feedback
0
4
8
12
16
0 4 8 12 16
Number of Processors
Spe
edup
![Page 28: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/28.jpg)
Summary
• Code Size Is Not An Issue
• Lock Coarsening Has Significant Performance Impact
• Best Lock Coarsening Policy Varies With Application
• Dynamic Feedback Delivers Code With Performance Comparable to The Best Static Lock Coarsening Policy
![Page 29: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/29.jpg)
Related Work
• Adaptive Execution Techniques (Saavedra Park:PACT96)
• Dynamic Dispatch Optimizations (Hölzle Ungar:PLDI94)
• Dynamic Code Generation (Engler:PLDI96)
• Profiling (Brewer:PPoPP95)
• Synchronization Optimizations (Plevyak et al:POPL95)
![Page 30: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/30.jpg)
Conclusions
• Dynamic Feedback• Generated Code Adapts to Different Execution
Environments
• Integration with Parallelizing Compiler• Irregular Object-Based Programs• Pointer-Based Linked Data Structures• Commutativity Analysis
• Evaluation with Three Complete Applications• Performance Comparable to Best Hand-Tuned
Optimization
![Page 31: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/31.jpg)
BACKUP SLIDES
![Page 32: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/32.jpg)
0
2
4
6
8
10
12
14
16
Spe
edup
0 2 4 6 8 10 12 14 16Number of Processors
Ideal
Aggressive
Bounded
Original
Barnes-Hut (16K Particles)
Performance Results : Barnes-Hut
![Page 33: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/33.jpg)
Performance Results: Water
Ideal
Aggressive
Bounded
Original
0
2
4
6
8
10
12
14
16
0 2 4 6 8 10 12 14 16
Spe
edup
Number of Processors
Water (512 Molecules)
![Page 34: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/34.jpg)
Performance Results: String
String (Big Well Model)
Spe
edup
Number of Processors
0
2
4
6
8
10
12
14
16
0 2 4 6 8 10 12 14 16
Ideal
Original
Aggressive
![Page 35: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/35.jpg)
Policy Switch
TimerExpires
Policy 1
Policy 2TimerExpires
![Page 36: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/36.jpg)
Motivation
Challenges:• Match Best Implementation to Environment• Heterogeneous and Mobile Systems
Goal: • Develop Mechanisms to Support Code that
Adapts to Environment Characteristics
Technique:• Dynamic Feedback
![Page 37: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/37.jpg)
Overhead for Barnes-Hut
0
0.1
0.2
0.3
0.4
0.5
0 5 10 15 20 25
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Original
Aggressive
Bounded
Barnes-Hut on DASH (8 Processors)FORCES Loop
Data Set - 16K Particles
![Page 38: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/38.jpg)
Overhead for Water
Water on DASH (8 Processors) INTERF Loop
Data Set - 512 Molecules
0
0.1
0.2
0.3
0.4
0.5
0 10 20 30 40 50 60
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Original
Bounded
![Page 39: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/39.jpg)
Overhead for Water
Water on DASH (8 Processors)POTENG Loop
Data Set - 512 Molecules
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Aggressive
Original
![Page 40: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/40.jpg)
Overhead for String
String on DASH (8 Processors)PROJFWD Loop
Data Set -Big Well
0
0.2
0.4
0.6
0.8
1
0 100 200 300 400 500
Sam
pled
Ove
rhea
d
Execution Time (Seconds)
Aggressive
Original
![Page 41: Dynamic Feedback: An Effective Technique for Adaptive Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California,](https://reader030.vdocument.in/reader030/viewer/2022032723/56649cfa5503460f949cb559/html5/thumbnails/41.jpg)
Dynamic Feedback
AggressiveOriginalBounded
Time
Ove
rhea
d
Sampling Phase Production Phase Sampling Phase
AggressiveCodeVersion