divisible load scheduling a tutorial thomas robertazzi university at stony brook

Divisible Load Scheduling Divisible Load Scheduling

A TutorialA Tutorial

Thomas Robertazzi

University at Stony Brook

What is a Divisible Load?What is a Divisible Load?

A computational & networkable A computational & networkable load that is arbitrarily partitionable load that is arbitrarily partitionable (divisible) amongst processors and (divisible) amongst processors and links. links.

There are no precedence relations.There are no precedence relations.

Simple Application Simple Application ExampleExample

Problem: Sum 1000 trillion numbersProblem: Sum 1000 trillion numbers

Approach: Partition the numbers Approach: Partition the numbers among 100 processorsamong 100 processors

But how?But how?

Simple Application Simple Application ExampleExample

To optimize solution time (maximize To optimize solution time (maximize speedup) one needs to take into speedup) one needs to take into account heterogeneous link and account heterogeneous link and processor speeds, computation and processor speeds, computation and communication intensities, communication intensities, interconnection topology and interconnection topology and scheduling policy.scheduling policy.

Divisible Load Scheduling Theory Can Divisible Load Scheduling Theory Can Do This!Do This!

Applications (Generic)Applications (Generic)

Grid Computing/Meta-computingGrid Computing/Meta-computing Data Intensive ComputingData Intensive Computing Sensor Processing Sensor Processing Image ProcessingImage Processing Scientific/Engineering ComputingScientific/Engineering Computing Financial ComputingFinancial Computing

Applications (Specific)Applications (Specific)

Pattern SearchingPattern Searching Database ComputationDatabase Computation Matrix-Vector ComputationMatrix-Vector Computation E&M Field Calculation (CAD)E&M Field Calculation (CAD) Edge DetectionEdge Detection

DLT Modeling AdvantagesDLT Modeling Advantages

Linear and Deterministic Modeling Linear and Deterministic Modeling Tractable Recursive/Linear Tractable Recursive/Linear

Equation SolutionEquation Solution Schematic LanguageSchematic Language Equivalent ElementsEquivalent Elements Many ApplicationsMany Applications

Interconnection Interconnection TopologiesTopologies

Linear Daisy ChainLinear Daisy Chain BusBus Single Level and Multilevel TreesSingle Level and Multilevel Trees MeshMesh HypercubeHypercube

Directions: ScalabilityDirections: Scalability

1 32

1 11

Sequential Distribution(Saturation)

Simultaneous Distribution(Scalable)

Hung & RobertazziCheng & Robertazzi

An ExampleAn Example

Model Specifications:Model Specifications:• A star network( single level tree A star network( single level tree

network), and multi-level tree.network), and multi-level tree.• Computation and transmission time is Computation and transmission time is

a linear function of the size of load.a linear function of the size of load.• Level to Level: Store and Forward Level to Level: Store and Forward

SwitchingSwitching• Same Level: Concurrent Load Same Level: Concurrent Load

Distribution.Distribution.

Children without Front End: Children without Front End: After receiving the assigned data, each After receiving the assigned data, each

child proceeds to process the data.child proceeds to process the data.

Timing Diagram (single level tree) : Timing Diagram (single level tree) : Children without Front End Children without Front End

m+1 unknows vs. m+1 unknows vs. m+1 Eqs.m+1 Eqs.

Recursive equations:Recursive equations:

Normalization equation:Normalization equation:

miTwTzTwTz

TwTzTw

cpiicmiicpiicmii

cpcmcp

,...,3,2 1111

111100

1...210

m

Distribution Solution:Distribution Solution:

11

0

1101

2 21

1

11

12

1

Here

11

1

Here

,...,3,2

k

TzTwTwk

qk

TzTw

TzTwq

miq

cmcpcp

m

i

i

ll

cmicpi

cmicpii

i

lli

The load distribution solution is The load distribution solution is similar to the solution of the state-similar to the solution of the state-dependent M/M/1 queuing system.dependent M/M/1 queuing system.

Similarities to Queueing Similarities to Queueing TheoryTheory

Linear model and tractable solutionsLinear model and tractable solutions

Schematic LanguageSchematic Language

Equivalent ElelementsEquivalent Elelements

Infinite Size NetworksInfinite Size Networks

Speedup AnalysisSpeedup Analysis

m

i

i

ll

cp

cp

mf

f

qk

k

Tw

Tw

T

TSpeedup

2 21

10

00

0

,

0,

11

11

1

Speedup Analysis (continued)Speedup Analysis (continued)

11

10

01

1

1

12

1

11

Here

11

1 Here

,...,3,2

kmk

zTwTTwk

mk

zTwT

zTwTq

miq

cmcpcp

cmcp

cmcpi

i

lli

10

00

0

,

0,

11

1

km

Tw

Tw

T

T

Speedup

cp

cp

mf

f

Tree Network Tree Network (Children without Front Ends)(Children without Front Ends)

Collapsing single level Collapsing single level treestrees

Bandwidth of Fat TreeBandwidth of Fat Tree

Definition: The bandwidth of level j Definition: The bandwidth of level j in a fat tree can be defined as pin a fat tree can be defined as p j-1j-1z. z.

zmzpj

l

lj

11

01

Directions: Directions: Sequencing and InstallmentsSequencing and Installments

Daisy Chain SurpriseDaisy Chain Surprise

Efficiency RuleEfficiency Rule

Ghose, Mani & Bharadwaj


Multi-installment for Sequential Multi-installment for Sequential DistributionDistribution

12

3

45

6

Ghose, Mani & Bharadwaj


Diminishing returns in using multi-installment distribution.Ghose, Mani & Bharadwaj


Drozdowski

Directions: Time Varying Directions: Time Varying ModelingModeling

Can be solved with integral calculus. Sohn & Robertazzi

Directions: Monetary Cost Directions: Monetary Cost OptimizationOptimization

MinMin C CTotal Total nnccnnwwnnTTcpcpn=1

N

Bus

Processors

Optimal Sequential Distribution if:

cn-1wn-1 less than cnwn for all n

Sohn, Luryi & Robertazzi

Directions: Monetary Cost Directions: Monetary Cost OptimizationOptimization

2 US Patents:2 US Patents:

Patent 5,889,989 (1999): Processor CostPatent 5,889,989 (1999): Processor Cost

Patent 6,370,560 (2001): Processor and Patent 6,370,560 (2001): Processor and

Link CostLink Cost

Enabling technology for an open e-Enabling technology for an open e-commerce commerce

market in leased proprietary computing. market in leased proprietary computing. Sohn, Charcranoon, Luryi & Robertazzi

Directions: Database Directions: Database ModelingModeling

Expected time to find multiple records in flat file database

Ko & Robertazzi

Directions: RealismDirections: Realism

Finite Buffers (Bharadwaj)Finite Buffers (Bharadwaj)

Job Granularity (Bharadwaj)Job Granularity (Bharadwaj)

Queueing Model IntegrationQueueing Model Integration

Directions: Experimental Directions: Experimental WorkWork

Database Join (Drozdowski)

Directions: Future Directions: Future ResearchResearch

Operating Systems:Operating Systems:

Incorporate divisible load scheduling Incorporate divisible load scheduling

into (distributed) operating systemsinto (distributed) operating systems

Measurement Process Modeling:Measurement Process Modeling:

Integrate measurement process Integrate measurement process

modeling into divisible scheduling modeling into divisible scheduling


Pipelining (Dutot)Pipelining (Dutot)

Concept: Distribute load to Concept: Distribute load to

further processors first further processors first for for

speedup improvementspeedup improvement

Improvement reported for daisy chainsImprovement reported for daisy chains


System Parameter Estimation (Ghose):System Parameter Estimation (Ghose):

Concept: Send small “probing” loads across Concept: Send small “probing” loads across

links and to processors to links and to processors to estimate estimate

available effortavailable effort

Challenge: Rapid change in link & processor Challenge: Rapid change in link & processor

statestate

DLT has a Good FutureDLT has a Good Future

Many Applications including Many Applications including

wireless sensor wireless sensor networksnetworks

Tractable (Modeling & Computation)Tractable (Modeling & Computation)

Rich Theoretical BasisRich Theoretical Basis

Thank you!Thank you!

Questions???Questions???

divisible load scheduling a tutorial thomas robertazzi university at stony brook

Documents