Result Reuse in Design Space Exploration: A Study in System Support

for Interactive Parallel Computing

Siu-Man Yau, (smyau@cs.nyu.edu), New York University Steven G. Parker (sparker@cs.utah.edu), University of Utah

Kostadin Damevski (damevski@cs.utah.edu), University of Utah Vijay Karamcheti (vijayk@cs.nyu.edu), New York University

Denis Zorin (dzorin@cs.nyu.edu), New York University

Multi-Experiment Studies

• Computational studies that require multiple runs of a simulation software. – Design Space Exploration

Interactive Parallel Computing

• Ubiquitous computing power– Multi-core chips– GPU w/general computing power

• Low-cost Supercomputing

• Opportunity for interactive parallel computing

• Requires new system paradigm

Outline

• Design Space Exploration (DSE): Defibrillator Design Application

• SimX System

• Reuse opportunities in DSE & System support for reuse

• Evaluation

• Conclusion

Defibrillator Simulation

Simulation (DefibSim):

• Utah Torso model

• Implemented on SCIRun Interactive Problem-Solving Environment

• Place electrodes, set voltage Torso potential

Defibrillator Simulation

Evaluation:

• Torso potential heart current

• Goal: – Maximize Activation

(% tissue above act.)– Minimize Damage

(% tissue above damage)

Defibrillator Study

• Pareto Frontier: set of inputs that cannot be improved in all objectives

Damage

Activa

tion

• Interactive Exploration of Pareto Frontier– Move the back electrode

– Change current threshold for activation

Design Space Exploration

SimX: Overview

• System support for Interactive Multi-Experiment Studies (SIMECS)

• View computational study as a whole

• For parallel, distributed clusters– Workers (Simulation code & Evaluation code)– Manager (UI, Sampler, Resource Allocator)– Spatially-Indexed Shared Object Layer

(SISOL)

• General DSE formulation: – Simulation code

Design Sim. result

– Evaluation code

Sim. result Perf. metric

– Design space & slice

– Performance space

SimX: Overview

XformModule

SimX: Overview

Front-end Manager Process

Worker Process Pool

User Interface: Visualisation &

Interaction

Sampler

ResourceAllocator

FU

EL

Inte

rfac

e

SISOL Server Pool

Data Server

Data Server

Data Server

Data Server

Dir

Ser

ver

TaskQueue

Simulationcode

FU

EL

Inte

rfac

eEvaluation

code

Reuse Opportunities

• Take application view

• Gather information from earlier runs

• Speed up or eliminate later runs

• Use domain knowledge

• 3 levels of information– Aggregate: Sampler, Cross-study– Individual: Simulation result, Performance

metric – Internal: Intermediate result, Preconditioner

Aggregate Result: Sampler

• Schedule runs from coarse to fine grid

• Use coarse level results to identify promising regions

• For Pareto Frontier discovery

1st Level 2nd Level 3rd Level

CustomSampler Xform

Module

Support for Sampler Reuse

Front-end Manager Process

Worker Process Pool

User Interface: Visualisation &

Interaction

Naïve (Sweep)Sampler

ResourceAllocator

FU

EL

Inte

rfac

e

SISOL Server Pool

Data Server

Data Server

Data Server

Data Server

Dir

Ser

ver

TaskQueue

Simulationcode

FU

EL

Inte

rfac

eEvaluation

code

RandomSampler

Active (Pareto) SamplerSampler

void setStudy(StudySpec)

void registerResult(experiment, performance)

experiment getNextPointToRun ()

SimX Sampler API

Individual Result: Simulation

• Increase activation current threshold

• Inputs to simulation code unchanged from previous study

• Simulation results may be reused

• Only Evaluation code needs running

XformModule

Support for Sim. Result Reuse

Front-end Manager Process Worker Process Pool

User Interface: Visualisation &

Interaction

Sampler

ResourceAllocator

FU

EL

Inte

rfac

e

SISOL Server Pool

Data Server

Data Server

Data Server

Data Server

Dir

Ser

ver

TaskQueue

Simulationcode

FU

EL

Inte

rfac

e

Evaluation code

SISOL API:

object StartRead(objSet, coord)

void EndRead(object)

object StartWrite(objSet, coord)

void EndWrite(objSet, object)

!!Spatially Indexed!!

Internal Reuse: Intermediate Result

• DefibSim solves 3 systems of equations

• Linearly combine solutions

• Same system needed by different experiments

• Cache the solutions Adx=bd

Aax=ba

Abx=bbAcx=bc

Store Ac-1bc and Ab

-1bb

XformModule

Intermediate Result Reuse

Front-end Manager Process Worker Process Pool

User Interface: Visualisation &

Interaction

Sampler

ResourceAllocator

FU

EL

Inte

rfac

e

SISOL Server Pool

Data Server

Data Server

Data Server

Data Server

Dir

Ser

ver

TaskQueue

SimulationCode

FU

EL

Inte

rfac

e

Evaluation code

Ab-1bb

Aa-1ba

SISOL API:

object StartRead(objSet, coord)

void EndRead(object)

object StartWrite(objSet, coord)

void EndWrite(objSet, object)

Evaluation

• 2 scenarios: – Increase activation current threshold– Move back electrode

• Measure response time

• Characterise reuse benefits

• 5 SISOL processes, 1 manager process, 1-128 worker processes

Increase Activation Threshold

65K sims @ 2 secs

1.6K sims @ 2 secs

1.2K sims @ 1.5 secs1.2K sims @ 0.3 secs

Move Back Electrode

65K sims @ 2 secs

7.3K sims @ 2 secs

6.5K sims @ 1.5 secs

Conclusion

• Result Reuse yields 8x-100x improvement in response time – Interactive rate: 12-120 secs

• Conclusions: – View from application level important

– Domain knowledge important

– System infrastructure to exploit domain knowledge important