Computational SteeringComputational Steering

Sathish VadhiyarSathish Vadhiyar

Sources/Credits: CUMULVS web Sources/Credits: CUMULVS web sites and paperssites and papers

IntroductionIntroduction

Parameters of the computation are adjusted on-Parameters of the computation are adjusted on-the-fly to gain understanding of the problem the-fly to gain understanding of the problem

Respond to simulation results as they occur by Respond to simulation results as they occur by manipulating the input parametersmanipulating the input parameters

For carrying out “what if” analysisFor carrying out “what if” analysis

For experimenting with program characteristics For experimenting with program characteristics whose effects are not easily understoodwhose effects are not easily understood

Allows interactive exploration of simulation in Allows interactive exploration of simulation in time and/or spacetime and/or space

Motivation: Falcon Molecular Motivation: Falcon Molecular Dynamics (MD)Dynamics (MD)

Simulation of system containing 4800 particles representing an Simulation of system containing 4800 particles representing an alkane film layered on 2700 particles representing a crystalline basealkane film layered on 2700 particles representing a crystalline baseSimulation stepsSimulation steps

Obtain location information from neighboring particlesObtain location information from neighboring particles Calculate intra-molecular forcesCalculate intra-molecular forces Calculate inter-molecular forcesCalculate inter-molecular forces Apply forces to yield new particle positionApply forces to yield new particle position Publish particle’s new positionPublish particle’s new position

Other stepsOther steps Determining system-wide characteristics such as atomic temperatureDetermining system-wide characteristics such as atomic temperature Performing on-line data analysis and visualizationPerforming on-line data analysis and visualization

Simulation algorithmSimulation algorithm Domain decompositionDomain decomposition Domains assigned to parallel processorsDomains assigned to parallel processors Cut-off radius in force calculationCut-off radius in force calculation

Steering opportunities in MDSteering opportunities in MD

Changing decomposition geometriesChanging decomposition geometries

Changing cut-off radius – speed/fidelity Changing cut-off radius – speed/fidelity tradeoffstradeoffs

Changing frequency of global temperature Changing frequency of global temperature calculations based on temperature stability calculations based on temperature stability of the systemof the system

Advantages of Steering in MDAdvantages of Steering in MD

Advantages of Steering in MDAdvantages of Steering in MD

CUMULVSCUMULVS

Collaborative User Migration, User Library for Collaborative User Migration, User Library for Visualization and SteeringVisualization and SteeringProvides 2 librariesProvides 2 libraries For application programFor application program For visualization and steering front endFor visualization and steering front end

Can dynamically attach multiple viewer front Can dynamically attach multiple viewer front ends to a running parallel applicationends to a running parallel applicationViewer programs can also steer one or more Viewer programs can also steer one or more user-defined parametersuser-defined parametersAlso provides fault tolerance and migration Also provides fault tolerance and migration across heterogeneous systemsacross heterogeneous systems

CUMULVS – Application and CUMULVS – Application and Visualization SpecificationVisualization Specification

Programmer specifiesProgrammer specifies Data distributionsData distributions Which parameters are allowed to be steeredWhich parameters are allowed to be steered

To reduce network traffic, front-end To reduce network traffic, front-end viewers can specify a region for viewers can specify a region for visualization including the granularity of visualization including the granularity of the desired datathe desired data

CUMULVS – Visualization and CUMULVS – Visualization and Steering – Coordination and Steering – Coordination and

SynchronizationSynchronizationConsistent view of the application’s state Consistent view of the application’s state by the viewer through loose by the viewer through loose synchronization with the applicationsynchronization with the application A viewer brackets the timesteps and ensures A viewer brackets the timesteps and ensures

that all tasks are on one of the timestepsthat all tasks are on one of the timesteps Tasks A, B, and C can be at timesteps Tasks A, B, and C can be at timesteps

10,12,1110,12,11

Steering parameter updates are marked Steering parameter updates are marked with “apply-at” timestepwith “apply-at” timestep

Execution orderExecution order

ExampleExample

Interface DescriptionsInterface Descriptions

Viewer and steering interfaceViewer and steering interface

Viewer programsViewer programs Text-basedText-based AVS-compatible viewerAVS-compatible viewer Custom TCL/TK viewerCustom TCL/TK viewer

Token scheme prevents conflicting Token scheme prevents conflicting adjustments to the same steering adjustments to the same steering parameter by different usersparameter by different usersAll tasks apply the steering changes in All tasks apply the steering changes in unisonunison

Viewer and Steering InterfaceViewer and Steering Interface

Interfaces forInterfaces for Initializing communications with the user applicationInitializing communications with the user application Requesting data fieldsRequesting data fields Collecting data framesCollecting data frames Allocating data frame storageAllocating data frame storage Dumping data field valuesDumping data field values

User’s data field requests:User’s data field requests: Set of data fieldsSet of data fields Specific region of computational domainSpecific region of computational domain Frequency of “data frames”Frequency of “data frames” Cell size for each axis, i.e., data strideCell size for each axis, i.e., data stride

Supports different Data decompositionsSupports different Data decompositions

Viewer interfacesViewer interfaces

stv_viewer_initstv_viewer_init Initializes link with a specific user application Initializes link with a specific user application Gathers information about the data fields and Gathers information about the data fields and

parameters that are available from that applicationparameters that are available from that application Returns instance identifying a particular applicationReturns instance identifying a particular application

Another call for selecting data fields and Another call for selecting data fields and specifying visualization regionsspecifying visualization regions Data type and storage order can be specifiedData type and storage order can be specified Set of fields specified in a single field request is called Set of fields specified in a single field request is called

“view field group”“view field group” Also specifies visualization regions – bounds and cell Also specifies visualization regions – bounds and cell

sizes of data fieldssizes of data fields

Viewer InterfacesViewer Interfaces

stv_viewer_request_field()stv_viewer_request_field() Field requestField request Returns VFG instance that represents the Returns VFG instance that represents the

group of data fields requestedgroup of data fields requested

stv_viewer_send_XONstv_viewer_send_XON Sends “XON” to the applicationSends “XON” to the application Application waits for XON message, polls for Application waits for XON message, polls for

permission to continue with the next iterationpermission to continue with the next iteration

Viewer Interfaces - MiscellaniousViewer Interfaces - Miscellanious

stv_viewer_set_VisRegion()stv_viewer_set_VisRegion() For modifying visualization region of a VFG on For modifying visualization region of a VFG on

the flythe fly Records new set of visualization bounds and Records new set of visualization bounds and

cell sizes and sends update to the applicationcell sizes and sends update to the application

set_viewer_set_VisFrequency()set_viewer_set_VisFrequency() Controlling frequency of data framesControlling frequency of data frames

Steering interfacesSteering interfaces

For iterative controlFor iterative control

stv_viewer_steering_initstv_viewer_steering_init Creates loosely synchronized connection with the Creates loosely synchronized connection with the

user applicationuser application Performs equivalent of special data field requestPerforms equivalent of special data field request

stv_viewer_steering_requeststv_viewer_steering_request Obtains steering token for a particular parameterObtains steering token for a particular parameter If token unavailable, i.e. another viewer possesses itIf token unavailable, i.e. another viewer possesses it

When the possessing viewer releases the token, CUMULVS When the possessing viewer releases the token, CUMULVS will broadcast a message informing all the requesting will broadcast a message informing all the requesting viewersviewers

Steering InterfacesSteering Interfaces

stv_viewer_steer_parameter, stv_viewer_steer_parameter, stv_viewer_steer_vparameterstv_viewer_steer_vparameter Changes values of parametersChanges values of parameters The routines copy viewer data to an internal The routines copy viewer data to an internal

data structuredata structure

stv_viewer_send_NewParams()stv_viewer_send_NewParams() Changed parameters are sentChanged parameters are sent

stv_viewer_steering_releasestv_viewer_steering_release Will relinquish steering tokenWill relinquish steering token

Protocols for attaching to running Protocols for attaching to running applicationapplication

CUMULVS uses application name CUMULVS uses application name supplied in the init call in the application supplied in the init call in the application for attachingfor attaching

Application name registered in a database Application name registered in a database that indicates how to contact instance 0 of that indicates how to contact instance 0 of the applicationthe application

Different phases – inquiry, request for Different phases – inquiry, request for attachment, data transfer, detachmentattachment, data transfer, detachment

Protocols - InquiryProtocols - Inquiry

InquiryInquiry Viewer looks up the application name, Viewer looks up the application name,

message contextmessage context Sends init0 message to task 0.Sends init0 message to task 0. Task0 reply: total number of tasks, data Task0 reply: total number of tasks, data

decompositions, steering parameter decompositions, steering parameter particulars, current time stepparticulars, current time step

Task0 forwards init0 message to other tasksTask0 forwards init0 message to other tasks Other tasks contact the viewer and send their Other tasks contact the viewer and send their

data and steering parametersdata and steering parameters

Protocols – Field RequestProtocols – Field Request

Field request – 3 phasesField request – 3 phases1.1. Viewer sends which fields are required. Viewer sends which fields are required.

Tasks return their current time steps, Tasks return their current time steps, timestep0. tasks compute till timestep0+1.timestep0. tasks compute till timestep0+1.

2.2. After viewer hears from all tasks, it After viewer hears from all tasks, it calculates maximum of timestep0s, calculates maximum of timestep0s, timestep1. sends timestep1 to taskstimestep1. sends timestep1 to tasks

3.3. Tasks compute until timestep1 and then Tasks compute until timestep1 and then send requested data fields to viewerssend requested data fields to viewers

ExampleExample

Simulating propagation of an acoustic Simulating propagation of an acoustic signal through a heterogeneous mediasignal through a heterogeneous media

Solved by finite difference equationsSolved by finite difference equations

Equation has pressure fields and sourcesEquation has pressure fields and sources

CUMULVS used to introduce arbitrary CUMULVS used to introduce arbitrary “thumps” “thumps”

OutputOutput

Example – CFD code Example – CFD code Supercomputing Conference (from Supercomputing Conference (from

CUMULVS web site)CUMULVS web site)

CFD code for simulation of airflow over jet wing

Peak in residual field due to an omitted column in computation

References / Sources / CreditsReferences / Sources / Credits

P. M. Papadopoulos, J. A. Kohl, B. D. Semeraro, ``CUMULVS: P. M. Papadopoulos, J. A. Kohl, B. D. Semeraro, ``CUMULVS: Extending a Generic Steering and Visualization Middleware for Extending a Generic Steering and Visualization Middleware for Application Fault-Tolerance,'' Proceedings of the 31st Hawaii Application Fault-Tolerance,'' Proceedings of the 31st Hawaii International Conference on System Sciences (HICSS-31), Kona, International Conference on System Sciences (HICSS-31), Kona, Hawaii, January 1998.Hawaii, January 1998.G. A. Geist, J. A. Kohl, P. M. Papadopoulos, ``CUMULVS: Providing G. A. Geist, J. A. Kohl, P. M. Papadopoulos, ``CUMULVS: Providing Fault-Tolerance, Visualization and Steering of Parallel Applications,'' Fault-Tolerance, Visualization and Steering of Parallel Applications,'' International Journal of High Performance Computing Applications, International Journal of High Performance Computing Applications, Volume 11, Number 3, August 1997, pp. 224-236.Volume 11, Number 3, August 1997, pp. 224-236.Falcon: On-line Monitoring and Steering of Parallel ProgramsFalcon: On-line Monitoring and Steering of Parallel Programs, , Weiming Gu, Greg Eisenhauer and Karsten Schwan, submitted to Weiming Gu, Greg Eisenhauer and Karsten Schwan, submitted to the the IEEE Transactions on Parallel and Distributed Systems IEEE Transactions on Parallel and Distributed Systems November 1994, revised September 1995. November 1994, revised September 1995.