1 software and services group 1 execution frontiers cnc support for highly adaptive execution kath...

1Software and Services Group 1

Execution FrontiersCnC support for highly adaptive execution

Kath Knobe Intel

12/07/12


Warning • This is all high level conceptual thinking• Many details to be determined• Today: just the basic idea without any concern for efficiency.• Lots of room for optimizing

Suggestions /comments more than welcome!


Motivation: Highly adaptive computing for exascale

Critical exascale issues (inspired by work on UHPC and X-Stack)Require the ability to move currently executing parts of the app to another place in the platform or to a later time.

• Resilience−Fragile components−Lots of them

• Power management−Power components off−Power components down

• Self-aware computing−Modify mapping based on feedback

• Change of goals−Between power and time to solution, for example

Thesis: management of the execution frontiers in CnC is a mechanism supporting highly adaptive computing for exascale.


Checkpoint/restart Hierarchical CnC

Hierarchical checkpoint/restart

Hierarchical checkpoint/restartFor adaptive execution

2 passes - Abstract: unlimited resources - Actual: with resource constraints

For faults


Outline• Abstract (platform has infinite memory and processors)

−Semantic state−Checkpoint/restart−Hierarchical CnC −Hierarchical checkpoint/restart

• Actual (with resource constraints)• Beyond faults


Outline• Abstract

−Semantic state−Checkpoint/restore−Hierarchical CnC −Hierarchical checkpoint/restart

• Actual • Beyond faults


Outline• Abstract




Semantics / execution model

Itemavail

tagavail



Itemavail

stepcontrolReady

stepdataReady

tagavail



Itemavail

stepcontrolReady

stepready

stepdataReady

tagavail



Itemavail

stepcontrolReady

stepready

stepdataReady

stepexecuted

tagavail



Itemavail

stepcontrolReady

stepready

stepdataReady

stepexecuted

tagavail

The primitive attributes come from below: available, executed The derived attributes propagate at this level: control_ready, data_ready, ready

2 levels:• Graph level (above)• User serial code level (below)


Execution frontier• An execution frontier is a CnC program state:

−The set of attributes of instances of steps, tags and items−The contents of available items

• CnC execution can proceed from a execution frontier

• Some examples of execution frontiers:− Normal program input (set of available items and tags)− Normal program output (set of available items and tags)− Any state during execution (more general)

• Perspective− Traditional focus:

> Data structure is items; computation is step.> step instance consumes and produces items.

− Alternate view: > Data structure is execution frontier; computation is step, subgraph or full program.> Applying a computation to an execution frontier yields another execution frontier.


Outline• Abstract

−Semantic state−Checkpoint/restart−Hierarchical CnC −Hierarchical checkpoint/restart



Checkpoint/restart summary(abstract)• Changes to the execution frontier are saved continuously as they occur

• Changes are saved in less volatile “place”• Asynchronous, no barriers• No programmer involvement• Saved state may not correspond to an actual state • Can restart from any saved state


Outline• Abstract




Cholesky domain spec

TrisolveTag: row, iter

CholeskyTag: iter

UpdateTag: col, row, iter

CONTROL TAG

CONTROL TAG

CONTROL TAG

Cholesky: iter

Trisolve: row, iter

Update: col, row, iter

COMPUTE STEP

COMPUTE STEP

COMPUTE STEP

Array : col, row, iter

DATA ITEM


Looks like a CnC spec at each level

<iterTag: iter>CONTROL TAG

COMPUTE STEP(C: iter)



iterations<iterTag: iter>CONTROL TAG

COMPUTE STEP(cholesky:)


COMPUTE STEP(TU:)



<iterTag: iter>CONTROL TAG


COMPUTE STEP(U:)

COMPUTE STEP(trisolve)

<rowTag: row>CONTROL TAG

COMPUTE STEP(cholesky:)

COMPUTE STEP(TU:)


get…get…… = .. + … *… /… = …if …put

Executed semantics: leafCOMPUTE STEP(trisolve: row)

Executed is a primitive attribute. It comes from below. - Leaf : termination of the serial code below


Executed semantics: non-leaf

COMPUTE STEP(U:)

COMPUTE STEP(trisolve)

<rowTag: row>CONTROL TAG

COMPUTE STEP(TU:)

Executed is a primitive attribute. It comes from below. - Leaf : termination of the serial code below- non-leaf: termination of the subgraph below


Hierarchical CnC application: execution is at the leaves only

Cholesky

trisolve

update


Hierarchical CnC application: intermediate nodes maintain state

State of each iteration

State of each row


Hierarchical view of the abstract platform tree

A node looks like a full machine at each level:a subtree of the memory hierarchy + the associated set of cores

Hierarchical platform node


Abstract platform:Depth and extent of platform hierarchy corresponds exactly

to the depth and extent of the dynamic application

The mapping is direct


Outline• Abstract




Hierarchical checkpoint / restart(abstract)

Hierarchical application node


Hierarchical checkpoint/restart(abstract)

Checkpoint for that application node





resides at the parent place





resides at the parent place


Distinct checkpoints residing at a single place remain separate.

We will see why later.


Abstract failure model

• The system knows if/when a node fails − We’re not talking about soft errors

• Abstract platform node fails temporarily then returns



1-level Checkpoint• Fault • Fullstop• Restart



Checkpoint in hierarchy• Fault • Fullstop• Restart




From above: step simply looks like it took longer than expected.

Checkpoint/fullstop at one node looks like checkpoint/continue for the whole program


Hierarchical checkpoint/restart:Summary

• Each node in a hierarchy has all the characteristics of a whole program checkpoint.

• Checkpoint/fullstop/restart at nodes in the hierarchy enables the application as a whole to adapt and continue through faults.


Outline• Abstract • Actual: with resources and resource constraints


• Beyond faults


Semantic state for execution(limited memory)

• Checkpointed information leaves the trailing edge of the execution frontier−Dead tags−Dead items−Dead stepsThis is the motivation for the term “execution frontier” as opposed to “execution state”. It’s only the relevant frontier of the state.

• Dead is a derived attribute. It doesn’t propagate up from the children. It is derived independently within each (sub)program.


Hierarchical CnC map to actual platformplatform: limited depth / limited extent at each level

Platform hierarchy

Application hierarchy


Hierarchical CnC map to actual platformflatten the depth

Platform hierarchy



Hierarchical CnC map to actual platformfold extent

Platform hierarchy



Actual failure model

• Platform node fails and may not return − or don’t want to wait until it returns

• Restart is at some other platform node


Remapping

A B

Map:


Remapping

A B

A B

Map:


Remapping

X

A BY

A B

Map: Original checkpoint of B is at XNew checkpoint of B is at YFollows the new platform location

A B

A B


Remapping

X

A BY

A B

Map: Original checkpoint of B is at XNew checkpoint of B is at YFollows the new platform location

A B

A B

This is why we don’t want to merge checkpoints of the application children at the platform parent.

We may want to relocate each child independently.


What do we have?

• A way of maintaining the execution frontier of −A running application−A running subgraph of an application

• A mechanism for taking an execution frontier and moving it−To another place−To a later time

• Use of this to cope with faults


Outline• Abstract • Actual: with resources and resource constraints• Beyond faults


Adaptive execution• If we can checkpoint and continue elsewhere on a fault, we

can checkpoint and continue elsewhere for our own reasons. Big relevant exascale issues:−Resilience• Actual/predicted failures

−Power management−Self-aware computing−Changes in goals

• Mechanism not policy!• Status:

−No staffing or funding yet.


Other uses of execution frontiers

• Mechanism for connecting reusable components• Low priority app

− Execute/checkpoint/restart one step at a time − Stop mid-step when high priority work arrives

• Long-lived app with very slowly arriving input − e.g., phylogenetic tree for SARS virus

• Debugging− View state− Reverse time (undo)

• Soft-errors−Compute more than once. Compare

• Something like out-of-core computation but not baked into application


Potential: Forms & operationsForms • As executing

− general, arrays, trees…• Serialized• Streaming• Encrypted• Compressed• Database • Excel • Human readable

Operations • Save/restore• Partition/specialize

−At fork into distinct large subgraphs

• Merge −At join of distinct large subgraphs

• Send • Compare (e.g., for fault

tolerance)• Explicitly modify (e.g., debug)• Rename collections (e.g., for

composition


Relook at motivation: Highly adaptive computing for exascale

Critical exascale issues:require the ability to move currently executing parts of the app to another place in the platform or to a later time.

• Resilience−Fragile components−Lots of them

• Power management−Power components off−Power components down

• Self-aware computing−Modify mapping based on feedback

• Change of goals−Between power and time to solution, for example

Looking forward to:• Lowering the design• Implementation• Experimenting

Looking for feedback and collaborators

1 software and services group 1 execution frontiers cnc support for highly adaptive execution kath...

Documents