1. INRIA Background & ActiveEon’s Mission

2. ProActive:ProgrammingSchedulingResourcing

3. Use Cases: GenomicsFinance

D. Caromel, C. Delbé

Accelerate onWindows Desktops, HPCS, Clouds

with ProActive Parallel SuiteSpeed: Application + Management: Productivity

1. Background & Mission

2

3

INRIA OASIS Team

3

A Research team, about 35 persons 2004: First ProActive User Group 2009, April: ProActive 4.1, Distributed & Parallel:

From Multi-cores to Enterprise GRIDs & Clouds

4

ActiveEon and INRIA synergy

Customer DrivenDevelopments

ResearchContracts

HighAdded Value Services

Customers Partners

Located in Sophia Antipolis, between Nice and Cannes, Visitors Welcome!

The best of research

and industry!

5

ActiveEon Company Overview (1)

Co-developing, Support for ProActive Parallel Suite Worldwide Customers: Fr, UK, USA

5

Selected Partners:

6

ActiveEon Company Overview (2)

Open Source Software Editor Born of INRIA in 2007

Core mission: Scale Beyond Limits

Co-developsProActive Parallel Suite®

Provides high level services for ProActive

Ten years experience in development of parallel computing

2. ProActive Parallel Suite

7

8

ProActive Parallel Suite

Java ParallelToolkit

Multi-Platform Job Scheduler

ResourceManager

Strong Features:Java Parallel Programming + Legacy + Integration + Portability: Windows, Linux, Mac +Versatility: Desktops, Cluster, Grid, Clouds = Perfect Flexibility

Used in Production everyday:50 Cores 300 Cores early 2010

9

ProActive Parallel SuiteThree fully compatible modules

Programming Scheduling

Resource Management

Clutch Power: Solid Building Blocks for Flexible Solutions

ResourcingScheduling

10

ProActive Programming

10

1111

Groups: Broadcast and Scatter (aka MPI)

11

JVM

JVM

JVM

JVM

agcg

ag.bar(cg); // broadcast cgProActive.setScatterGroup(cg);ag.bar(cg); // scatter cg

c1 c2 c3c1 c2 c3

c1 c2 c3c1 c2 c3c1 c2 c3

c1 c2 c3

s

c1 c2 c3

s

Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

1212

Dynamic Dispatch Group

12

JVM

JVM

JVM

JVM

agcg

c1c2c3

c4c5

c6c7

c8c0c9c1

c2c3

c4c5

c6c7

c8c0c9

c1c2c3

c4c5

c6c7

c8c0c9

Slowest

Fastest

ag.bar(cg);

13

Optimizing GUI: IC2D

13

14

Optimizing GUI: IC2D

14

15

Optimizing GUI: IC2D

15

1616

ProActive Scheduling

16

17

ProActive Scheduling Big Picture

RESOURCES

Multi-platform Graphical Client (RCP)

File-based or LDAP authentication Static Workflow Job Scheduling, Native and

Java tasks, Retry on Error, Priority Policy, Configuration Scripts,…

Dynamic and Static node sources, Resource Selection by script, Monitoring and Control GUI,…

ProActive Deployment capabilities: Desktops, Clusters, Clouds,…

ProActiveScheduler

ProActiveResource Manager

18

Job

TaskFlow Example : Picture Denoising

Split

Denoise DenoiseDenoiseDenoise

Merge

• with selection on native executable availability (ImageMagik, GREYstoration)• Multi-platform selection and command generation

• with file transfer

1919

ProActive Resourcing

19

20

Desktops + Clusters + Clouds

20

21

Versatility: Desktops, Grids, Clouds (EC2),Multi-Core, VMs (Hyper-V, Xen, VMware, Kvm)

ProActiveSchedulerProActive

Resource Manager

Dedicated resources

LSF

Static Policy

Amazon EC2

EC2

Dynamic Workload Policy

Desktops

SSH

Timing Policy 12/24

22

RESOURCING User Interface

22

2 mn Video:Scheduler, Resource Manager

23

3. Use Cases

24

Bio-Tech: Genomic Sequencing

25

2626

Resources set up

16nodes Cluster

Desktops

CloudsAmazon EC2

SOLID machine from

Nodes can be

dynamically added!

2727

Scalability:Internal SOLID Cluster to INRIA Cluster The distributed version with ProActive of Mapreads has been tested on the INRIA cluster

with two settings: the Reads file is split in either 30 or 10 slices

Use case: matching 31 millions sequences with the human genome (M=2, L=25)

Reference point with 16 cores(same as in SOLiD machine)

4 Time faster from 20 to 100Speed Up of

80 / Th. Sequential50 Hours 35 Minutes

28

Benchmark: Internal vs. EC2 cloud only

Execution time (min)

Cost (US$)

Standard PBS config 300 NA

ProActive Amazon EC2 340 20 US$

For only $3,2/hour, the EC2 setup has nearly the same performances as the local SOLiD cluster

29

ProActive in Finance with Price-It Excel

3030

Use case: Price-It and ProActive Price-It®

C++ library developed by Pricing Partners Pricing solution dedicated to highly complex

financial derivatives

Accelerated Price-It® A collaboration between Pricing Partners and ActiveEon An implementation based on ProActive Scheduling

Specification and constraints Accelerate Price-It® Excel product

▫ Built on Price-It® library, this product integrates an interface with Excel for input data management and results display

Focus on highly parallelizable Greek computation Operating system: Windows

3131

Price-it Sequential Computing

Price-it®library

Price-it®Computation

DLL

C/C++

Before distribution

3232

Price-it Accelerated by ProActive

Scheduler

Worker Worker

Price-it®Computation

DLL

Price-it®Computation

DLL

C/C++

Price-it®library

Price-it®Computation

DLL

Using distributionSmooth integration of legacy code!

3333

Accelerated Price-It performances Use case: Bermuda Vanilla, Model American MC Test conditions

One computation is split in 130 tasks that are distributed Each task uses 300ko

Sequential Distributed

More than 3 times faster with only 4 nodes!

4 nodes 5 nodes 6 nodes 7 nodes 8 nodes 9 nodes

Even 6 times faster with 9 nodes!

3434

Windows HPCS Benchmarks

35

Overall User Domain and Use Cases

Use Cases:

35

Finance

Life Science and BioTech

Engineering: Accelerating Matlab® and Scilab

IT

Numerical SPMD

Wrap Up

36

37

ProActive Parallel Suite

Java ParallelToolkit

Multi-Platform Job Scheduler

ResourceManager

Flexibility Clutch Power

Portability: Windows, Linux, Mac

Versatility: Desktops, Grids, Clouds

Free Professional Open Source SoftwareCommunity and Enterprise Editions

Choose the Edition you needCommunity EnterpriseUsers support Professional support

www.activeeon.com/downloads

Extra Material

38

39

ActiveEon: Mission, Services, ProductMission: Accelerate your applications:

Development, Deployment, Execution Optimize your Hardware Infrastructure utilization Easy and Accessible Parallel Computing & Clouds

High-End Services & Support: Training, Consulting, Integration, Development Technical Support and Subscription Partnerships (ISVs, VARs for OEM)

Open Source Products:

4040

ProActive : Active objects

40

A

Proxy

Java Object

A ag = newActive (“A”, […], VirtualNode)V v1 = ag.foo (param);V v2 = ag.bar (param);...v1.bar(); //Wait-By-Necessity

V

Wait-By-Necessity is a

Dataflow Synchronization

JVM

A

JVM

Active Object

Future Object Request

Req. Queue

Thread

v1v2 ag

WBN!