cse 403 lecture 02

8/3/2019 CSE 403 Lecture 02

1/21

7

DEMOCRITOS/ ICTP ADVANCED SCHOOL ON HPC TOOLS FOR E_SCIENCE

ICTP Trieste March

High Performance Computing (HPC)

performance is everything (well, almosteverything):

I want ...my calculation run faster and faster...

it ranges from your laptop to the cutting edgesupercomputers

it is not only on hardware but involves softwareand people as well

8/3/2019 CSE 403 Lecture 02

2/21

8


ICTP Trieste March

How to run application faster ?

There are 3 ways to improve performance:

Work Harder

Work Smarter

Get HelpComputer Analogy

Using faster hardware

Optimized algorithms and techniques used to solve

computational tasks

Multiple computers to solve a particular task

8/3/2019 CSE 403 Lecture 02

3/21

9


ICTP Trieste March

defining parallel computing

Parallel computing is the simultaneousexecution of the same task (split up andspecially adapted) on multiple processors in

order to obtain results faster.

The process of solving a problem usually canbe divided into smaller tasks, which may becarried out simultaneously with somecoordination. [from wikipedia]

8/3/2019 CSE 403 Lecture 02

4/21

10


ICTP Trieste March

high performance problem example:

A PARALLEL SOLUTION!

picture from http://www.f1nutter.co.uk/tech/pitstop.php

8/3/2019 CSE 403 Lecture 02

5/21

11


ICTP Trieste March

analysis of the parallel solution:FUNCTIONAL PARTITIONING

different people are executing different tasks

DOMAIN DECOMPOSITION

different people aresolving the same globaltask but on smaller subset

8/3/2019 CSE 403 Lecture 02

6/21

14


ICTP Trieste March

Parallel computers

Tons of different machines !Flynn Taxonomy (1966): helps (?) us inclassifying them:

Data Stream

Instruction Stream

Name Instruction

stream

Data stream

SISD Single Single

SIMD

Single MultipleMIMD Multiple Multiple

MISD Multiple Single

8/3/2019 CSE 403 Lecture 02

7/21

15


ICTP Trieste March

Flynn Taxonomy ( graphical view)

8/3/2019 CSE 403 Lecture 02

8/21

16


ICTP Trieste March

Another important question:

MEMORY: The simplest and most useful way toclassify modern parallel computers is by theirmemory model:

SHARED MEMORYDISTRIBUTED MEMORY

8/3/2019 CSE 403 Lecture 02

9/21

17


ICTP Trieste March

Shared vs Distributed ?

Distributed Memory each processorhas its own local memory. Must domessage passing to exchange databetween processors

multicomputersmulticomputers

Shared Memory

single address space. All

processors have access to a poolof shared memory.

Multiprocessors (MPs)Multiprocessors (MPs)

8/3/2019 CSE 403 Lecture 02

10/21

18


ICTP Trieste March

Shared Memory: UMA vs NUMA

Uniform memory access(UMA):Each processor hasuniform access to memory.

Also known as symmetricmultiprocessors (SMP)

Non uniform memory access(NUMA): Time for memoryaccess depends on location ofdata. Local access is fasterthan non local access.

8/3/2019 CSE 403 Lecture 02

11/21

19


ICTP Trieste March

Distributed memory architecture:

Clusters !Subject: Re: [Beowulf] about concept of Beowulf clustersDate: Thu, 24 Feb 2005 19:41:22 0500 (EST)From: Donald Becker

CLUSTER: independent machines combined into a unified systemthrough software and networking

Inter processor connection mechanism.

Memory

Processor

Memory

Processor

Memory

Processor

Memory

Processor

8/3/2019 CSE 403 Lecture 02

12/21

20


ICTP Trieste March

Beowulf Clusters !Subject: Re: [Beowulf] about concept of beowulf clusters

Date: Thu, 24 Feb 2005 19:41:22 0500 (EST)From: Donald Becker

The Beowulf definition is commodity machines connected by a privatecluster network running an open source software infrastructure forscalable performance computing

this means:

commodity machines: exclude custom built hardware e.g. a singleAltix is not a Beowulf cluster (or even a cluster by the strict definition)

connected by a cluster network: These machines are dedicated tobeing a cluster, at least temporarily. This excludes cycle scavengingfrom NOWs and wide area grids.

running an open source infrastructure The core elements of thesystem are open source and verifiable

for scalable performance computing The goal is to scale upperformance over many dimensions, rather than simulate a singlemore reliable machine e.g. fail over. Ideally a cluster incrementally

scales both up and down, rather than being a fixed size.

8/3/2019 CSE 403 Lecture 02

13/21

21


ICTP Trieste March

The Cluster revolution in HPC

The adoption of clusters, virtually exploded sincethe introduction of the first Beowulf cluster in 1994.

The attraction liesin the (potentially) low cost of both hardware and

softwarethe control that builders/users have over their system.

The problem lies:you should be an expert to build and run efficiently your

clustersnot always the problem you have fit into a clustersolution (even if this is cheap!)

8/3/2019 CSE 403 Lecture 02

14/21

29


ICTP Trieste March

Why Linux ?

Access to cheap hardwareAccess to Source code is needed to implementdesired features.

Availability of software

Access to cheap graduate students

Access to large community

response speed from community sometime

much better then vendor/support ones.open source/ free software: no license Issues.

Availability of Scientific Tools/Resources.

8/3/2019 CSE 403 Lecture 02

15/21

30


ICTP Trieste March04/03/07 30

Cluster Computer Architecture

Sequential Applications

Parallel Applications

Parallel Programming Environment

Cluster Middleware

(Single System Image and Availability Infrastructure)

Cluster Interconnection Network/Switch

PC/Workstation

Network Interface Hardware

Communications

Software

PC/Workstation


Communications

Software

PC/Workstation


Communications

Software

PC/Workstation


Communications

Software

Sequential ApplicationsSequential Applications

Parallel ApplicationsParallel Applications

8/3/2019 CSE 403 Lecture 02

16/21

31


ICTP Trieste March

Building your own HPC infrastructure

HPC infrastructure was extremely expensive a fewyears ago

based on supercomputers

Open source software + commodity off the shelfhardware provides now tools to build low cost HPCinfrastructure

based on clusters

GREAT CHANCE FORLOW BUDGET INSTITUTIONS

8/3/2019 CSE 403 Lecture 02

17/21

32


ICTP Trieste March

Elements of an HPC infrastructure

Hardware

The basic bricks

Software

To make hardware usable

People

installers/sys adm. /planners/ users etc..

Problems to be solvedAny action in building such an infrastructure shouldbe motivated by real needs

8/3/2019 CSE 403 Lecture 02

18/21

33


ICTP Trieste March

Which architectures in yourinfrastructure ?

Parallel computing:

single systems with many processors working onsame problem

Distributed computing:many systems loosely coupled by a scheduler towork on related problems

Grid Computing:

many systems tightly coupled by software, perhapsgeographically distributed, to work together onsingle problems or on related problems

8/3/2019 CSE 403 Lecture 02

19/21

35


ICTP Trieste March

Why the GRID?

Motivation: When communication is close tofree we should not be restricted to localresources when solving problems.

A Grid Infrastructure built on the Internet andthe Web to enable and exploit large scalesharing of resources

It should provides Scalable Secure Reliable

mechanisms for discovery and for remoteaccess of resources.

8/3/2019 CSE 403 Lecture 02

20/21

8/3/2019 CSE 403 Lecture 02

21/21

38


ICTP Trieste March

Wrap upHPC and GRID computing are now fundamentaltools for scientific research

HPC means parallel computing

HPC experienced a great change in the last tenyears: from custom machine to Beowulf clusters

The challenge is now to build your own HPCinfrastructure driven by real needs.

HPC and GRID computing are not mutuallyexclusive but can be both used to addresscomputational resources in a transparent way.

cse 403 lecture 02

Documents