program systems institute of the russian academy of sciences research center for multiprocessor...

Program Systems Institute of the Russian Academy of Sciences

Research Center for Research Center for Multiprocessor SystemsMultiprocessor Systems

Doctor Sergei AbramovDoctor Sergei Abramov

Supercomputers and multiprocessor software.

Technologies for regional computer networks building.

Functional programming, supercompilation and metacomputation theories, and their applications to practical programming.


The SKIF Supercomputer ProjectThe SKIF Supercomputer Project of the Russia-Belarus Union State of the Russia-Belarus Union State

Participants: structureParticipants: structure

The standing committee of the Union State

National Academy of Sciences, Belarus

The Ministry of Education and

Sciences

UIIP of NAS of Belarus PSI RAS

Participants from Belarus

Participants from Russia


2000-2004: 16 clusters 2000-2004: 16 clusters producedproduced

2000 2001 2002 2003 2004

“ВМ -5100”48/26G

16-2UIntel P-IV-1500

“M yrin”89/59G

8-1UIntel Xeon 2.8

“К-500”717/415G

64-1UIntel Xeon 2.8

“К-1000”2534/2030G

288-1U ; IB 4x; AM DOpteron 248(2.2)

“ТКС”403/230G

36-1UIntel Xeon 2.8

“Кардиология”9/5 G

3+1-1U+4UIntel P-IV-1266

“Первенец”20/11G

16-3U ;Intel P-III-600

“С тудент”11/6G

9-M iniTowerIntel P-III-600

“Первенец-M ”98/57G

16-3U ; AM DAthlonM P1800+

НИ И мех М ГУ49/28G

4+4-4U+5U; AM DAthlonM P1800+

“T-Forge32”115/74G

16-2U ; AM DOpteron 224(1.8)

“T-Forge48”230/184G

24-1U ; IB 4x; AM DOpteron (2.4)

“T-Bridge8i”45/37G

4-1U ; IB 4x Intel Itanium 2 (1.4)

“T-Forge20H B”88/70G

10-HB; IB 4x; AM DOpteron 248(2.2)

“Гибрид”2.4/1.2G2-4U+6U;

Intel P-III-800

“Первенец”20/11G

16-3U ;Intel P-III-600


Peak performance of the clustersPeak performance of the clusters

Перве-нец

ВМ5100

Кардио-логия

Перве-нец-М

Гиб-рид

НИИмех.М ГУ

Сту-дент

M yrinT-

Forge32

T-Forge20HB

T-Forge

48

T-Bridge

8i

СКИФК-500

СКИФК-1000

ТКС ЕС1710.03

2000 2001 2002 2003 2004

2001 2002 2003 2004G Flops25001000

100

10

1

20

98

717

2534

48

x125 ratio


Linpack-performance of the clustersLinpack-performance of the clusters

Перве-нец

ВМ5100

Кардио-логия

Перве-нец-М

Гиб-рид

НИИмех.М ГУ

Сту-дент

M yrinT-

Forge32

T-Forge20HB

T-Forge

48

T-Bridge

8i

СКИФК-500

СКИФК-1000

ТКС ЕС1710.03

2000 2001 2002 2003 2004

2001 2002 2003 2004G Flops

20001000

100

10

1

11

57

472

2032

26

x185 ratio


2000-2004: Supercomputing Industry. 2000-2004: Supercomputing Industry. Linpack-performance GrowthLinpack-performance Growth


Linpack: performan

ce

Four-year growth (times)

Linpack: performan

ce

Four-year growth (times)

TopSKIF

185 Top200 14

Top1 14 Top300 15

Top10 11 Top400 15

Top100 11 Top500 15

2000-2004: Supercomputing Industry. 2000-2004: Supercomputing Industry. Linpack-performance GrowthLinpack-performance Growth


SKIFSKIF K-500K-500((OctoberOctober 2003, 2003, MinskMinsk))

DevelopersDevelopers

• UIIPUIIP of NAS, Belarusof NAS, Belarus

• NII EVMNII EVM

• The T-Platforms The T-Platforms CompanyCompany

• PSI RASPSI RAS

The project was completed The project was completed in monthsin months


SKIFSKIF K-500: testingK-500: testing((September 2003September 2003, , MoscowMoscow))

The project has The project has been completedbeen completed

in 4 monthsin 4 months,,cost percost per 1 1 TFlops TFlops

((peakpeak))::underunder $700,000$700,000


MinskMinsk: К-500: К-500 ((SeptemberSeptember 2003) 2003)

Peak / Linpack performance 716,8 / 423,6(471.6) GFlops

Processors (Intel Xeon 2.8) 642 =128 items

Number of computation nodes: 64 items.

Main memory: 642 = 128 GB

Disk memory: 6460 = 3 840 GB

System network 4x4х4 3D-top SCI, D336

Accessory network GB Ethernet


SKIFSKIF К-1000 К-1000

Developers UIIP of NAS, Belarus NII EVM PSI RAS T-platforms

Development stages Variant investigation, concept development:

December 2003 — March 2004 Specification preparation: April 2004 Preparation of tender documents, tender:

May 2004 Implementation: July 15 — October 1, 2004, 2½

months


SKIFSKIF К-1000 К-1000

Peak performance2,5 Tflops

Linpack- performance2,0 Tflops

Efficiency factor = 80.1 %

SKIF К-1000 project cost: $ 1 800 000 (several times cheaper than the competition)

Competitions: IBM, HP, Fujitsu-Siemens


"Leaf" IB коммутатор: 24 штук

"Core" IB коммутатор: 12 штук

"Core" GbEth коммутатор: 2 штуки

"Leaf" Eth коммутатор: 8x2=16 ш тук

Управляю щий узел: 1 штука

Рабочий узел: 288 штук

Всего в 8 х 42 = 336 U343 устройствуст ановлено

SKIFSKIF К-1000: К-1000: hardwarehardware

SKIF К-1000:convenient components design,

easily scalableup to 15 TFlops

SKIF К-1000:convenient components design,

easily scalableup to 15 TFlops


SKIFSKIF К-1000 К-1000 is ranked #98 in

the world (the Top500 rating)

Only four countries take the first 100 ranks: Union State, USA, Japan, and China

November 2004: first place in the world according to the TopCrunch rating (the 3 Cars Crash Test)

November 2004: first place on the territory of ex-USSR (the Top50 national rating)


SKIF in theSKIF in the Top50 ratingTop50 rating On December 7, 2004 seven SKIF computers

were listed of fifty most powerful supercomputers of the Commonwealth of Independent States (Тор50, http://www.supercomputers.ru) and took the following places: 1, 6, 8, 20, 22, 32, 34

Installations of the SKIF family take 14% of all places in the list

Their total peak performance amounts to 25% from the sum of all peak performances of all the systems presented in the Top50 list

Their total Linpack performance amounts to 33% from the sum of all Linpack performances of all systems presented in the Top50 list


Linux-SKIF OS kernel PVFS-SKIF — parallel file

system OpenPBS-SKIF — batch

queue FLAME-SKIF — the system

of monitoring and control (reset, power on/off) over installations of the SKIF family

Cluster-level Software Cluster-level Software SuiteSuite



OpenTS — open architecture T-system the TG++ compiler for the

T++ language the TF2TC

translator (T-Fortran T++) TDB — distributed

interactive debugger for MPI programs supporting T-programs debugging (TotalView counterpart)



6 software applications in the OpenTS environment

12 adapted free packages, libraries, and applications

14 in-house designed applications (3 of them in the AI field)

The platform is compatible with commercial engineering packages (over 6 fields of application)


ServNET: ServNET: cluster managementcluster management ( (JuneJune 200 20033))

RS-485 links :up to 125 nodes, up to 1,000 meters

Several links (equaling the number of COM-ports) may be connected to the control station.


ServNET: simple hardwareServNET: simple hardware

Single-sided board. One Atmel microcontroller Two Maxim chips (MAX232A и

MAX487) One chip for ferric RAM (FRAM)

RAMTRON.

Few auxiliary components.


ServNETServNET: functionality: functionality

Selective node reset (SW: groups). Selective switching on/off of node’s

power supply (SW: groups, also “smooth”, with a target latency “between nodes”).

Access to the serial console of the node allowing...


Access to the serial console of the node allowsAccess to the serial console of the node allows the followingthe following::

To change the parameters of the BIOS node. LILO: select an OS to be loaded and the

loading parameters of the Linux core. To execute any commands in a console

regime. To monitor critical messages of the OS. To read several last console messages from

the nonvolatile memory of the ServNET board after failure.


ServNET: productionServNET: production Development – PSI of RAS,

Production – Computer Research Institute, Minsk.

A pilot lot of more than 200 items has been produced. ServNET has been installed in: T-Forge32 (16 nodes). “SKIF К-500” (64 nodes) and other clusters in

Minsk. Clusters in Pereslavl. Clusters in the Research Computing Center of the

Moscow State University (58 ServNET boards). ServNET v.2: 2004, developed by PSI of RAS

and T-platforms, produced by T-platforms.


SKIF-ServNet v.2SKIF-ServNet v.2

The size has become half as large (66 × 33 mm) if compared with the previous version.


The Botik Lab.: ActivitiesDevelops cost effective solutions for

regional computer networks (BOTIK Technologies)

Implements the technology in Pereslavl region for testing and enhancement

Transfers the approved technology to Russian regions and the CIS countries

Design and Development ofDesign and Development ofRegional Telecommunication Regional Telecommunication

SystemsSystemsHead: Yury Shevchuk, Ph.D.


The BOTIK NetworkThe BOTIK Network

Started in 1994: 11-year historyState-of-the-art

More than 900 subscribersMore than 3000 networked PCsExternal traffic: 250 Gb/month

4 technology transfer contracts Almaty, KazakhstanMoscowRamenskoye, Moscow regionSeveromorsk, Murmansk region


The BOTIK Technologies: CoreThe BOTIK Technologies: Core

PC routerIBM PC + Linux + in-house designed HW & SW

Other hardware solutionsResistance to low-quality power supply,

temperature difference, and vandalismSelf-recovery and self-monitoring of devicesWireless solutions for rural regionsEtherbox: sensor networks device

Software solutions for regional networksNetwork Administration System: +10,000

lines…


Urban Networks: Pereslavl-Urban Networks: Pereslavl-ZalesskyZalessky


Rural Networks: Pereslavl RegionRural Networks: Pereslavl Region


Functional programming, Functional programming, supercompilation and supercompilation and

metacomputation theories, and metacomputation theories, and their applications to practical their applications to practical

programmingprogramming


Functional programming Functional programming in IPS RASin IPS RAS

Supercompilation, program specialization. Metacomputation, semantics modifiers –

reusing “semantics components”. Implementation of functional programming

languages. Applied computer algebraic libraries.


Reasons for program Reasons for program optimizationoptimization

The methods can do almost nothing with programs that are already carefully optimized by a professional programmer in a lower-level language. The methods can clean a program of natural inefficiencies if the program has been developed “inefficiently” in a structured way, using various “high-level” techniques like

1. interpretation of specialized application-oriented languages and

2. component programming from libraries of general re-usable software, etc.

Thus, the methods are directed to provide degrees of freedom for new software technologies rather than to optimizing programs written in an old style.


Specialization: the main ideaSpecialization: the main idea

Let human be a program with two parameters knowledge and problem. Then creating a specialist humanknowledge from human and knowledge is a good example of specialization:

humanknowledge(problem) = human(knowledge,problem)

Specialist humanknowledge can solve problems much quicker than an ordinary human when the problems are covered by his specific knowledge.


A number of tasks for A number of tasks for specializationspecialization

<F x0, y>The first argument of a program F is given, while the second is unknown.

<F <G x, y> , z >Let two programs F and G be given, specialize a composition of applications of the programs. That is specialization with respect to acontext of application.

<IntL ( Program ) e.data > <GO >

An interpreter IntL of a programming language L is specialized with respect to a given program. IntL is written in a language M, while theprogram is written in L, so we expect an optimal program written in M asa result of specialization. Thus a specializer may be used as a compiler from L into M, where M is the subject language of the specializer.


What is supercompilation ?What is supercompilation ? Supercompilation is a technique of specialization of programs written

in a functional programming language. The technique was introduced in

the 1970s by V. F. Turchin. He proposed a task of creating tools to observe operational semantics of a program, when a function F that is to be

computed by the program is fixed. As a result of such observations a new algorithmic definition of an extension of the function F must be constructed. His ideas were studied by a number of authors for a long time.

The main aim of a supercompiler is to perform as many actions of a

given parameterized application of a program uniformly on the parameters

as possible.

Also supercompilers can be used • as theorem provers for program verification;• as compilers by specialization of operational specifications; • for porting a non-standard semantics from one programming language to another; by specialization of a semantics modifier.


The Supercompiler SCP4The Supercompiler SCP4

is an experimental specializer for a functional language Refal-5. (There are no special restrictions on the input language.) SCP4 has been implemented once again using Refal-5. Sources of the supercompiler, executable modules and sources of Refal-5 are available for immediate free download: http://www.botik.ru/pub/local/scp/refal5/

Windows 98 Windows NT/2000/XP Linux (Intel)


Specialization of interpretersSpecialization of interpreters

<IntL ( Program ) e.data > <GO >

An interpreter IntL of a programming language L is specialized with respect to a given program. IntL is written in a language M, while the program is written in L, so we expect an optimal program written in M as a result of specialization. Thus a specializer may be used as a compiler from L into M, where M is the subject language of the specializer.

Semantics modifiers (a class of programs that allow the development of general and reusable “semantics components”) can be specialized alike interpreters.


Verification of parameterized Verification of parameterized systems systems

by the supercompiler SCP4by the supercompiler SCP4Successful experiments on verification

of cache coherence protocols:IEEE Futurebus+, MOESI, MESI, MSI,

“Illinois”, “Firefly”, “Berkeley”.More parameterized protocols:

Java Meta-Locking Algorithm, Reader-Writer protocol.


ReferencesReferences[1] Abramov S.M., and Glueck R. From standard to non-standard semantics by

semantics modifiers. International Journal of Foundation of Computer Science, Vol. 12 No. 2, pp:171-211, 2001.

[2] Nemytykh A.P., and Turchin V.F. The Supercompiler Scp4: sources, on-line demonstration. http://www.botik.ru/pub/local/scp/refal5/ ,2000.

[3] Nemytykh A.P., The Supercompiler Scp4: General Structure., LNCS vol. 2890, pp.162-170, 2003.

[4] Nemytykh A.P., A Note on Elimination of Simplest Recursions. In Proceedings of the ACM SIGPLAN Asia-PEPM'02, 138-146. ACM Press, 2002.

[5] Korlyukov A.V., and Nemytykh A.P., Supercompilation of Double Interpretation. (How One Hour of the Machine's Time Can Be Turned to One Second). (In English), Vestnik natcional’nogo tekhnicheskogo universiteta “Khar’kovskogo

politekhnicheskogo instituta”, Khar’kov, No. 1, 2004.

[6] Lisitsa A., and Nemytykh A.P., Verification via Supercompilation. http://www.csc.liv.ac.uk/~alexei/VeriSuper/ , 2005.

program systems institute of the russian academy of sciences research center for multiprocessor...

Documents