sky not the limit

8/14/2019 Sky Not the Limit

http://slidepdf.com/reader/full/sky-not-the-limit 1/54

1

Serious Supercomputing

The Sky is not the Limit!Presentation to the

Institute for Information Management - Perth

9 December 2009

Brian Haines

BJH Consulting

[email protected]

0412-274-262




Supercomputers




What IS a “Supercomputer”?

A term applied to a ‘computer’ that is at the frontline of current processing capacity -particularly with regard to speed of calculation - at a given point in time.

Calculation speed is measured in Floating Point Operations Per Second (Flops)

(ie Flops = very sophisticated calculations/sec)

1990 Top Supercomputer: The Cray Y-MP. 1 billion operations (flops) per second

(1 billion flops = 1 Gigaflop)

2008 Top Supercomputer: IBM Blue Gene. 1,000 trillion operations (flops) per second

(1,000 trillion flops = 1,000,000 Gigaflops = 1,000 teraflops = 1 Petaflop)

Supercomputers: provide the capacity to undertake research to solve problems that

are impossible to do without such resources

They constitute a service you access, rather than a machine you sit at.




From Gigaflops to Exaflops

Processing Speed Example

1000 Megaflops = 1 Gigaflop 1990 Cray Supercomputer

1000 Gigaflops = 1 Teraflop 150 Teraflops = iVEC HPC power by Jun 2010

1000 Teraflops = 1 Petaflop 1.2 Petaflops

= power of IBM ‘Roadrunner’ Supercomputer

= initial power of Pawsey Supercomputer

1000 Petaflops = 1 Exaflop 1 Exaflop = potential power of Supercomputer required to process the SKA data in 2021

FLOPS - are about speed of operation to produce data

- but you then have to store the data produced as a result of allthese trillions of flops..

- which brings us to BYTES




Storing the Processed Data: A Whole Lotta Bytes!

Data Volume Paper Electronic storage

8 Bits - 1 Byte 100 bytes = av sentence

1000 Bytes = 1 Kilobyte (KB) 100KB = A4 page

1000 Kilobytes = 1 Megabyte (MB) 1MB = small book on a 3.5” floppy disk

Hucklebery Finn = 0.5MB

1.4MB disk for PC insert

1000 Megabytes = 1 Gigabyte(GB) 1GB = 10metre bookshelf or 1000 books

PC internal hard drive: 160 - 500GB

1000 Gigabytes = 1 Terabyte (TB) 1TB = 1000 copies of Encyclopedia Britannica.

15TB = all printed vols of USALibrary of Congress

1 or 2 TB external hard drive for PC

1TB = likely PC internal hard drive by late2010

1000 Terabytes = 1 Petabyte (PB)

1 million Gb = 1 Petabyte

1 billion Mb = 1 Petabyte

1PB = 20 million 4-drawer filingcabinets full of A4 text.

1PB = 500 billion pages of text200PB = all printed material inthe world

iVEC1PB = current size of data storage facility at

iVEC.Some files are 1Terabyte in size.50 million files in all

1000 Petabytes = 1 Exabyte 5 Exabytes = 37,000 Libraries of Congress

Multi Exabytes = data storage archive for SKA in 2021

1000 Exabytes = 1 Zettabyte

1000 Zettabytes = 1 Yottabyte Current content of the world wide web?




1975: State of the Art

PDP11 Computer

Not as powerful as your personal laptop 2009



7


2007 IBM Blue Gene

478 Teraflops

(IVEC HPC = 150 Teraflops)



8


2009 IBM Roadrunner:

US$133 million

1.03 Petaflops

12,240 cell chips, 6.560 dual-coreprocessors

Occupies 278 refrigerator-size server

racks

Twice as fast as IBM Blue Gene

1 hour processing task on 1998machine = 3.6 secs on Roadrunner

Uses 2.5 Mwatt = 2,350 av homes



9


2009 Cray XT5 - Jaguar

362 Terabytes memory10 Petabyte file system

1.64 petaflops speed

(overtakes Roadrunner as theNo 1 Supercomputer in theworld)

Occupies 284 refrigerator-size server cabinets

……….a space the size of a basketball court

Consumes 7 Megawatts - half for operating,half for cooling

( = power for a small town)

Jaguar would take a single

week to run a calculation thatthe fastest supercomputer 10

years ago would have needed20 years to complete




Significance of Petaflop-level Supercomputers

Magnitude of the processing power and storage capacity of the current range of supercomputersmakes them a game-changing technology that is spinning off previously unimagined opportunitiesfor businesses through the capacity to:

Analyse data volumes on a scale never before available, thereby enabling the solution of complexprocess and product issues

Develop new products more quickly through simulation and modeling, thereby achievingsignificant cost savings through the reduction in research and development timeframes

Utilise simulations to augment or replace experimentation in cases where experiments arehazardous, expensive or even impossible to perform or to instrument.

Rapidly evaluate design alternatives, thus improving the quality of engineered products essentialfor industrial prowess and economic competitiveness




A New Paradigm for Conducting eScience

The scientific method has changed for the first time since Galileo invented the telescope.

Supercomputers have pushed simulation

from:

-supporting theory & experimentation

to

- occupying the foreground of scientific research.

Supercomputers have brought us to the point where simulation is actually the third branch of science

Scientists will be able to run new and vastly more accurate models of complex phenomena:

• Climate models will have dramatically higher resolution and accuracy,• New materials for efficient energy transmission will be developed




Supercomputers in action in the real world

Animal Logic, an Australian animation company, was only able to create the penguins in the award-winning animated movie Happy Feet by using the processing capacity of Australia’s sole ‘Top 500’supercomputer.

Boeing used supercomputer simulations to reduce the number of physical wing tests for the 787Dreamliner to just 11, versus 77 for the 767

Proctor & Gamble used computational models to solve the problem of the ‘Flying Pringles’. Without theright geometric shape, the potato chips would literally fly off the manufacturing line instead of dropping intothe can. Simulation is an integral part of P&G’s design processes for many of its numerous products.

Alcoa - WA - used HPC at iVEC to simulate the most efficient movement of fluids in a tank(“Computational Fluid Dynamics’)

Alcoa - USA was able to cut the $100,000 normally required for designing a beverage can in the laboratoryto $2,000. - using simulations.- Also saved about $200 million annually through the reduction in metal required for the manufacture of

cans, afforded by the analytical powers of the supercomputer.

Through HPC simulation, Goodyear’s expenditure on tyre building and testing dropped from occupying40% of the research, design and engineering budget, to just 15%

.iVEC research partners in private industry have used iVEC computing facilities as a test bed - successfuloutcomes have justified the org buying its own (modest) HPC facility

REFERENCE: http://www.compete.org/publications/




Burj Dubai hotel: 818metres high, 162 floors

(Tallest building in the world)

Number of today”s laptops with 3 Gigabytesmemory to equal Jaguars’s 363 Terabytesmemory: 120,660

120,660 laptops = a stack 2.4 kilometres high

= 3 x height of Burj Dubai

A layman’s analogy



14


iVEC HPC Cluster

(at ARRC)

(Not equivalent to 2.4km stack of laptops)

HPC room = approx 36 sqm



15


iVEC Petabyte Data Store



16


iVECPetabyte Data Store



17


iVEC Petabyte Data Store



18


•2000 – 2001 Disk had started doubling incapacity

• Tape capacity stagnated•people claimed tape was dead

• By 2002 tape had started to increase

capacities•road map to double tape capacity every 18months

• Today – 1Terabyte will fit on a single tape

• Tape is greener – minimal environmental

footprint

Why Tape?



19


About

iVEC• Five members:

– Four publicuniversities

– CSIRO

• Three facilities• Education and

IndustryPrograms

• State and federal

governmentfunding




iVEC: The hub of advanced computing in WA

Three Facilities:Aust Resources Research Centre (ARRC) Technology Park, Kensington, WA

WA Supercomputer Program (WASP) - University of WA

Informatics (Murdoch University)

Informatics helps develop new uses for information technology in order to design solutions that reflectthe way people create, use and find information. It takes into account the social, cultural andorganizational settings in which those solutions will be used.

Three Programs

Education

IGUP

eResearch

Technology

HPC

Hi-Speed Communications via Access Grid 10Gigabits/sec nationally

Petabyte data storage

Scientific Visualisation




QuickTime™ and adecompressor

are needed to see this picture.



23




24




25


Broadening iVEC’s HPC Userbase

2002 - 2003 2007 - 2008Chemical Sciences Chemical Sciences

Agric, Veterinary & Environ Sci Agric, Veterinary & Environ Sci

Physical Sciences Physical Sciences

Earth Sciences Earth SciencesEngineering & Technology

Medical & Health Sciences

Biological Sciences

Information, Computing &Communication Sciences

iVEC Industry Programs

4 Disciplines - 11 Projects 9 Disciplines - 76 Projects



26


iVEC Provides eResearch Infrastructurea.k.a. eScience infrastructure or Cyber-infrastructure

Hardware, software, networking and human resources

Components of each project may include: Shared large, expensive instruments

High performance computing Scientific visualisation equipment & processes High speed networks Collaboration tools, e.g. Access Grid Shared data repositories Web portals (access resources and share information)

Researchers choose a palette of components

iVEC researchers produce 60 - 70 research papers eachyear

S i S i



27


27

So, what is eResearch infrastructure about?

Goals

– To enable researchers to accomplish studies of national significance which are

beyond the scope of individuals and to tackle problems otherwise impossible

– To build a foundation eResearch capability that provides an innovative,

sustainable approach to ICT enhanced research

Outcome

– A globalised, information-rich, collaborative research environment that

accelerates and enhances national research priorities

– - all hinges on the availability of high-speed communication networks

S i S ti



28


WASP (UWA): Immersive Dome - Mawson’s Hut, Antarctica

S i S ti




The WASP iDome at University of WA

iDome - is an immersive environment for exploring data and virtualenvironments, and the evaluation of game engines (eg SecondLife) as ameans of presenting and collaborating with 3 dimensional data.

Outcomes from these activities include an iDome installation at SciTech, afreely downloadable interactive game based upon the ASKAP radiotelescope, and a number of seminars and papers on these topics.

S i S ti



30


$80 million Super Science funding for

Pawsey HPC Centre for SKA Science

Federal Budget announcement 12th May 2009

“… to be established in Perth to host new high performance computing facilities and expertise to support SKA research and other high-end science ”

EasternAustralian

telescopesonly Steve Tingay, 2009

S i S ti




WA’s New Supercomputer

In its May 2009 Budget the Commonwealth Government announced a Super Science Program that willfund a new major supercomputer in Western Australia to be designed and managed by iVEC.

When the supercomputer goes live and is connected to the Australian National eResearch Grid in2012/13, it will have involved the commitment of more than $100 million and will be amongst the world’stop 20 most powerful supercomputers.

A specific purpose building adjacent to the Australian Resources Research Centre at the WesternAustralian Technology Park will be designed and constructed over the next two and a half years, to housethe high performance computing system. The supercomputer itself will be designed and assembled over the next four years.

In technical terms the supercomputer system will be a shared petascale facility to serve the needs of leading Australian computational researchers with a special focus on radio astronomy.

The funding for the new supercomputer constitutes the largest ever single investment in scientificinfrastructure in Western Australian history and will revolutionize scientific research in the state.

Serio s S percomp ting




The Pawsey HPC Centre for SKA Science

The Building:

Approx 2,500 sqm, adjacent to ARRC, Technology Park

Construction: Jan - Dec 2011

The Supercomputer

1+ Petaflop for diverse usage requirements

Scoping: Jan - Dec 2010

Procurement: Jan - Dec 2011

Installation: Jan 2012 - June 2013

Funding Agreement with Commonwealth Government signed 4 Dec 2009





The Pawsey HPC Centre for SKA ScienceIn 2010

Fifteen million dollars will be spent on upgrading iVEC’s existing HPC capacity to 150TFlops.

Design and planning work for the building to house the petascale supercomputer will beundertaken.

Design work on the specifications of the supercomputer will begin.

In 2011 The new Pawsey Centre building will be constructed and further detailed planning for the supercomputer will occur.

In 2012 The petaflop supercomputer will be procured and assembled

In 2013 During the second quarter of 2013 the supercomputer will be commissioned and thePawsey Centre will be open for business.

By June 2013 Australia will have at Technology Park in Perth, one of the world’s top 20supercomputers servicing the needs of SKA science, marine science, resources, andnanotechnology amongst others.




34


SKA

The Square Kilometre Array Project





The SKA Project

An international, multi-billion dollar program that will require innovation to deliver unprecedented

technology capabilities and address complex engineering challenges.

Will be implemented through an international program comprising 19 participating countries and over 50 research institutions.

Site selection in 2012 - operational over 2 phases,by 2021 - lifetime of 50 yrsEstimated implementation budget of 1.5 billion Euros.

Will enable observations of the Universe further back in time, over a larger volume of space and withimproved spatial resolution.

Will produce enormous data volumes that will require transmission, processing, storage and retrieval,and distribution to scientific communities worldwide.

From an ICT perspective, the challenges are immense. A lot of the technology that will be required in

2021 has not been invented yet!




36


SKA: How it will Work





The SKA Project

SKA is the next generation radio telescope- 50 times more sensitive than the largest existing radio telescope- capable of surveying the sky 10,000 times faster, than existing facilities.

SKA will go:

WIDER (range of data scanning)

DEEPER (back in time)

LOWER (trap low radio frequencies never before traceable)

The Radio telescope is effectively a time machine - astronomers can look

back into the past to study the Universe as it was billions of years ago





SKA: Overview of the Process





Receiving Signals

Fifty percent of the collecting area = an inner core spanning a five kilometre diameter.

The remaining collecting area distributed across an area spanning 3,000 to 5,000kilometres.

If the SKA is centered in Western Australia, a 5,000 kilometre radius will require stationsdistributed across the Australian continent as well as some located in New Zealand.

The next slide shows the inner core, including part of the tile array and surrounding dishantennas.





Receiving Signals





Distribution of Antennae

The next slide shows the distribution of antennae in a spiral pattern radiating from the inner core. (clusters of dishes).

Signals from the separate antennas will be digitally combined to simulate a single telescopewith a diam equal to the distance separating the two furthest antennas (ie 3-5K km). The resultis a series of consolidated images drawn from thousands of dish arrays.

Within the inner array (central core) data will be transported at rate of 80 gigabytes/second.

Longer links servicing the outer clusters of dishes (remote array stations) will need a capacityof approx 2 Terabytes/second/station. This is more than the current total internet traffic in

Europe.





Location??Murchison RAObservatory (MRO)

Mid-West Region

Western Australia

‘Meerkat’

Northern CapeProvince

South Africa





Pathfinders

Australian SKA Pathfinder

http://www.atnf.csiro.au/projects/askap/ Murchison Widefield Array http://astronomy.curtin.edu.au/research/mwa.cfm

Meerkat http://www.ska.ac.za/index.shtml




45


Australian SKA Pathfinder A new radio telescope for

Australia1% of the SKA in sizeData collected in first 6 hours

= all data collected , ever!Built on the new Murchison

Radio-astronomy Observatory

A demonstration of quality of the Australian site

A showcase for Australiantechnology and science

Fully funded with $106m inFederal funding

First data 2010, Operational2013

Australian SKA Pathfinder http://www.atnf.csiro.au/projects/askap/




Se ous Supe co pu g

ASKAP: Model of Key Components




47

p p g

MWA Tile Array




p p g

ASKAP (1% of SKA) SKAConstn Phase 2009 - 2012 2012 - 2021

Dish Antennas 36 3,000+

Receivers 7,200 600,000+

ICT Domain

SoftwareEngineering

approx 50 person years devlpttime

approx 500+ person years devlpttime

HPC 100 Teraflops to 1 Petaflop 100’s of Petaflops to 1 Exaflop

Data Storage Product rate: Terabytes/dayData Archive: 10 Petabytes

Product rate: Petabytes/dayData Archive: Exabytes

DataTransmission

160 Gigabits/sec 1,600 Gigabits/sec

ASKAP - SKA Comparison




p p g

Challenges to be Faced

Meeting the power requirements SuperComputers- 2009: Jaguar (Petaflop) - Consumes 7 Megawatts of power - of which approx40% is used to cool the processing units

- 2020: An Exaflop computer - May consume 100 Megawatts (enough for a city)

- 1 Megawatt of power costs approx $1 million

- Pawsey is investigating geo-thermal solutions to heating and coolingrequirements, as a way of reducing running costs

Educating potential users of supercomputers

- developing knowledge & understanding of application of supercomputing tobusiness & government

- building the technical skills required to operate supercomputers




p p g

All the elements of an InformationManagement Policy frameworkwill need to be re-cast in thecontext of distributed petabytedata storage & retrieval

Implications for IM




g

Implications for Information Management

Cannot just ‘store’ Excel, Word docs in a petabyte data store - they are proprietary formats -

(will you be able to retrieve and open a Word document in 15 years time?

Key areas of interest from a data store perspective:

Description (consistency of metadata used eg across all govt depts)

Ownership (custodianship, accountability)

Compliance (new standards & protocols for data store compatability)Sharing (eg making data searchable from client end)

PLUS

Document creators, custodians & managers will need to learn new skills in order to be ableto extract & export docs from traditional document management systems and translate theminto a format suitable for Data Store import.




Broader Implications for Information Management

An new era of collaborative research using distributed services, virtual storage,cloud computing.

Exponential rate of change in search and retrieval tools.

Dominance of digital documentation - with a corresponding decline inpaper-based archives.

New parameters for data storage, archiving, access, retrieval, distribution.

Hidden cost of physical facilities (power consumption & cooling costs).

Cloud computing “the next big thing” - software as a service, virtual access &storage of information.

•All the major software players and telcos are getting involved.

•A technology still in its infancy - but plenty of early adopters.




The Future?

In 1994 - the internet was in its absolute infancy, the web ran atcrawl speed - and we were in awe of this new technology thatpromised to transform our relationship with information and withothers.

15 years later - we are facing the emerging paradigm of supercomputing, as it starts to enter our consciousness andcreate yet another transformation in our view of the world and our

capacity to engage with information in ways never previouslyimagined.




Acknowledgements

Swinburne images: Image: Artist’s impression of ASKAP at the MurchisonRadio-astronomy Observatory (MRO)

-Credit: Swinburne Astronomy Productions. Design data provided by CSIRO.

Scitech images: Images courtesy of Scitech and Questacon

iVEC images: Images courtesy of iVEC

sky not the limit

Documents