1 using the weather to teach computing topics b. plale, sangmi lee, aj ragusa indiana university

1

Using the Weather to Teach Computing Topics

B. Plale,Sangmi Lee,AJ Ragusa

Indiana University

2

Outline• Forecasting Severe Storms

– Why a better computing infrastructure is needed– Grid computing addresses the problem– Work being done in context of LEAD project

• http://lead.ou.edu

• Computing architecture to enable better weather forecasting

• Demo

3

Motivation for LEAD• Each year, mesoscale weather – floods, tornadoes,

hail, strong winds, lightning, and winter storms – causes hundreds of deaths, routinely disrupts transportation and commerce, and results in annual economic losses > $13B.

4

Conventional Numerical Weather Prediction

OBSERVATIONS

Radar DataMobile Mesonets

Surface ObservationsUpper-Air BalloonsCommercial Aircraft

Geostationary and Polar Orbiting Satellite

Wind ProfilersGPS Satellites

5

OBSERVATIONS





Analysis/Assimilation

Quality ControlRetrieval of Unobserved

QuantitiesCreation of Gridded Fields


6




Prediction

PCs to Teraflop Systems


OBSERVATIONS





7




Prediction


Product Generation, Display,

Dissemination


OBSERVATIONS





8




Prediction



Dissemination

End Users

NWSPrivate Companies

Students


OBSERVATIONS





9




Prediction



Dissemination

End Users


Students


OBSERVATIONS





The Process is Entirely Serialand Pre-Scheduled: No Response

to Weather!

The Process is Entirely Serialand Pre-Scheduled: No Response

to Weather!

10




Prediction



Dissemination

End Users


Students

The LEAD Vision: No Longer Serial or Static

OBSERVATIONS





11




Prediction



Dissemination

End Users


Students

The LEAD Vision: No Longer Serial or Static

OBSERVATIONS





12

The Value of Being Able to Respond to

the Weather: Dynamic Adaptivity

13

Radar Radar ObservationsObservations

of aof aStorm SystemStorm SystemIn Kansas onIn Kansas on20 June 200120 June 2001

14

11-hr Forecast 11-hr Forecast

15


16


17


Moral: Need to do more short forecasts, because they are more accurate

18

The Value of Local Observations

19

What Do Operational Forecast Models Currently Predict?

Bands of rain, and high and low pressure, but that’s about it.

20

What Causes the Problems?

Do we really understand the conditionsthat result in a funnel cloud?

21

Why the Lack of Detail in Current Forecasts?

This ThunderstormFalls Through the Cracks

22

Why the Lack of Detail in Current Forecasts?

23

The Solution….

Fine-Scale Local Observations

Fine Grid Spacingin Forecast Models+

24

Example: The March 28, 2000 Fort Worth

Tornado

25

TV Radar Image of the Hook Echo

26

NWS 12-hr Forecast Valid Near Tornado Time(shading indicates precipitation)

27

6 pm 7 pm 8 pmR

adar

Hourly Radar Observations(Fort Worth Shown by the Pink Star)

28

6 pmR

adar

Com

pu

ter

For

ecas

t

2 hr

29

6 pm 7 pmR

adar

Com

pu

ter

For

ecas

t

2 hr 3 hr

30

6 pm 7 pm 8 pmR

adar

Com

pu

ter

For

ecas

t

2 hr 3 hr 4 hr

31

Fcs

t w

/o R

adar 2 hr 3 hr 4 hr

Rad

ar6 pm 7 pm 8 pm

32

Outline• The Weather

– Why cyberinfrastructure is needed– LEAD project – addressing the problem

• Cyberinfrastructure based on a web service architecture for the Grid

• Prototype demo

33

What is the Grid?• A collection of resources (computers, databases,

telescopes, etc.) that can be used by a wide range of users with a wide range of skills.

• More than the Internet– Built on top of the Internet

• The “Grid” is a collection of web services layered on top of the Internet.

SecuritySecurityData Management

Service

Data ManagementService

AccountingService

AccountingServiceLogging

Logging

Event ServiceEvent Service

PolicyPolicy

Administration& Monitoring


Grid OrchestrationGrid Orchestration

RegistriesRegistries

Reservations And Scheduling


Web Services layer

Internet

Physical Resource Layer

34

Predicting Severe Storms

Lightning Data Server

NEXRAD Radar

Data Server

SatelliteData Server

Surface and Upper-Air

Data Server

SUNY AlbanySUNY Albany

Wisconsin/SSECWisconsin/SSECNASA, NOAAPortNASA, NOAAPortEROS Data CenterEROS Data Center

I D D

I D DI D D

I D D

Historical Observations and

Model Output

Operational Model Grids and Server

Project Project CONDUITCONDUIT

I D D

NOMADSNOMADSNCDCNCDC

HydrologicData Server

I D D

NWS RiverNWS RiverForecast CentersForecast Centers

Air Quality Data Server

EPAEPA

I D D

GPS Meteorological

Data Server

SuomiNetSuomiNet I D D

Oceanographic Data

DODSDODS

Digital Library Holdings

DLESEDLESE

DemographicData Server

Field Program & User Generated

Data

UCAR/JOSSUCAR/JOSSIndividual InvestigatorsIndividual Investigators

Abilene/NGI

I D D

I D D

I D

D

Large scale, real-timeSimulation Grid

The LEAD projectUniv. of Oklahoma

35

Very Simple Scenario to Run Forecast

• Search for data set, run simulation, and catalog results.– Query metadata catalog for dataset– Use result of query a large WRF simulation– Allocate storage on remote resource– Move WRF output to that allocated space– Record output location and computation history in a metadata

catalog.

• How does a user describe such a scenario as a workflow or distributed application?

• How do we free the user from details of distributed computing in a service oriented architecture?

• What does a service architecture mean in this context?• Can it be done by a component composition approach?

36

Web Services• Why does the web work?

– A language with few verbs (get, put, post) and many nouns (documents).

• Corba & Java RMI are object models which present a problem. – Object identity and lifetime is bound to its container, – Whereas a web address is persistent.

• RPC/RMI requires too much synchronization– For reliability make “connections” implicit.– Communicate with simple “standard” message

exchanges.

37

So what is a web service?• A network “endpoint”, i.e. server, that implements one or

more “ports”– Each port is defined by the message types that accepts and the

messages it returns.

• A Web Service is specified by a “Web Service Definition Language” xml document.– Given the WSDL for a web service you know all you need to interact

with it.

• Web Service Standards exist for security, policy, reliability, addressing, notification, choreography and workflow.– It is the basis for MS .NET, IBM Websphere, SUN, Oracle, BEA,

HP, …– It is the basis for the new Grid standards like WSRF and OGSA.

38

Web Site vs Web Service• The Web Site

– Designed to pass http get/post/put request to between a browser and a web server.

– Google has a web site.

• The Web Service– Designed for services to talk

to other services by exchanging xml messages

– Google also provides a web service so Google may be used in distributed apps

Client’s Browser

WebServer

WebServer

WebService

WebService

WebService

WebService

WebService

WebService

39

An Example

• The program:– Run a query against a metadata catalog and

extract simulation boundary conditions– Allocate storage for simulation output– Run the simulation– Save result metadata reference for output to

the metadata catalog.– Record event log of execution to the catalog.

• Services/components in our example are– Metadata catalog– Storage Allocator– WRF Simulation Engine– Execution history recorder

Metadata Catalog

Metadata Catalog

query

input

output

Query results

Metadata Catalog

Metadata Catalog

referenceinput

output

notification

mdata

40

The Workflow – as specified by the scientist

WRFFactory

Storage requirements SpaceAllocator

SpaceAllocator

File MoverFile Mover

“done”

“done”

Metadata Catalog

Metadata Catalog

“done”

Resource info

Experiment Name(Notification Topic)

Output URL

NotificationBroker

Final URL

Parameter file

EventListener

EventListener

“done”

Metadata Catalog

Metadata Catalog

query

41

The Portal

• User’s View of the Grid

• A very sophisticated web browser.

• Lets a classroom teacher create an experiment (to run a forecast model for Hurricane Ivan), then submit the experiment to the “Grid” for computing. The results can be viewed graphically within the portal.

42

Portal as Point of Access to Grid

SecuritySecurityData Management

Service

Data ManagementService

AccountingService

AccountingServiceLogging

Logging

Event ServiceEvent Service

PolicyPolicy



Grid OrchestrationGrid Orchestration

Registries andName binding

Registries andName binding



Open Grid Service Architecture Layer

Web Services Resource Framework – Web Services Notification

Grid Portal Server

Grid Portal Server

https

Physical Resource Layer

SOAP & WS-Security

Keeps informationabout all thedifferent users

43

Portal Architecture (OGCE)• Building on Standard Technologies

– Portlet Design (JSR-168) IBM, Oracle, Sun, BEA, Apache

– Grid standards: Java CoG, Web/Grid Services• User configurable, Service Oriented• Based on Portlet Design

– A portlet is a component within the portal that provides the interface between the user and some service

– Portlets can be exchanged, interoperate

Po

rta

l co

nta

ner

LocalPortlets

Grid Service Portlets

JavaCOGAPI

Java CoGKit

Grid Services

GridProtocols

GRAM,MDS-LDADMyProxy

SOAP ws call Grid Services

Web Services

Client’s Browser

44

FactoryFactory

myLEADagent

myLEADagent

WRF modelWRF modelData miningtask

Data miningtask

workflowworkflow

myLEADservice

myLEADservice

LEADPortal service

LEADPortal service

Storage Repository

service

Storage Repository

service

myLEAD portlet

/var/tmp/wrf_tmp

IU NCSA

1

4

32

2

8

9

3

6

7

5

Putting it together

45

Managing Workflow

1. Portlets exist to submitjobs to a condor web-service and monitorresults

2. BPEL4WS is web-serviceworkflow standard. Interface is underdevelopment.

3. CCA componentscan also be managedfrom the portal.

46

Science Portal Deployments in Collaboration with OGCE, DOE Fusion Portal, NCSA,NPACI/SDSC and others

51

Thank You

1 using the weather to teach computing topics b. plale, sangmi lee, aj ragusa indiana university

Documents

mesoscale weather floods

computing architecture

grid computing

computing topics

lead vision

severe storms

winter storms

context of lead project