© geodise project, university of southampton, 2004. geodise: grid-enabled toolkits for the...

© Geodise Project, University of Southampton, 2004.http://www.geodise.org/

GEODISE: Grid-enabled toolkits for the Engineer

Andrew PriceUK e-Science Programme, Booth 2140

SC2004, Pittsburgh, USA

10th November 2004


Grid Enabled Optimisation and Design Search for Engineering (GEODISE)

Simon Cox- Technical Director Southampton e-Science Centre. Grid/ W3C Technologies and High Performance Computing

Andy Keane- Director of Rolls Royce/ BAE Systems University Technology Partnership in Design Search and Optimisation

Mike Giles- Director of Rolls Royce University Technology Centre for Computational Fluid Dynamics

Carole Goble- Ontologies and DARPA Agent Markup Language (DAML) / Ontology Inference Language (OIL)

Nigel Shadbolt- Director of Advanced Knowledge Technologies (AKT) IRC

BAE Systems- Engineering

Rolls-Royce- Engineering

Fluent- Computational Fluid Dynamics

Microsoft- Software/ Web Services

Intel- Hardware

Compusys- Systems Integration

Epistemics- Knowledge Technologies

Condor- Grid Middleware


The GEODISE Team• Richard Boardman• Sergio Campobasso• Liming Chen• Mike Chrystall• Trevor Cooper-Chadwick• Simon Cox• Mihai Duta• Clive Emberey• Hakki Eres• Matt Fairman• Mike Giles• Carole Goble• Ian Hartney• Tracey Hunt• Zhuoan Jiao

• Andy Keane• Marc Molinari• Graeme Pound• Colin Puleston• Nicola Reader• Angus Roberts• Mark Scott• Nigel Shadbolt• Wenbin Song• Paul Smart• Barry Tao• Lisa Telfer-Brunton• Ivan Voutchkov• Jasmin Wason• Fenglian Xu• Gang Xue


Outline

• Description of Design Search and Optimisation

• GEODISE Solution

• Compute Toolbox

• Database Toolbox

• Applications


Design Optimisation

• What is Engineering Design Search & Optimisation (EDSO)? Improving designs by exploiting engineering modelling and analysis. Systematically modifying variables in a design to increase, or reduce a

quality measure (called the objective function), within certain constraints.

Involves computational and data intensive processes.


Geodise Solution

• Bring Grid Technology to Engineers Develop tools that are easy to learn and to use. Integration into environments familiar to engineers (e.g. Matlab, Jython).

• Geodise Toolboxes

Compute Toolbox – enables applications to run on the Grid.

Database Toolbox – manages data produced by the grid applications.

XML Toolbox – converts Matlab/Jython data structures to/from XML.

Knowledge Toolbox – uses knowledge from domain experts to provide advice; tools to support function and workflow annotations.

Workflow Tool – helps engineers to construct, execute, monitor and manage workflows.


Compute Toolbox Architecture

Matlab Environment

Globus GT2 Resource GRAM

ServiceGridFTPService

gd_jobsubmit.m

Java CoG

gd_getfile.m

script.m

GridFTP file transfer

GRAMjob submission


Certificate Management Functions

gd_certinfo Returns information about the user's certificate

gd_createproxy Creates a Globus proxy certificate

gd_proxyinfo Returns information about the user's proxy certificate

gd_proxyquery Queries whether a valid proxy certificate exists

gd_destroyproxy Destroys the local copy of the user's Globus proxy certificate


Certificate Management

C:\GEODISE

>> gd_createproxy


Job Submission Functions

gd_jobstatus Gets the status of a Globus GRAM job

gd_jobsubmit Submits a compute job to a Globus GRAM job manager

gd_jobpoll Queries the status of a Globus GRAM job until complete

gd_jobkill Kills a Globus GRAM specified by a job handle

gd_chmod Changes file permissions of a file on a Globus resource

gd_condorsubmit Submits a job to a Condor pool via a Globus resource

gd_submitunique Submits a GRAM job to a unique working directory

gd_listjobs Returns all the job handles belonging to the user


Job Submission

C:\GEODISE

>> RSL = '&(executable="/bin/date")(stdout="date.out")';>> host = 'grid-compute.oesc.ox.ac.uk';>> jobhandle = gd_jobsubmit(RSL, host)

jobhandle =

https://escience-dept2.sesnet.soton.ac.uk:30001/30705/1098694366/

>> isdone = gd_jobpoll(jobhandle)

isdone =

1

>>


GridFTP File Transfer

gd_getfile Retrieves a remote file using GridFTP

gd_putfile Puts a remote file using GridFTP

gd_transferfile Performs a third-party file transfer using GridFTP

gd_makedir Creates a remote directory using GridFTP

gd_listdir Lists the contents of a directory on a GridFTP resource

gd_fileexists Tests the existence of files on a Globus resource

gd_rmdir Deletes a remote directory using GridFTP

gd_rmfile Deletes a remote file using GridFTP

gd_rmuniquedir Deletes a remote directory and its contents


File Transfer

C:\GEODISE

>> exists = gd_fileexists(host, 'date.out')

exists =

1

>> gd_getfile(host,'date.out','localfile.txt');>> type('localfile.txt')

Mon Oct 25 09:52:46 BST 2004

>> gd_rmfile(host, 'date.out');>> exists = gd_fileexists(host, 'date.out')

exists =

0

>>


Data Management Challenges

• Data may be generated from distributed applications running on the Grid.

1 0 00 2 00 0 3

• Traditionally data stored on file systems with little descriptive information – hard to find and share.

• Engineers may want to access the data from distributed locations.


Need Metadata

Globus Server

ArchiveFiles

ArchiveData Structures

DatabaseArchive

Metadata

Locate Files

Query

Retrieve Files

Retrieve

Structu

res

Metadata

obj=2

airfoil

cad

1 0 00 2 00 0 3

x.y.z = 43


Database Toolbox Overview

• Store data with descriptive information Standard and application specific metadata. Query over metadata to easily locate required data. Data retrieval based on ID rather than location.

• Familiar interface for engineers Wrap toolbox as Matlab/Jython functions – easy integration. Can be used in Matlab/Jython scripts – popular among engineers.

• Support data aggregation (data groups) concept.

• Central and local databases (shared vs. personal).

• Secure Web service access to central database over SSL. Certificate-based authentication and authorisation.


Architecture

JavaClient Code

CoG

ApacheAxis

JavaClient Code

CoG

ApacheAxis

JavaClient Code

CoG

ApacheAxis

GeodiseMatlab

Functions

Globus ServerGlobus ServerGlobus ServerGeodise Database

Toolbox

Metadata Database

Client Grid

SOAP

GridFTP

Jython

GeodiseJython

Functions

WCE

Geodise Database Web Services

AuthorisationService

LocationService

Metadata Archive & Query

Services


Applications - Geodise

Negative Scarf InletConventional Inlet

0 1 2

CFD 3D engine nacelle optimisation

• Goal - reduce ground noise generated by fan when plane takes off.

• Optimise aerodynamic performance when scarf angle is varied.


Applications - GEM

• Large number of designs, parameters and solutions.

• Query for a particular data range to post-process.

Electromagnetic optimisation

Transmission of light properties through a photonic crystal.

Parameters are radius of holes and light wavelength.


Applications – GENIE

• GENIE supports environmental scientists modelling long term climate change.

• Optimising a parameterised model over a multi-dimensional state space.

• Database used for monitoring, sharing, post-processing and fault tolerance.

Tuning Earth system model components

Default

Tuned

Target


Summary

• Design Search and Optimisation

• Computation/ Middleware Compute Toolkit

• Globus & Condor• Using UK Level 2 Grid

Workflow in Matlab

• Database XML Toolkit Database Toolkit

• Archive for Files• Archive for Metadata• Query & Retrieve

• Geodise toolkits exploited by a variety of applications

APPLICATION SERVICE

PROVIDERCOMPUTATION

GEODISE PORTAL

OPTIMISATION

Engineer

Parallel machinesClusters

Internet Resource ProvidersPay-per-use

Optimisation archive

Intelligent Application Manager

Intelligent Resource Provider

Licenses and code

Session database

Design archive

OPTIONSSystem

Knowledge repository

Traceability

Visualization

Globus, Condor, OGSA

Ontology for Engineering,

Computation, &Optimisation and Design Search

CAD SystemCADDSIDEASProE

CATIA, ICAD

AnalysisCFDFEMCEM

ReliabilitySecurity

QoS

“Build complex

things from lots of simple

things”

© geodise project, university of southampton, 2004. geodise: grid-enabled toolkits for the...

Documents

globus gram job gd

globus proxy certificate

knowledge toolbox

grid applications

globus gram job manager

users certificate gd

job handle gd

compute job