Folie 1Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Scientific Data and Knowledge Management in Aerospace Engineering
Korea e-Science AHM 2008 (September 8th 2008, Daejeon)Andreas Schreiber <[email protected]>German Aerospace Center (DLR), Colognehttp://www.dlr.de/sc
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 2
Outline
German Aerospace Center (DLR)IntroductionExcurse: DLR CFD CodesGrid Computing: D-Grid and AeroGridSoftware Tools: DataFinder, RCE, and XPS4CFDConclusion
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 3
The DLRGerman Aerospace Research Center Space Agency of the Federal Republic of Germany
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 4
5,700 employees working in 28 research institutes and facilities
at 13 sites.
Offices in Brussels, Paris and Washington. Köln
Lampoldshausen
Stuttgart
Oberpfaffenhofen
Braunschweig
Göttingen
Berlin-
Bonn
Trauen
HamburgNeustrelitz
Weilheim
Bremen-
Sites and employees
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 5
IntroductionSimulation in Aerospace and Avionics
Designing new space and aerospace vehicles require high-resolution numerical simulation steps conducted in complex workflows
The complete simulation of all flow phenomena throughout the entire flight envelope including the multidisciplinary simulation of all involved disciplinesThe multidisciplinary optimization of the overall aircraft design as well as the design of major parts, such as the turbine engines
Involved disciplines:Aerodynamics – Structure – Heat – Flight mechanics – Radar & Infrared signature – Materials (physics/chemistry) – Combustion …
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 6
Software Technology for Aerospace SimulationsNumerical Codes and Supporting Tools
Highly sophisticated and optimized numerical simulation codesFor example, high-fidelity CFD codesMany codes available (free, commercial, proprietary) To major CFD codes of DLR
TAUTRACE
Simulation infrastructure and supporting toolsGrid environments Data and workflows management toolsKnowledge management ( documentation!)Preprocessing, post processing, visualization
IntegratedenvironmentsIntegratedenvironments
Folie 7Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
DLR CFD Codes
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 8
DLR TAU CodeFinite Volume CFD Solver
Reynolds-averaged Navier-Stokes (RANS) code Steady and unsteady flow, hybrid gridsState-of-the-Art turbulence modelsAdaptation module for local refinement Chimera methodCompletely parallelized (MPI)Used in European industry (Airbus, EADS)
Contact:DLR Institute of Aerodynamics and Flow Technologyhttp://www.dlr.de/as/case
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 9
DLR TRACE CodeTurbo machinery CFD
TRACE: Turbo machinery Research Aerodynamics Computational Environmentunsteady Reynolds-averaged Navier-Stokes (URANS) code Steady and unsteady flow (finite volumes)Hybrid multi block methodCompletely parallelized (MPI)Used in European industry (Siemens, MTU Aero Engines, Rolls Royce)
Contact:DLR Institute of Propulsion Technologyhttp://www.dlr.de/at
Folie 10Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Simulation InfrastructureGrid Computing in Germany – D-Grid
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 11
German D-Grid InitiativeGrid Infrastructure for e-Science
ObjectivesBuilding a Grid Infrastructure for science and business in GermanyCombine existing German grid activities Integration of middleware components developed in Community Grids
Three components of the infrastructureNetworks and associated toolsSoftware layer for connecting tools and servicesScientific information and knowledge
Important aspectsContinuing sustainable production grid infrastructure after the end of the funding periodIntegration of additional grid communities (2./3. generation) Business models for grid services
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 13
Ast
ro-G
rid
C3-
Grid
HEP
-Grid
In-G
rid
Med
iGrid
Text
Grid
WIS
ENT
Aer
oGrid
Bau
VOG
rid
Part
nerG
rid
● ● ●
D-Grid ProjectsCommunity Grid Projects
Integration Project
Grid Middleware und Grid Services
User Services (Portals)
Knowledge Management
Business Services, SLAs, SOA Integration, Virtualization
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 14
Nutzer
Applicationdevelopment
anduser access
GAT API
higherGrid
Functions
Baseservices
distributeddata archives
User/Application
Network-infrastructure
LCG/gLite
Globus
AccountingBilling
User/VO-Mngt
SchedulingWorkflow Management
Datenmanagement
Security
Plug-In
UNICORE
distributed compute
Resources
GridSphere
Monitoring
Data/Software
General D-Grid Architecture
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 15
Specifics of the D-Grid Architecture
D-Grid supports three middleware and two data access protocols.
Combines requirements of all communities.
Goal: Support of all middleware systems by all resource providers
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 16
D-Grid Infrastructure (2007)
Folie 17Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
AeroGrid Project
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 18
AeroGridProject Data
Grid-based cooperation between industry, research centres, and universities in aerospace engineering
Runtime: April 1, 2007 – March 30, 2010Website: http://www.aero-grid.de
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 21
AeroGridUse Cases and Project Goals
Usage scenariosUse of computing resources via the AeroGridCollaboration in designing engine components Co-operative further development of TRACE code
Project goalsAllow cooperation in research and development projectsUse of up-to-date program versions, data, and compute resources across all locationsDetailed documentation of history of a computational process that leads to a certain result (“Provenance”)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 22
UserUser
Service-Provider AService-Provider A
ResourcesResources
Data/Meta data
Tools
CPU Resources
WorkflowClient*
WorkflowClient*
DataManagement
Client
DataManagement
Client
WebPortalWeb
Portal
GridSphereServer
GridSphereServer
WebDAV/AFS/SRB/GridFTP/…
WebDAV/AFS/SRB/GridFTP/… U
NIC
OR
E6
UN
ICO
RE
6Tool
CodeDeveloper
CodeDeveloper
SimulationUser
SimulationUser
WorkflowService*
WorkflowService*
Service-Provider BService-Provider B
ResourcesResources
Data/Meta data
Tools
CPU Resources
GridSphereServer
GridSphereServer
WebDAV/AFS/SRB/GridFTP/…
WebDAV/AFS/SRB/GridFTP/… U
NIC
OR
E6
UN
ICO
RE
6
WorkflowService*
WorkflowService*
. . .
Grid Interfaces
* Workflow service and client are not part of the project.They will be added for later user communities.
Cooperation in AeroGrid
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 23
ProvenanceOrigin and Authenticity of Results
Provenance of computational processesRecorded documentation of processes as they take placeWith this documentation, we can determine:
the origin of electronic data andthe compliance of the process that led to the data
Definition of provenanceThe Provenance of some information is the history of its creation
Related Terms“Data Lineage” is another word for ProvenanceProvenance is NOT “Logging”
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 24
Provenance Example Queries
Given some data item, what was the simulation case?
Given some parameter, in what simulation(s) has it been used?
What data has been recorded in a simulation with a specific parameter?
What simulations have been run using a given model (aircraft design)?
Given two/more simulations with the same setup, what is the result and the difference in provenance?
What have been the initial conditions for this simulation?
What were the termination criteria?
How often has the workflow been executed?
What specific algorithm/version has been used in computations?
What were the hosts participating in the simulation run?
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 25
ProvenanceBenefits for Aerospace Applications
Clean documentation of distributed simulation workflows
Ability to analyze and reason over all conducted computationsUnderstanding of computations and their resultsTool based support for analyses
Allows checking for requirementsCompliance to legal and business regulations
Proof and reproducibility of results
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 27
Interactions:
Configuration
Process flow
Monitoring
Data management
ExampleProvenance Information in Simulations
Processcontrol File-Server
Pre-Processing
Parametervariation
Simulation Visualization
i0
i1 i2
i3
i4
m1
d1
d2
c1 c-1 c2 c-2 c3 c-3
c4c-4
Relationen:- r0: i0 causes i1- r1: i1 causes i2- r2: i2 causes i3- r3: i2 causes i4- r4: i3 causes i2- r5: i2 causes m1- r6: i2 causes d1- r7: i0 causes d2
Stateof the Actors
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 28
AeroGrid User Interfaces
PortalWeb-basaed accessDevlopment based on GridSphere
Client applicationsAutomation of recurring tasksIntegration in existing working environments
Folie 29Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Data Management Client SoftwareDataFinder
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 30
DataFinderShort Overview
DataFinderEfficient management of scientific and technical dataFocus on huge data sets
Development of the DataFinder by DLRAvailable as Open-Source-Software
Primary functionalityStructuring of data through assignment of meta information and self-defined data modelsComplex search mechanism to find dataFlexible usage of heterogeneous storage resourcesIntegration in the working environment
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 31
DataFinderUses
Large-scale simulationsaerodynamicsmaterial scienceclimate…
Measured datawind-tunnel experimentsearth observationstraffic data…
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 32
DataFinder OverviewBasic Concept
Client-Server solutionBased on open and stable standards, such as XML and WebDAVExtensive use of standard software components (open source / commercial), limited own development at client side
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 34External Medias
(CD, DVD,…)
DataFinderMass Data Storage using “Data Stores“
Logical View User ClientStorage
Locations“Data Stores”
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 36
Data StructureMapping of Organizational Data Structures
User
Project A
Project B
Project C
File 1
File 2
Simulation I
Experiment
Simulation IIProject MegaCode UltraUser EddieKey Value
Object(collection)
Object(file)
Relation Project MegaCode UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey ValueProject Mega
Code UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey Value
Project MegaCode UltraUser EddieKey Value
Attributes(meta data)
Folie 38Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
DataFinder in AeroGridDataFinder in AeroGridTurbine SimulationTurbine Simulation
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 39
Simulation steps (example):1. splitCGNS
Preparing data for TRACE2. TRACE (CFD solver)
Main computation3. fillCGNS
Conflating results4. Post Processing
Data reduction and visualization
Automation with customized DataFinder
Turbine SimulationData Model
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 40
Turbine Simulation: Graphical User Interface
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 41
Turbine Simulation: Customized GUI Extensions11
22
3
4
55
1.1. Create new simulationCreate new simulation2.2. Start a simulation Start a simulation 3.3. Query statusQuery status4.4. Cancel simulationCancel simulation5.5. Project overviewProject overview
Customization basedon user requirements!Customization basedon user requirements!
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 42
Turbine Simulation Starting External Applications
1. CGNS Infos / ADFview / CGNS Plot2. TRACE GUI3. Gnuplot
1
2
3
Integration with other tools!Integration with other tools!
Folie 43Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Integration Platform and Workflow ManagementReconfigurable Computing Environment (RCE)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 44
Beyond DataFinder…
Use cases for “lightweight” tools like DataFinderSimple recurring tasksEasy and fast customization (but “static” workflows!)Instant access to distributed storage resourcesEasy installation
Additional complexity requires more sophisticated systemsDynamic (large) workflowsComplex configurable user interfaceUse of existing services
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 45
Reconfigurable Computing Environment (RCE)
The RCE general purpose integration platformUsed to manage collaborative engineering processesProvision of commonly used software components
Data management, graphical user interface, distributed computing, workflow component
Concentration on the application specific software components
Developed as a service oriented software frameworkUsing OSGi as service platform (for Java)Eclipse for graphical user interface
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 46
Java Virtual Machine
OSGi/Eclipse Equinox
App
licat
ion
RCE Architecture
Reconfigurable Computing Environment
App
licat
ion
Serv
er
Config.
Wra
pper
App
licat
ion
GU
I
Ext. Code
…
Dat
a
Notification
Priv
ilege
Bro
ker
Com
mun
icat
ion
RMI
SOAP
…
Macro
…
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 51
Java Virtual MachineOSGi/Eclipse Equinox
RCE
Wra
pper
Sim
ulat
ion
Appl
ic. S
erve
r
Dat
a
Priv
ilege
Bro
ker
Com
mun
icat
ion
RMISOAP
…
Ext. Code
RCEDistributed System
Java Virtual MachineOSGi/Eclipse Equinox
RCE
Appl
ic. S
erve
r
GU
I
Dat
a
Priv
ilege
Bro
ker
Com
mun
icat
ion
RMISOAP
…
Java Virtual MachineOSGi/Eclipse Equinox
App
licat
ion
RCE
Appl
ic. S
erve
r
Dat
a
Priv
ilege
Bro
ker
Com
mun
icat
ion
RMISOAP
…
DB Server
GUI
Simulation
Desktop
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 52
GUI ExampleShipbuilding Industry
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 54
Use Cases at DLRRCE as integration platform
IMENS (Multi disciplinary design of space re-entry vehicles)Simulation of hot structuresRCE-based environment for coupled,distributed simulation in Grids
CEF (Concurrent Engineering Facility)Collaborative early design of space vehiclesMission planningExperts for all disciplines in one roomRCE-based software infrastructureReuse of existing Excel-spreadsheets
XPS4CFD (Expert System for CFD)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 55
Integration with Microsoft Excel
If you cannot avoid Excel:Solution 1: Use Excel, integrate interfaces to computational resources into ExcelSolution 2: Integrate Excel into execution environment and workflow system
RCE-Excel-IntegrationExcel user interface integratedinto RCE/EclipseExcel files stored in RCE datamanagementSharing data with other RCEcomponents
Excel is nice! It's ugly to use, but nice. And in the end all data ends in Excel.
Hans Rosling
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 56
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 57
Folie 58Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Expert System for Aerospace EngineersXPS4CFD
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 60
XPS4CFD (1)Expert System for Aerospace Engineers
Assistance of users of DLRs CFD-software TAUProvide best-practices and guidelines, depending on specific problems and factsRule-based system using JBoss Drools as the rule engineEclipse Rich Client Platform application, based on RCE
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 61
XPS4CFD (2)Expert System for Aerospace Engineers
Combined with a Lucene search engine for finding relevant documentsEasily maintainable and extensible by experts, not programmers
Forms and graphical editors for adding, changing and deleting rules, workflows and documentsGenerators for automatic creation of the fact model and UI elements like interview-components
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 62
Choosing and Planning a Scenario
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 63
Guiding Users Through a TAU Workflow (1)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 64
Guiding Users Through a TAU Workflow (2)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 65
Guiding Users Through a TAU Workflow (3)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 66
Creating Rules
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 67
Creating Forms
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 68
Creating Workflows
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 69
Expert SystemAdditional Use Cases (related to space/aerospace)
Health monitoringMonitoring health status of astronauts and pilotsManaging knowledge about irregular health parametersSend alarms
Astronaut assistanceAssist astronauts in standard procedures and problem situationsConfigurable for all subsystems of space crafts
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 70
ConclusionPutting All Together…
UserShares KnowledgeUses KnowledgeWorks with dataSearch meta dataSelect resources
Software toolsGenerate workflow descriptionExecutes workflows in GridsRecord Provenance info
ProvenanceStore
generate workflow
execute workflow
feedback analysis
trace workflowexecution
trace user action
ExpertSystemExpertSystem
Data &Workflow
Management
Data &Workflow
Management
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
Folie 71
Questions?Questions?