Ocean Observatories Initiative

OOI Cyberinfrastructure

Architecture OverviewMichael Meisinger

September 29, 2009

Overview

• Integrated Observatory Network

• Deployment Strategy: Capability Container

• Integration Strategy: Reliable Messaging

• Service Architecture and Subsystems

• Scope of Release 1

2

Integrated Observatory Functions

• Interactive Ocean Observing• Interactive Ocean Modeling & Data Assimilation• Automated Data Product Generation• Discipline-Driven Semantic Organization of Data• Interactive Instrument Network• Integrated Observatory Management• User-Driven Integration of Resource

ObservatoryActivity Model

Open System

Integrated Observatory Users• Scientists

– Principal Investigator– Project Scientist – Support Scientist– Graduate Student

• Engineers– Project Engineer – Instrument Provider – Instrument Technician – Application Developer– Mission safety and security

• Data Professionals– Data Analyst– Data Modeler– Archivist/Curator

• Operations– Program Manager– Observatory Manager– Logistic Coordinator– Mission Planner– Scheduler

• Educators– Developer of Educational

Materials– Teacher– Librarian

• General Public– Outreach material

providers– Science Public– Everyone else

[Persona Whitepaper]

Integrated Observatory Extent

SensorPlatforms

PhysicalInterface

MarineOperations

Shore SideOperations

End UserOperations

MobileCore

FixedCore

ResearchPlatforms

AdministratorCyber OperatorMarine Operator

User

Marine

Cyber

Operational Authority

ObservatoryOperations

Science Teams & Educators

ObservatoryManagement

MarineManagement

User EnvironmentManagement

Marine Net Observatory Net

Capability Container

Infrastructure

Data Sets

Observation Plans

Processes

Instruments

RecoveryPolicies

Execution

Messaging

Management

Agreements

Models

Provision

Routing

Monitoring

Education & Participation

Community Collaboration

OrchestrationGovernance

State Mediation

Activities Resources

Knowledge

Interfaces

Repository

Catalog

Resources Interactions Communications Processing

ScientificInvestigation

CyberInfrastructure Capability Container

CI Interfaces and Subsystems

Interactive Observatory

Operating System with Core

Applications

Deployment Scenario

8

Execution Environments• Strategy

– One software development and one operational environment across all execution environments: Capability Container

– Specific profile and adapter per execution env.• Marine Observatory Infrastructure

– Mooring DCL/CPM embedded CPUs– Mobile asset (AUV, Glider, Profiler) embedded

• Terrestrial Infrastructure– Local clusters– National infrastructure (TeraGrid, OSG)– Commercial Cloud Infrastructure

9

Network Architecture

11

Network Deployment

Integration Infrastructure• The Common Operating Infrastructure (COI)

– Distributed, secure, reliable, service-oriented, message-based– Integration and communication infrastructure

• The Common Execution Infrastructure (CEI) – virtualization and provisioning of computational and storage resources

Common Operating Infrastructure

Data Management

(Science)

Sensing & Acquisition

Data Management (Information Distribution)

Analysis & Synthesis

Identity Management

State Management

GovernanceFramework

Resource Management

Planning & Prosecution

Exchange

Service Framework

Presentation Framework

Common Execution

Infrastructure

Scope of Release 1• Common Operating Infrastructure

– Messaging Service in Federated Facility– Identity and Policy Management– Distributed Service Application Framework

• Common Execution Infrastructure– Management of Executable Processes– Elastic Computing– Resource Management

• Data Management– Common Data and Metadata Model– Dynamic Data Distribution and Data Catalog– Persistent Archive Framework

• Sensing and Acquisition– Instrument Direct Access– Data Acquisition– Instrument Management and Agent Framework

13

Release 1: Data

Distribution Network

Data Collection Workflow

Deployment Pattern

15

OOI CI Kick-Off MeetingSept 9-11, 2009

Subsystem Dependencies

16

OOI CI Kick-Off MeetingSept 9-11, 2009

Application Services

17

OOI CI Kick-Off MeetingSept 9-11, 2009

Infrastructure Services

18

OOI CI Kick-Off MeetingSept 9-11, 2009

Development Dependencies

19

Technology Readiness LevelTRL Definition Description

1 Basic principles observed and reported.

Lowest level of technology readiness.  Scientific research begins to be translated into applied research and development.  Examples might include paper studies of a technology's basic properties.

2 Technology concept and/or application formulated.

Invention begins.  Once basic principles are observed, practical applications can be invented.  Applications are speculative and there may be no proof or detailed analysis to support the assumptions.  Examples are limited to analytic studies.

3 Analytical and experimental critical function and/or characteristic proof of concept.

Active research and development is initiated.  This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology.  Examples include components that are not yet integrated or representative.

4 Component and/or breadboard validation in laboratory environment.

Basic technological components are integrated to establish that they will work together.  This is relatively "low fidelity" compared to the eventual system.  Examples include integration of "ad hoc" hardware in the laboratory.

5 Component and/or breadboard validation in relevant environment.

Fidelity of breadboard technology increases significantly.  The basic technological components are integrated with reasonably realistic supporting elements so it can be tested in a simulated environment.  Examples include "high fidelity" laboratory integration of components.

6 System/subsystem model or prototype demonstration in a relevant environment.

Representative model or prototype system, which is well beyond that of TRL 5, is tested in a relevant environment.  Represents a major step up in a technology's demonstrated readiness.  Examples include testing a prototype in a high-fidelity laboratory environment or in simulated operational environment.

7 System prototype demonstration in an operational environment.

Prototype near, or at, planned operational system.  Represents a major step up from TRL 6, requiring demonstration of an actual system prototype in an operational environment such as an aircraft, vehicle, or space.  Examples include testing the prototype in a test bed aircraft.

8 Actual system completed and qualified through test and demonstration.

Technology has been proven to work in its final form and under expected conditions.  In almost all cases, this TRL represents the end of true system development.  Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.

9 Actual system proven through successful mission operations.

Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation.  Examples include using the system under operational mission conditions.

Technology CatalogSubsystem Functional Component Technology TRLAnalysis & Synthesis Analysis Application/Language Matlab 9Analysis & Synthesis Workflow Execution Engine Kepler 6Analysis & Synthesis Meta-Workflow Resource Mapper Pegasus 9Analysis & Synthesis Visualization Toolkit VTK 9Analysis & Synthesis Graphics Engine OpenSceneGraph 9Analysis & Synthesis Visualization Application IDV 9Analysis & Synthesis Spatial Display Application OSSIM Planet 7Analysis & Synthesis Spatial Display Application GoogleEarth & Maps 9Data Management Dataset Catalog THREDDS 7Data Management Semantic Framework VSTO Semantic Framework 8Data Management Semantic Query ESG Facetted Search 8Data Management Data Integration with Applications NetCDF lib 8Data Management Data Integration with Applications Matlab lib for OpenDAP access 8Data Management Dataset Management & Distribution OpenDAP Hyrax Server 7Data Management Dataset Preservation iRODS 7Data Management General Purpose Database MySQL cluster 9Planning & Prosecution Resource Planner ASPEN 9Planning & Prosecution Controller CASPER 9Planning & Prosecution Autonomous Comm Middleware MOOS 8Planning & Prosecution Autonomous Helm MOOS-IvP Helm 8Sensing & Acquisition Virtual Resource Framework IaaS 6Sensing & Acquisition Instrument Drivers Antelope ORB 9Sensing & Acquisition Instrument Drivers SIAM 9Sensing & Acquisition Instrument Drivers PUCK 8Sensing & Acquisition Resource Monitoring Application Intermapper 9CEI Grid Scheduler Condor 9CEI Cloud Computing Environment Amazon EC2 9CEI Cloud Computing Environment Nimbus 8CEI VM Packaging & Distribution Elastic Server (CohesiveFT) 8CEI Virtualization Layer Software Individual virtual-ization vendors, e.g., Amazon AMI)9COI Message Broker System RabbitMQ 8COI IOC Container Spring 9COI Enterprise Service Bus (ESB) MULE 8COI Identity Management & Federation COmanage 7COI Identity Management & Federation Shibboleth 9COI Identity Management & Federation GridShib 7COI IdM, Governance Globus Toolkit 9COI Identity Management & Federation MyProxy 9COI Identity Management & Federation ProtectNetwork 9COI Policy Management & Governance Sun XACML Implementation

Technology Catalog

Subsystem Functional Component Technology TRLAnalysis & Synthesis Analysis Application/Language Matlab 9Analysis & Synthesis Workflow Execution Engine Kepler 6Analysis & Synthesis Meta-Workflow Resource Mapper Pegasus 9Analysis & Synthesis Visualization Toolkit VTK 9Analysis & Synthesis Graphics Engine OpenSceneGraph 9Analysis & Synthesis Visualization Application IDV 9Analysis & Synthesis Spatial Display Application OSSIM Planet 7Analysis & Synthesis Spatial Display Application GoogleEarth & Maps 9Data Management Dataset Catalog THREDDS 7Data Management Semantic Framework VSTO Semantic Framework 8Data Management Semantic Query ESG Facetted Search 8Data Management Data Integration with Applications NetCDF lib 8Data Management Data Integration with Applications Matlab lib for OpenDAP access 8Data Management Dataset Management & Distribution OpenDAP Hyrax Server 7Data Management Dataset Preservation iRODS 7Data Management General Purpose Database MySQL cluster 9Planning & Prosecution Resource Planner ASPEN 9Planning & Prosecution Controller CASPER 9Planning & Prosecution Autonomous Comm Middleware MOOS 8Planning & Prosecution Autonomous Helm MOOS-IvP Helm 8Sensing & Acquisition Virtual Resource Framework IaaS 6Sensing & Acquisition Instrument Drivers Antelope ORB 9Sensing & Acquisition Instrument Drivers SIAM 9Sensing & Acquisition Instrument Drivers PUCK 8Sensing & Acquisition Resource Monitoring Application Intermapper 9CEI Grid Scheduler Condor 9CEI Cloud Computing Environment Amazon EC2 9CEI Cloud Computing Environment Nimbus 8CEI VM Packaging & Distribution Elastic Server (CohesiveFT) 8CEI Virtualization Layer Software Individual virtual-ization vendors, e.g., Amazon AMI)9COI Message Broker System RabbitMQ 8COI IOC Container Spring 9COI Enterprise Service Bus (ESB) MULE 8COI Identity Management & Federation COmanage 7COI Identity Management & Federation Shibboleth 9COI Identity Management & Federation GridShib 7COI IdM, Governance Globus Toolkit 9COI Identity Management & Federation MyProxy 9COI Identity Management & Federation ProtectNetwork 9COI Policy Management & Governance Sun XACML Implementation

23

Thanks !

Subsystem

• Marine Observatory Facility – Observatory Management– Instrument Management & Instrument Activation Services– Marine Resource Scheduling for Power, Bandwidth, etc.– Instrument Resource Repository

• Marine Infrastructure Interface– Instrument and Platform Agents– Instrument Direct Access

• Data Acquisition and Processing– Raw Data Acquisition, Buffering, Metadata association and Persistence– Data Processing, QA/QC, Calibration & Validation– Data Product Generation, Publication & Persistence– Data Process Repository – Data Product Catalog & Repository

Sensing & AcquisitionSensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Subsystem

• Data and Information Access– Search & Navigation– External observatory access (IOOS, Neptune Canada, …)

• Transformation and Mediation– Attribution & Association– Aggregation– Syntactical Transformation– Ontology-based mediation between vocabularies

• Dynamic Data/Information Distribution– Persistent Archive– Information Catalog & Repository

Sensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Data Management

Subsystem

• Laboratory & Classroom Facility– Interactive Analysis and Visualization– Workflow Management– Event Detection

• Numerical Model Integration– Data Assimilation– Model Development, Testing & Validation– Model Parameterization & Execution– Model Catalog & Repository

Sensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Analysis & Synthesis

Subsystem

• Interactive Observatory Facility– Observation Planning– Mission Coordination and Asset Safety

Protection– Mission Catalog & Repository– Mission Simulator Framework

• Event Response Services

• Adaptive Observations

• Portable Control Software

Sensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Planning & Prosecution

Subsystem

• Computing Management– Resource Provisioning and Management– Execution Engine Catalog & Repository– Process Management– Process Catalog & Repository

• Elastic Computing

• Integration with National Computing Infrastructure

Sensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Common Execution Infrastructure

Subsystem

• Operating and Integration Framework– Communication Services– Service Framework– Presentation Framework

• Resource Lifecycle Management

• Security and Policy Management– Identity Management & Authentication – Policy Authoring, Management & Governance

• Facility Services

Sensing & Acquisition

Data Management

Planning & Prosecution

Analysis & Synthesis

Common Execution Infrastructure

Common Operating Infrastructure

Capability Container

Common Operating Infrastructure

Integration StrategyThe CI’s Common Operating Infrastructure mitigates technology risk, maintainability and scalability issues through an integration strategy for application and resource integration:

1. Service-oriented, message-based integration infrastructure • Reduces inter-dependencies between technologies

2. Virtualization of computing and storage• Enables scalability meeting demand

3. Flexible and dynamic packaging and deployment• Enables CI deployment where needed

4. Multi-facility concept based on agreements• Enables new facilities to join the OOI network