caa/cfa review | andrea laruelo | estec | may 19 2011 cfa development status caa/cfa review estec,...

CAA/CFA Review | Andrea Laruelo | ESTEC | May 19 2011

CFA Development Status

CAA/CFA Review

ESTEC, May 19th 2011

European Space AgencyAndrea Laruelo


Development Schedule-Creating Basic Infrastructure-


Understanding the project


CFA Entity ModelUML Diagram From v0.1 to v0.7

http://esavo02.esac.esa.int/trac/CFA/attachment/wiki/UML/CFA_uml_v0_7.png

Diagram that shows the relationship among the entities (and their attributes) for which metadata will be stored in the database.

Drawn using UML class diagram nomenclature

Basis for the Entity-Relationship (E-R) database model diagram among other things.

http://esavo02.esac.esa.int/trac/CFA/raw-attachment/wiki/UML/CFA_uml_v0_7.png


Hardware

Two SUN servers running Linux which would offer redundancy and load balancing

Special attention will have to be given to the disk infrastructure for the CFA

Start planning for 100TB, but with the capacity to eventually increase it up to 200TB

100 TB of disk storage

Model: NetApp FAS 3270

Status: In purchase process

Currently: Development Virtual machine


Free/open source version control system.

Manages files and directories, and the changes made to them, over time.

Allows to recover older versions and examine the history of how data has changed.

Operates across networks, which allows it to be used by people on different computers.

CFA Version Control System-Subversion-


CFA Tracking System -Track management tool-

Open source project management tool

Enhanced wiki and issue tracking system for software development projects.

It provides an interface to Subversion

Seamless references between bugs, tasks, changesets, files and wiki pages.

A timeline shows all current and past project events in order, making the acquisition of an overview of the project and tracking progress very easy.

Roadmap listing the upcoming milestones.

http://esavo02.esac.esa.int/trac/CFA/

Ticketing system during development phase

Relevant information

Related documentation

Access to svn code


CAA-CFA ICD-Version 0.3-

Defines the interfaces between the CAA and the SAT/CFA

Requirements pertaining to the transfer of data and metadata.

Two different phases:

- Initial synchronization of CAA and CFA repositories

- Regular transfer of new products

CAA File System

CAA CFA

Ingestion Request File

Data transfer

Ingestion Request Error File

IngestionSystem

CFA DB

ingest

CFA File System

import


Migration Plan-Released v1.0-

CAA Public Access

2011

2012

2013

2014

2015

2016

TCFAB

TCFAPB

R

TCAAE

TDATAE

CAA data ingestion only

CFA BETA testingCFA Public Beta Access

CAA hibernation

CFA Public Access Only

TCFAPR

Provides an overview of the necessary steps to be performed for the CAA to CFA migration.

Overall schedule for the transition.


CFA Database -Design and installation-

Scientific data model v0.7

PostgreSQL Three different databases schemas:

Scientific schema: It needs to be consistent with the entity model

Controls schema: Used to log the daily actions performed with the archive.

Ingestion schema: Used to log the ingestion process.


Postgres DBPostgres DB

FTP ServerFTP Server

CFA serverCFA server

UsersUsers

TracTrac

SVNSVN

CFA Development Machine

Apache

Web server

http://subversion.apache.org/


Development Schedule-Archive Subsystems development-


Status of archive subsystems development


Storage Layer

Test data ingested


Data Layer Implementation

Stand-alone application

The only entry point to the database

UML Diagram


Entity Model Implementation

Diagram that shows the relationship among the entities (and their attributes) for which metadata will be stored in the database.

Create the entities by extending ABSI AbstractEntity class.The entity name is the same one used in the entity model diagram and the attributes are those inside the entity itself

Create the model class (ABCModel), which extends AbstractControlsModel

Create the transfer objects This module contains the POJOs (Plain Old Java Objects) that will carry the metadata across all layers of the archive when the system is running.

Implementation


Server Layer

The server layer is the hub for the requests made to the archive.

It basically decides what to do when a new request is performed

Calls the backend modules (Query Manager and Data Distribution sub-systems) through appropriate protocols (basically RMI)

This module implements the Command design pattern


User Interface

GUI to access current CFA prototype

User registration

Basic queries beased on:

Experiment

Measurement Type

Instrument Type

Time

Implements the Model View Controller (MVC) design pattern

Follows the absi ui frame

Common layout to all Scientific Archives

Uses infonode and jgoodies as third-party libraries


Development Schedule-Future Activities. Adding functionality-


Questions ?→

Suggestions

caa/cfa review | andrea laruelo | estec | may 19 2011 cfa development status caa/cfa review estec,...

Documents

cfa migration

cfa database design

cfa entity modeluml

caacfa icdversion

software development

issue tracking system

entity model controls

scientific schema