the eumetsat satellite application facility on nowcasting and very short range forecasting

1st IPWG Workshop. Madrid 23-27 Sep 2002

The EUMETSAT SAF on Nowcasting & VSRF

1

The EUMETSAT Satellite Application Facility on

Nowcasting and Very Short Range Forecasting

1st International Precipitation Working Group WorkshopMadrid, 23-27 September 2002

L.F.López Cotín – IOP NWC SAF Manager (INM)



2

BACKGROUND

INITIAL OPERATIONS PHASE: Plans & Services

NWC SAF ConsortiumGeneral ObjectivesPhases

IOP NWC SAF ObjectivesPlaned ActivitiesPrimary ServicesSoftware Upgrades Support Services

DEVELOPMENT PHASE OUTPUTS

ProductsOutputs



3

The NWC SAF Consortium

RDT: MF reassignement

•INM:Leading Institute(Spain)

•Météo-France(France)

•SMHI(Sweden)

•ZAMG (Austria)



4

NWC SAF General Objectives

Provide tools for the operational services to ensure the optimum use of meteorological satellite data on support to Nowcasting and Very Short Range Forecasting activities

Development, distribution and maintenance of appropriate SW Packages as well as of all related tasks for user's support

Centre of Excellence for Nowcasting in EUMETSAT

Type A SAF

Both MSG & PPS

Products will be produced by each SAFNWC User at home basis, running the software packages delivered and

supported by the Nowcasting SAF



5

Development Phase

IOP

FOP

Feb 1997

Feb2002

Feb2007

The NWC SAF Phases

6 months Potential extension



6

Development Phase: Products

• Cloud Mask

• Cloud Type

• Cloud Top Temperature & Height

•Precipitating Clouds

•Convective Rainfall Rate

•Total Precipitable Water

• Layer Precipitable Water

• Stability Air Imagery

• High Resolution Wind

• Automated Satellite Interpretation Imagery

• Rapid Developmet Thunderstorms

• Air Mass Analysis• Cloud Mask

• Cloud Type

• Cloud Top Temperature/Height

•Precipitating Clouds

MSG Products

PPS Products



7

• Two independent Packages:

SAFNWC/MSG: integrated by

SAFNWC/PPS : integrated by

NWC SAF Packages Strategy



8

Steering Group

IOP Manager [INM-LE]

System EngineeringIntegration and Coherence [INM-LE]

MSG Manager [INM-LE] PPS Manager [SMHI]

Scientific Team

- PPS Delta Develp.- Scientific Activities

EUMETSAT OPS SAFNWC Users' Group

MSG OPSService [INM-LE]

PPS OPSService [SMHI]

INM Sc.Development

MF Develop+SW Maint.

SMHI MSG Dev+Maint

ZAMG Dev+Maint

The IOP NWC SAF Consortium Structure

Coherence &

Synergy

Coherence &

Synergy



9

Products: Clouds Products

Provide information on the possible occurrence of clouds within each pixel. The central aim is to delineate all absolutely cloud-free pixels in a satellite scene with a high degree of confidence

•Cloud Mask (Cma)

•Cloud Type (CT)

•Cloud Top Temperature & Height (CTTH)

The main objective of this product is to support detailed cloud analysis

Contains information on the cloud top temperature and height for all pixels identified as cloudy in the satellite scene with the highest possible spatial and temporal resolution



10

Products: Precipitation Products

•Precipitating Clouds (Pc)

•Convective Rainfall Rate (CRR)

Provides information on the probability of weak, moderate and strong precipitation

Provides the maximum level of information on convective rainfall from the SEVIRI channels



11

Products: Precipitation Clouds PC

Based on SMHI’s long experience on precipitating cloud products

As output of the SCANDIA cloud classification scheme

Stadistical scheme making use of IR/VIS channel and forcing parameters from the NWP data

Channels used: VIS 0.6, VIS 0.8, NIR 1.6, IR 3.9, IR 10.8, IR 1.0

Scientific Report on www.inm.es/wwg and www.smhi.se

http://www.inm.es/wwg







12

Products: Convective Rainfall Rate CRR

Based on RAINSAT methodology statistical relationship between satellite (multi-band imagery) and rainfall rate data obtained using radar data.

CRR value for each pixel is obtained from calibration matrices.

Calibration matrices are different if a pixel belongs to day or night mask.

Day pixels CRR calibration data is a matrix 3-D (bands: IR10.8, WV6.2, VIS0.6.)

Night pixels calibration values are stored in a 2-D matrix (bands: IR10.8, WV6.2.)

Normalised visible counts are obtained using Binder algorithm.

Smoothing process performed to eliminate stratiform rain data -not associated with convective clouds.

The obtained CRR significant data are set to zero if all the nearest pixels in a box of 14x14 do not have an equal or higher value than 3mm/h.



13

Products: Convective Rainfall Rate CRR

The “RAINSAT” output is used as the input to the Auto-estimator algorithm proccess,

This included adjusdtements by

-humidity: TPW x HR

-Orography

-Clod top area evolution

-Cloud top gradient

-Etc



14

Products: Clear Air Mass Products

•Total Precipitable Water (TPW)

•Layer Precipitable Water (LPW)

•Stability Air Imagery

•Air Mass Analysis

Total atmospheric water vapour in a column of air extending from the Earth's surface to the "top" of the atmosphere

Atmospheric water vapour contained in a vertical column of unit cross-sectional area in three layers in the troposphere

LI index summarising the information of the SEVIRI channels

Evaluates basic quantities that describe air masses (upper and middle level humidity, mean temperature, atmospheric stability, cloud pattern, etc) and to combine them into one integrated classification of the air mass



15

Products: Winds and Conceptual Model Products

•High Resolution Wind (HRW)

•Rapid Development Thunderstorm (RDT)

•Automated Satellite Interpretation Imagery (ASII )

Provides information on mesoscale wind vectors using data from the HRVIS channel at two spatial resolution: 25 and 10 km.

Twofold objectives: identification, monitoring and tracking of intense convective systems, and detection of rapidly developing convective cells

Provides an automatic interpretation of features seen on satellite images. Hence, the product identifies fronts, wave structures, areas of intensification at fronts by jet streak crossing, position of the jet axis, comma clouds, enhanced convection areas, etc.



16

Products: Products Dependencies



17

• All (MSG/PPS) modules shall be integrated in a single SW (MSG/PPS) applications

•The (MSG/PPS) applications shall be fully configurable

–Products to be generated

–Geographical Area / Full Frame

–Execution priorities (PGE / regions)

•The (MSG/PPS) packages shall be robust

–Execution will not need human assistance

–Schedule of tasks automatically fixed by the Task Manager

–Information will be logged to monitor the processing and to extract statistics

•The (MSG/PPS) applications shall permit the processing of real-time and archived data

Main SW Requirements



18

Architecture Design

Meteorological Product and Engineering SW development shall be as de-coupled as possible:

Task Manager

NWCLIB (Common functions)

PGEs (Product Generation Elements)



19

Task Manager• Interface between the user and the SAFNWC/MSG

product generation• Initialise and set-up of the system according the

configuration• Drive and optimise the execution of the PGEs

– Execute pre-processing tasks required by the PGEs– Schedule the execution of the PGEs according their

dependencies and priorities– Monitor the processing and logs information and

statistics– Execute other programmable activities defined by

the user



20

NWCLIB• The NWCLIB common library provides common

functionalities for the SAFNWC/MSG product generation

SEVEXT Management o f SEVIRI 1.5 input files

Data extraction Calibration

NWPMAP Management of GRIB input files (NWP) (helped by the EMOS library)

Data decoding and reading Re-mapping to SEVIRI projection

PRODIO Management of SAFNWC/MSG output files

Read Write

LIBNWC

Processing specific functions Region setting Navigation routines Sun and Satellite Angles Time conversion routines Log and Error reporting Other

RTTOV Fast Radiative Transfer Model (external to the SAFNWC. Used at it is)



21

PGEs• Implements the scientific algorithms in charge of

extracting the Meteorological Products• Coded as stand-alone applications, supported by the

NWCLIB functions• Fully configurable

– Region– Parameters used by the algorithms

• Common External InterfacePGEnn <YYYYMMDDhhmm> <region conf. file> <model conf. file>nn: PGE numberYYYYMMDDhhmm: nominal date of the SEVIRI slot to be processed<region conf. file> Name of the Region Configuration File<model conf. file> Name of the Model Configuration File (model parameters)



22

Development Phase: MSG Outputs

SAFNWC / MSG version 0.0

• Task Manager: TM

• Common Library: NWCLIB

• Product Generation Elements: PGE01, ... PGE12

SW

Documentation

Scientific: Plans and Reports on prototypes

SW Engineering: SRD, ADD (System and PGE level), SVD, Tests Reports, Test Results.

Users Manuals: Master, Engineering, Scientific parts



23

MSG Outputs - characteristics

SAFNWC / MSG version 0.0

MSG Launch delay impact

Engineering purposes

Supported on SUN

Current committed Platforms: SUN, HP, DEC-Alpha, SGI

Portability tasks on stable version 1.0 (validated with SEVIRI data)



24

Development Phase: PPS Outputs

SAFNWC / PPS

• Product Generation Elements: PGE01, ... PGE04

• SMHI specific library functions and scripting environment to run PGE’s

SW

Documentation

Scientific: Plans and Reports on prototypes

SW Engineering: SRD, ADD, ICD’s (PGE level), SUM



25

PPS Outputs - characteristics

•NOAA satellite data was available during Development phase

•Resources for software package development (common library functions and task manager) made available for IOP

•PPS PGE’s scientifically mature, no initial tuning needed

•Software engineering at different stages than for MSG

• Different content for early PPS / MSG release version numbers



26

INITIAL OPERATIONS PHASE: Objectives 1/3

Complete the development of the MSG products, up to their validation and tuning using real SEVIRI data, as available from the MSG-1 mission;

Validate the SAFNWC/MSG products with real SEVIRI data;

Improve the SAFNWC/MSG products and plans according to the results of the validation activities;



27


Complete the engineering of the SAFNWC/PPS package: common SW developments and integration into a single package;

Adjust PPS software and algorithms to be able to process EPS/MHS data when MetOp is available

Validate the SAFNWC/PPS products.

Improve the SAFNWC/PPS products and plans according to the results of the validation activities;



28


Deliver updated SW Packages for SAFNWC/MSG and SAFNWC/PPS during IOP as necessary to support maintenance and planned improvements;

Give support to users by means of a Help Desk (web site) and training activities;

Perform Quality Control on the Nowcasting SAF Production and Services;

Collect new User Requirements for further developments;

Implement new developments according to the upgraded User Requirement Document; and

Be a reference for Nowcasting aspects for EUMETSAT.



29

• No homogeneous activities planned along the IOP Life Cycle; therefore sub-phases are identified with different durations and specific objectives

• Sub-phase Outputs are upgraded SW versions

• Management Control of Services and Progress of the project are ensured by means of Technical Revisions (upgraded SW deliveries)and Management Reviews (yearly)

INITIAL OPERATIONS PHASE: Planned Activities



30

The Nowcasting SAF will deliver upgraded versions and related documentation

INITIAL OPERATIONS PHASE: Primary Services

Software Maintenance:

Corrective Adaptive Perfective Evolutive



31

INITIAL OPERATIONS PHASE:Algorithm Improvements

Associated to Perfective and Evolutive SW maintenance concept

Based on current products improvements or new products developments

Performed by the Development Teams as planned, and considering users’ inputs

Supported through the Visiting Scientist Programme



32

Development +

Preparations

Scientific validation +

Requirements SeekingUpgrading According

New Requirements

Additional validation+

FOP preparations

March2002

End2003

March2005

July2006

Feb2007

SEVIRI EPS

MSG 1.2PPS 1.1 MSG 2.0

PPS 2.0MSG 2.1PPS 2.1

MSG 1.1

INITIAL OPERATIONS PHASE: Software Upgrades

Mid2003

MSG 1.0

MSG 0.1PPS 1.0



33

SAFNWC/MSG 0.1: System Validated (Engineering)

Upgraded according planned scientific developments and system validation

Real SEVIRI data

Feedback from Beta-testers

Planned Scientific Outputs

Not operational

Running on SUN platform

System validated

Upgraded in line with the planned scientific development

Purpose:

Relevant inputs:

Characteritics:

Mid2003



34

SAFNWC/MSG 1.0: Scientifically checked

First version delivered to users

Data-set of stable and well calibrated real SEVIRI data

Operational

Supported on SUN platform

Scientifically checked and first tunned

Upgraded according maintenance activities

Intended to be portable (not completely ensured)

Purpose:

Relevant inputs:

Characteritics:

End2003



35

SAFNWC/MSG 1.1: Full portable

Ensured full portable

Real SEVIRI data

Feedback from Beta-testers

Operational

Fully portable on committed plattforms

Upgraded in line with the planned scientific development

Not delivered if 1.0 is fully portable

Purpose:

Relevant inputs:

Characteritics:

Beg2004



36

SAFNWC/MSG 1.2: Scientifically validated

Improve product quality, refining algorithms in accordance with the validation results

1 year of real, stable, validated SEVIRI data

Reports on Product (Scientific) Validation

Operational

Fully portable on committed plattforms

Refined algorithms according validation reports

Upgraded perfective (+ corrective, adaptive) maintenance

Purpose:

Relevant inputs:

Characteritics:

Spring2005



37

SAFNWC/MSG 2.0: New User Requirements

Implement improvements according the upgraded URD

Feedback from Users

PAR outcomes (Product Assessment Review)

Updated URD

Operational

Fully portable on (new?) committed OS/plattforms

Upgraded in line with new scientific development

Upgraded according perfective, adaptive, corrective, evolutive maintenances

Purpose:

Relevant inputs:

Characteritics:

Summer2006



38

SAFNWC/MSG 2.1: Tunned new upgrades

Check new implementations of SAFNWC/MSG 2.0

Feedback from Users

Validation reports

Operational

Fully portable on committed OS/plattforms

Upgraded according maintenance procedure

Purpose:

Relevant inputs:

Characteritics:

Feb2007



39

SAFNWC/PPS 0.20: Beta release August 2002

First release to beta testers to address portability issues

AVHRR/AMSU up to NOAA 16

NWCLIB developed under IOP

Not operational, but scientifically correct

No Task Manager

CTTH for opaque clouds only

Purpose:

Relevant inputs:

Characteritics:

Aug2002



40

SAFNWC/PPS 0.2x: Beta release, planned updates 2002

Gradually ensure full functionality of PPS package

Software problem reports from Beta users

planned software developments

Capable to process NOAA17

Taskmanager included

Bug fixes and minor software updates

Purpose:

Relevant inputs:

Characteritics:



41

SAFNWC/PPS 1.0: Fully functional and portable

First version delivered to users

Feedback from betatesters

ORR acceptance

Operational

Portable on comitted platforms (HP Unix possibly open if no beta tester is found)

Purpose:

Relevant inputs:

Characteritics:

Mid2003



42

SAFNWC/PPS 1.1: Scientifically validated

Improve product quality, refining/tuning algorithms in accordance with the validation results

Reports on Product (Scientific) Validation

Operational

Refined algorithms according to validation reports and planned improvements

Upgraded perfective (+ corrective, adaptive) maintenance

Purpose:

Relevant inputs:

Characteritics:

Spring2005



43

SAFNWC/PPS 2.0: New User Requirements & adapted to EPS data

Implement improvements according the upgraded URD

adapt software to EPS data Feedback from Users & updated URD

PAR outcomes (Product Assessment Review)

Frozen EPS specifications/ EPS simulated data

EPS/MHS real dataOperational

Fully portable on (new?) committed OS/plattforms

Upgraded in line with new scientific development

Upgraded according perfective, adaptive, corrective, evolutive maintenances

Purpose:

Relevant inputs:

Characteritics:

Summer2006



44

SAFNWC/PPS 2.1: Tunned new upgrades

Check new implementations of SAFNWC/PPS 2.0

Validated with real EPS data

Feedback from Users

Validation reports

Operational

Fully portable on committed OS/plattforms

Upgraded according maintenance procedure

Purpose:

Relevant inputs:

Characteritics:

Feb2007



45

INITIAL OPERATIONS PHASE: Support Services

The User’s Support will be based on: Help Desk Facility Training Activities



46

Support Services: Users

• EUMETSAT Central Facility

• EUMETSAT members states and cooperant states National Meteorological Services

• Other official duties Institutions of EUMETSAT members states and cooperant states

• Potential users of WMO RA I EUMETSAT Users

Data Policy applied EUMETSAT Users

Data Policy applied



47

Support Services: SAFNWC Users’ Group

The support to users will be carried out by the Operations Leading Entity to all targeted users through the SAFNWC Users' Group, which is the group of people nominated to act as focal points at User Sites



48

User’s roles and responsibilities

•Three different categories of SAFNWC/MSG "users" are foreseen

–System Administrator: responsible of the operational execution of the SAFNWC/MSG application

–Scientific Administrator: in charge of configuring all the scientific aspects of the system and adjusting the generation of the meteorological products according the needs of the final users

–Final Users (forecasters), who will receive the products as they are for analysis and exploitation



49

NOWCASTING SAFNOWCASTING SAF

SW Distribution Software ReportsSPR /SMR

Information Training Inquiry on User Req. & Product Applicability

SAFNWC Users' GroupSAFNWC Users' Group

Help DeskHelp Desk

Docs

CD ROM

Mail Box

Survey

Workshops & STMWorkshops

& STM

Docs

CD ROM

Users Workshop Users

Workshop

Support Services: SAFNWC Users’ Group Interfaces



50

Support Services: Help Desk Strategy

Status:Upgrading information in the Current Web PageRequirement of Help Desk definedHelp Desk Development on going

Status

Help Desk Development Completion; butHelp Desk Functions will be implemented progressively.

Web Site Areas:Free Access;Restricted to SAFNWC Users’ Group; and Private Area to the Consortium

Plans



51

Support Services: Training Activities

PPS Beta Users Workshop Editions 1 and 2MSG Information Meeting (Beta-testers)

Status

December 2001 & April 2002,13 institutions interested,11 licence agreement signed,All committed platforms except HP, additionally beta-testers on Linux,Delivered 9 August,Installations (Sun Solaris8, Linux Debian) completed after one week

Very positive Conclusions. Beta testing

Activities:On going

Very positive Conclusions. Beta testing

Activities:On going

September 2002,14 institutions interested,Licence agreement signature process on going,Delivery planned October 2002,Several beta-testers interested onchecking on different platformsapart from SUN

MSG PPS



52

Support Services: Training Activities

2 Joint MSG-PPS Workshops on major upgraded deliveries:

–Version MSG 1.0 and PPS 1.0–Version MSG 2.0 and PPS 2.0

Visiting Scientist Activities (Seminars, Workshops, etc.)

Plans



53

Support Services: Upgrading Requirements Plan

Based on:

New URs Implementation planned baseline:

versions 2.0

Help DeskVSA

SeminarsWorkshop

PAC Workshop

Mid Term Review

Upgrade URD

Devoted Survey



54

CONCLUSIONS:

Based on decentralised EUMETSAT policy

MSG Launch delay impact, and EPS future adaptation considered

Development concluded, but improvement planned based on products validation, users applicability and new requirements

Visiting Scientist Frame for scientific cal/val cooperation

Delivery according EUMETSAT data policy

User support ensured through the SAFNWC Users’ Group, workshops and training activities

55The EUMETSAT SAF on Nowcasting & VSRF


KEEP IN TOUCH WITH NWC SAF:www.inm.es/wwg - [email protected]






the eumetsat satellite application facility on nowcasting and very short range forecasting

Documents