11° cosmo general meeting – offenbach 07-11 sept 2009 versus2 priority project report

11° COSMO General Meeting – Offenbach 07-11 Sept 2009

VERSUS2 Priority ProjectReport


VERSUS2 Priority ProjectReport phase1

OPEN POINTS: VERSUS was not planned to cover all the existing verification methods, as well as all the possible products from a NWP and not all types of observation

• implementation of probabilistic and ensemble forecasts verification;

• implementation of obj-based and fuzzy verification methods applied to precipitation;

• New scores for extreme events (e.g. extreme dependency score);

• Statistical features like Confidence intervals and Bootstrap method;

• Use of non conventional obs (e.g. radar, satellite, raingauges) and gridded observations (precipitation analysis);

• ....................and so on----- VERSUS2!!!



PHASE 1: COSMO year 2009Task 0: Requirements specification for VERSUS and accepted reviewMain Activities (or subtasks)Preparation of a “forum” and start of a survey Survey analysis: Delivery of System Design Overview (SDO))Review of System Design Overview Drawing up the project Final acceptance of the System Design Overview and SAD

Deliverables : Ok. Some updates on SAD doc still pending



PHASE 1: COSMO year 2009 Task 1a: Implementation of a multiplatform RDBMS version of VERSUS Main ActivitiesAnalysis, design, development, and implementation of DB classes;GUI, R package and front-end integration with the above DB classesTest phase

Deliverables: This task has been deleted. No requirement from any Cosmo countriesAll will install VERSUS package with the present software architecture



PHASE 1: COSMO year 2009 Task 1b: Installation of multiplatform DB version of VERSUS at every service within COSMOThe installation and the use at every service should be reached before the implementation of the any other requirements and functionalities from phase 2.Main ActivitiesInstallation and use of VERSUS package original version at every service within COSMO Functionality tests, configuration and production of verification scoresInstallation of updated multiplatform DB version of VERSUS at every service within COSMOHelp Desk by PL team



PHASE 1: COSMO year 2009 Task 1b: Installation of multiplatform DB version of VERSUS at every service within COSMO

Deliverables : Installation and use of VERSUS

Poland, Greece Romania: ok. They installed and used VERSUS to produce plots. Operational use is the next step in few months.

MCH, DWD and Russia: partially installed, but still few problems setting up the system . Hopefully after GM the installation (and use) will be completed.


autumn winter

spring

2m Temperature

summer


autumn winter

spring summer

10m Wind Speed


summerspring

winterautumn

12h FBI - range: 0 to ∞, unbiased score=1Indicates tendency to underforecast (BIAS<1) or

overforecast (BIAS>1) events.


VERSUS2 Priority ProjectHighlights of phase2

PHASE 2: COSMO year 2010

RULES

- Any single phase should be concluded in just one COSMO year

- Delay in the realisation of one single phase will result in a delayed start of the next one

- Every year, based on the real deliverables produced, the plan will be updated accordingly along with a quick review of the requirements of the next phase,




Priorities from VERSUS workshop in Langen in march.ù

3 groups of priorities: A, B,C

2010: some selected “A” priorities by WG5

2011: remaining A and B+C priorities

If not enough or need for more requirements extension 2012 ?



PHASE 2: COSMO year 2010 Task 0: Help Desk, bugs fixing activities and release of VERSUS update

Main Activities Help Desk activities for VERSUS usersSoftware maintenanceActivity of bugs fixings and test/release of package updates to be delivered to the users through ftpPatches test on at least another Installed package

Deliverables :Package updates.

Estimated resources0,4 FTEs for PL and PL team0,1 FTEs for Greece (?)Expected Start: October 2009



PHASE 2: COSMO year 2010 Task 1: Implementation of COSI global Index

Main Activities (or subtasks)Task 1a: Preparation of a technical reference document for COSI index: deadline end of OctoberTask1b: Implementation of the index: deadline mid December 2009DeliverablesUpdate of VERSUS package will include COSI index calculation and graphical/text presentation. Deadline: December 2009.Estimated resources0,15 FTEs for PL and PL team for the preparation of the document and implementation/test of the index;0,21 FTEs (total amount) for all the other members for preparation and review of the document (0.03 each)Expected Start: October 2009



PHASE 2: COSMO year 2010 Task 2: Implementation of weather type dependent verification and improvement of CV..Main ActivitiesTask 2a: Preparation of a Guidelines document (WDV);Task 2b: Conditional verification on different space/timeTask 2c: Development of VERSUS software modulesDeliverablesUpdate of VERSUS package to include WDV and CV improvement. Deadline: March 2010.Estimated resources0,14 FTEs for all the COSMO countries to prepare and review the Guidelines document (0.02 each)0,03 FTE Guidelines for CV on different space/time (MCH)0,25 FTE PL and PL team for the implementation and testExpected Start: October 2009 for Guidelines; December 2009 for implementation.



PHASE 2: COSMO year 2010 Task 3: Use of new Observation data

Main ActivitiesTask 3a: Identification and analysis of Observations to be loaded following SDO priorities: Airep, Amdar, Windprofiler, TEMP (non standard pressure level), Metar;Task3b: Implementation of Loader module (msgs coded in BUFR format) and creation of tables in DB;Task3c: Integration of the new Upper air observation with the existing ones for production of Upper air verification Task3d: Implementation of verification of Wind gusts using Metar;Task3e: Analysis and description of Radar wind data for use in VERSUS system(VAD, radial);Task3f: Implementation of loader for radar wind data;



PHASE 2: COSMO year 2010 Task 3: Use of new Observation data

DeliverablesUpdate of VERSUS package with the new functionalities. Deadline: June 2010Estimated resources0,05 FTE for COSMO member (DWD,MCH?) for detailed requirements for upper air verification and wind gusts verification0,05 FTE for detailed description and requirement on use of radar wind data for verification0,30 FTE for the implementation of new modules by PL teamExpected Start: November 2009 for requirements. February 2010 for implementation..



PHASE 2: COSMO year 2010 Task 4: Use of gridded observation (analysis fields)The use of gridded observation (analysis field) is crucial to check the performance of a model compared to an objective analysis (model oriented verification) and to apply special verification methods for very high-resolution models, as Fuzzy verification (through the use of Fuzzy toolbox) and later object oriented verification (MODE and SAL).Radar precipitation data, raingauges and synop data as well as satellite precipitation estimation should be used in a gridded precipitation analysis.Gridded observation will have to be given to the system for external procedure as input data. Interpolation on the same grid (model on analysis or vice versa) could be provided by VERSUS system (TBD)



PHASE 2: COSMO year 2010 Task 4: Use of gridded observation (analysis fields) Main ActivitiesTask4a: Individuation of some aggregation methods for standard scores.Task4b: Implementation of Loader module for gridded observation (GRIB coded);Task4c: Detailed documentation of Fuzzy verification toolbox for its integration and implementation in VERSUS packageTask4d: Re-writing Fuzzy IDL code in a more manageable C or Fortran code (TBD) Task4e: Implementation of Fuzzy toolbox and updating of WEB GUI accordingly



PHASE 2: COSMO year 2010 Task 4: Use of gridded observation (analysis fields)DeliverablesUpdate of VERSUS package with the new functionalities. Deadline: March 2010Estimated resources0,03 FTE for COSMO member (DWD) for detailed description of Fuzzy toolbox0,02 FTE aggregation method for standard scores0,15 FTE for Fuzzy toolbox C or Fortran version (TBD, MCH?) : as IDL is not available for any COSMO member and the spirit of free-licensing software of VERSUS should be respected, this subtask 0,2 FTE for the implementation of new modules and GUI in VERSUS by PL teamExpected Start: October 2009 description of toolbox and rewriting of package, February 2010 for implementation.



PHASE 2: COSMO year 2010 Task 5: Identification of scores for Ensemble and probabilistic forecastsThis task is a prerequisite for Phase3 where loader module for probabilistic forecasts and probabilistic scores verification will be implemented.Main ActivitiesTask 5: Identification and full description of probabilistic scores to be implemented with description and example plots to be implemented for these probabilistic score.DeliverablesGuidelines document on probabilistic scores and plots to be implements in Phase 3. Example of algorithm to be implemented. Analysis of new techniques. Deadline: June 2010.Estimated resources0,25 FTE for the responsible member.(Greece?)Expected Start: February 2010.



PHASE 2: COSMO year 2010 Task 6: Production of common statistical scores and scores graphicRequirements for Common scores plots from WG5 CoordinatorMain ActivitiesTask6a: Preparation of data to be sent to the responsible member;Task6b: Production of seasonal common scores plots and preparation of presentation for GM;



PHASE 2: COSMO year 2010 Task 6: Production of common statistical scores and scores graphicDeliverablesA complete set of common plots following specifications of WG5 Coordinator. Deadline: end of July for SON, DJF and MAM seasons. Beginning of September 2010 for JJA 2010 (depending of the beginning of next GM).Estimated resources0,12 FTE for all the COSMO member (0.02 each) to prepare data to be collected by the responsible member) 0,3 FTE for Poland for collection and loading of observations and forecasts data, production of statistical scores and graphics, following the specifications of WG5 coordinator and preparation of presentation for GM.Expected Start: December 2009.



PHASE 2: COSMO year 2010 Task 7: Preparatory work on feedback files implementationFeedback files contain the observation and their deviation from forecasts for analysis and whole model runs. They do not contain only traditional observation and include information on data quality and status. Their implementation will result really useful for verification purposes, provided that all COSMO member will be able to carry out feedback files from their operational and experimental runs.This task is a prerequisite for Phase 3 implementation of FB files.

Main ActivitiesTask7a: Fully description of feedback files;Task7b: Implementation of loader module for feedback files in VERSUS system (creation of tables, php modules...) to be integrated in VERSUS_FE loader.



PHASE 2: COSMO year 2010 Task 7: Preparatory work on feedback files implementation

DeliverablesA front end loader for feedback files integrated in VERSUS_FE module and DB tables where data for FB files will be stored. Deadline: September 2010.Estimated resources0,30 FTE for DWD0,05 FTE for PL team for support of the activity.Expected Start: March 2010.



PHASE 2: COSMO year 2010 Task 8: Statistical features like confidence intervals and the Bootstrap method (to be approved)

Main activities: Implementation of CI and BM and others possibly using “R” Verification package. Analysis of the package and inclusion in VERSUS or development of a new modules (Fort coding or C coding)

DeliverablesSoftware modules integrated in VERSUS and update of WEB GUIDeadline: September 2010.Estimated resources0,25 FTE for Russia0,05 FTE for PL teamExpected Start: January 2010.




FTEs for 2010 Cosmo year 3.15

TaskContributing scientist(s) FTE- year Start Deliverables Date of delivery Preceding tasks

Task 0PL team – ItalyPatches and new vers. test – Greece (?)

Total 0.50,40,1

01.10.2009 Help Desk, bugs fixing activities and release of VERSUS update 01.09.2010 none

Task 1.1a.1b

All Cosmo membersPL team -

Total 0.36 0,21(0,03e.)0,15

1.10.2009 Implementation of COSI global Index 31.12. 2009 none

Task 2.2a.2b.2c

All COSMO memberMCH (?)PL team

Total 0.420.14 (0.02e.)0.030.25

01.10.2009 Implementation of weather type dependent verification and improvement of CV.

31.03.2010 none

Task 3.3a.3b,c,d,f.3e

DWD (?)PL teamMCH (?)

Total 0.400.050.300.05

01.11.2009 Use of new Observation data 30.06.2010 none

Task 4.4a.4b,e.4c.4d

(?)PL TeamDWD(?)Rumania (?)

Total 0.400.020.20.030.15

01.10.2009 Use of gridded observation (analysis fields) 31.05.2010 none

Task 5 Greece (?) 0.25 01.02.2010 Identification of scores for Ensemble and probabilistic forecasts 31.05.2010 none

Task 6.6a.6b

All COSMO memberPoland (?)

Total 0.420.120.3

01.12.2009 Production of common statistical scores and scores graphic GM 2010 Phase 1 completed

Task 7.7a.7b

DWDDWD,

Total 0.400.050.30 – 0..05

01.03.2010 Preparatory work on feedback files implementation GM 2010 none

Task 8RussiaItaly

Total 0,30.250.05

01.01.2010 Statistical features like confidence intervals and the Bootstrap method (to be approved)

July 2010 none

11° cosmo general meeting – offenbach 07-11 sept 2009 versus2 priority project report

Documents