Edward Tollerud

(Funded by USWRP)

HMT-DTC Collaboration

HMT-DTC Collaboration Goals

Implementation and demonstration of new verification capabilities for high-resolution NWP

Develop DTC capabilities for ensemble model prediction

Data Impact StudiesImpact studies of model physics and

parameterizations relevant to HMT research

HMT-DTC-USWRP: Common Goals

Major Accomplishments for FY 2009

Demonstration of MET-based real-time QPF verification for WRF ensemble forecasts during the HMT West Winter Exercise (10/2009-4/2010)

Retrospective MODE-based analyses of Pacific atmospheric river (AR) observations and forecasts

MET utilities to facilitate regionalized verification at RFC and basin scales

Major Accomplishments for FY 2009

Major Accomplishments for FY 2009

Major Accomplishments for FY 2009

72h 48h 24h SSM/I

MODE Object Comparison of GFS Forecasts with SSM/I Observation

for 25 February, 2004 (Clear Cut Case)

Area=312Area=369 Area=306 Area=127

Wally Clark

72 h

48 h

24 h

SSM/I

Major Accomplishments for FY 2009

John Halley Gotway, Huiling Yuan, 10/1/2009

Proposed Activities for FY 2010

Expand and maintain real-time QPF verification demonstration system and analyze results

Develop and assess prototype MODE-based spatial verification techniques to verify Eastern Pacific Atmospheric River events in near-real-time

Addition of new ensemble-based probabilistic techniques to the MET, including spatial ensemble verification prototypes

Anticipated Major Accomplishments for FY 2010

Conference paper(s) and report summarizing WRF ensemble model performance during HMT-West 2009-2010 Winter exercise

Added capability in MET to compute new ensemble-based probabilistic methods including the RPS and CRPS for QPF and PQPF verification

Evaluation of diagnostic experiments to assess the impact of verification dataset choices on verification results

Resource RequirementsStaff Category FTE Cost

Scientists 1.04 $234.3 K

Software Engineers 0.49 $84.8 K

Travel $3.5 K

Tara Jensen

DTC/HWT Collaboration

What is the HWT?

NOAAHazardous Weather Testbed(HWT)

NOAANational Severe Storms Lab

(NSSL)

NOAAStorm Prediction Center

(SPC)

OU - Cooperative Institute for

Mesoscale Meteorological Studies (CIMMS)

BRINGING RESEARCHto FORECAST OPERATIONS

The mutual interests of forecastersfrom the SPC, researchers from NSSL,and collocated joint research partnersfrom CIMMS inspired testbed formation.

OtherCollaborators

Goal: Give forecasters first-hand look at the

latest research concepts and products Immerse researchers in the challenges,

needs, and constraints of front-line forecasters

Approach: Forecast teams gather in Norman each

week from late April to early June. Each day consists of:

Daily Briefing Evaluate Previous Day’s Forecast Selection of Current Day’s Forecast

Area Forecasters split into 2 teams to

predict Chance of Severe Wx

HWT Spring Experiment

Years2000 2005

2001 2007 2002 2008 2003 2009 2004 2010

Motivation for DTC/HWT CollaborationOne of the recommendations from the 2007

External Review of DTC Initial focus

Introduction of objective evaluation into HWT Verification of storm-scale prediction (<= 4km) for severe

weatherExpanded focus

Evaluation of ensemble forecasts and post-processingVerification of storm-scale prediction (<= 4km) for SPC

aviation weatherProvides DTC with prominent role in the direct

infusion of research models and evaluation into operational environmentExploration of how forecasters can effectively use

objective verification

DTC/HWT Collaboration Goals

Supplement HWT Spring Experiment subjective assessments

Provide objective evaluation of experimental forecasts contributed to Spring Experiment

Expose the forecasters and researchers to both traditional and new approaches for evaluating forecasts

Assist HWT in addressing model improvement problems and investigating other research topics

Major Accomplishments for FY 2009

Implementation of a near real-time evaluation system for• 1 hr Accumulated Precipitation• Composite Reflectivity

Assess impact of radar assimilation on short-term forecast using both:• Traditional Methods – grid to grid statistics• Spatial Methods – object oriented evaluation

Education and Outreach• DTC staff participation during every week of Spring

Experiment (SE) 2009• Presentation at WAF/NWP Conference, Poster at WRF User’s

Workshop

SE 2009 Evaluation

Composite reflectivity and 1-hr Accum. Precip. Forecasts

NSSL NMQ-Q2 observed fields

3 high-resolution modelsCAPS 4km SSEF control

(with radar assimilation)CAPS 4km SSEF perturbation

(no radar assimilation)HRRR 3km

(different radar assimilation)

Question: How does radar

assimilation impact 0-12hr forecast?

RESULTS:

Radar assimilation appears to improve 0-6hr skill scores

Lack of clear difference in skill scores during 6-12 hr lead times suggests model physics taking over

Preliminary 2009 Results

Results were aggregated over Spring Experimenttime period and the median values are plotted

Radar20dBZ

No Radar20dBZ

14 May 2009 Init: 00 UTC Spatial Thresh: 30dBZ

No Radar

ObjectsForecast

FieldObserved

Field

Radar

FCSTOBJ

OBSOBJ

High Impact Displayallowed HWT to identifyphase and convective modeerrors

Still working on how to quantify objectively

Proposed Activities for FY 2010

Expand evaluation to include all 20 members of the CAPS ensemble – retain evaluation of HRRR

Have DTC staff available at SE to introduce methods and discuss objective evaluation

Include HWT requirements in METviewer (database and display system) beta version.

Generalize system architecture to allow for retrospective studies.

Support HWT goals by expanding evaluation to include one or two aviation related variables.

Anticipated Major Accomplishments for FY 2010

Report on objective evaluation of impact of radar assimilation on 0-12hr.

Install beta METviewer at HWT.Retrospective testing and evaluation

system developed.DTC staff participation during each week

of Spring Experiment 2010.Manuscript submitted to AMS Weather and

Forecasting journal.

Resource RequirementsStaff Category FTE Cost

Scientists 0.54 $81.8 K

Software Engineers 0.32 $50.1 K

Students 0.15 $7.4 K

Travel $9.0 K

HWT 2008

•Introduce Objective Evaluation

HWT 2009

•Realtime system

•Address scientific question

HMT

2010

•1st Ensemble evaluation

•Satellite data into MET

HWT 2010

•Add Ensemble methods

•DTC METviewer

HMT

2011

•Refine Ensemble methods

•Data Impact Studies

METDevelopm

ent*TressaFowler

discussedearlier

DTCEnsembleTestbed*Zoltan

Tothwill

discussnext

etc…

Zoltan Toth and Tara Jensen

DTC ENSEMBLE TESTBED(DET)

Historical OverviewGlobal ensemble forecasting operational at NCEP since

1992Mesoscale ensemble since 2000

Recognition of need for / value of ensembles by communityNRC Report on Forecast Uncertainty – 2007

NWS response to NRC reportForecast Uncertainty (NFUSE) program & decision support

initiativeTri-agency (NOAA, Navy, US Air Force) initiative

National Unified Operational Prediction Capability (NUOPC)Weather Enterprise response to NRC Report

AMS Ad Hoc Committee on Forecast Uncertainty (ACUF)

STRONG COMMUNITY INTEREST

National Workshop On Mesoscale Probabilistic PredictionHeld at NCAR, Sept. 2009

Organized by DTCWell attended by ensemble community (60+

attendees)Academia, agencies, private sector, international invitees

Assessed current state of operations & research related to mesoscale ensemblesInvited and contributed presentations

Working Group discussions aimed at developing roadmapDrawing on ongoing research (THORPEX), agency

(NUOPC) & community (ACUF) planning & activitiesWhite paper being drafted

Will be valuable advice & resource for DTC

Formation of DET in FY2010

Confluence of necessary ingredients

NeedStrong interest from both community & agencies

Knowledge baseRoadmap from Sept 2009 Workshop

OpportunityIncrease in funding for DTC from NOAA

Initiative DTC Director’s Office sets planning process in

motion (Oct 09)

Goal of DET

“Provide an environment in which extensive testing and evaluation of ensemble-related techniques developed by the NWP community can be conducted such that the results are immediately relevant to the operational centers (e.g., NCEP/EMC and AFWA)”

Initial focus on mesoscales

Major Activities for Yr-1Form DTC Ensemble Advisory Panel (DEAP)

Establish DET infrastructure – 70% effort in FY10Codes/scripts from operational centers for

Distribution to community useUse as benchmark in testing new methods developed by

community

Test & evaluate new methods developed by community – 30%Links with other testbeds / programs using

ensemble approach

Yr-1 Task – Form DTC Ensemble Advisory Panel (DEAP)

PurposeDEAP will provide advice to DET on

Software and hardware infrastructure necessary and desirable for the testing of new ensemble-related methods developed by the community

Selection and prioritization of ensemble-related methods to be tested by DTC prior to their potential transition to operations

Proposed membershipCliff Mass (UW), Brian Colle (SUNY), Bill Gallus (Univ in

Illinois), David Stauffer (Penn State Univ), Chris Snyder (NCAR), Evan Kucera (AFWA), Craig Bishop (NRL), David Bright (BPA), Tony Eckel (NWS/OST?), Jun Du (NCEP/EMC), Russ Schneider (NCEP/SPC), Mike Brennan (NCEP/TPC), Zoltan Toth (ESRL/GSD)

Required Characteristics of DET InfrastructureCompatibility with NOAA Environmental Modeling

System (NEMS)Preparing for future use of ESMF by agencies

PortabilityTo serve diverse user base utilizing various

computational platformsCommunityDifferent agenciesDTC (limited computational resources)

ModularityTo facilitate testing of new community developed

components against operational benchmark

Yr-1 Tasks for Infrastructure Development

Ensemble configuration moduleDesign, basic development

Initial perturbations moduleDesign, basic development

Model perturbations moduleDesign

Statistical post-processing moduleDesign

Probabilistic product generation moduleAssess available tools; Design

Verification moduleAssess/augment MET

Yr-1 Tasks – Test & Evaluation

Develop test/evaluation protocol forAssessing scientific value of community

contributionsInitial assessment of readiness for transition

to operations

Initial testing of HMT or HWT or HFIP ensemblesEngage with broader ensemble forecast user

community

Anticipated Major Accomplishments for FY 2010Establish DTC Ensemble Advisory Panel

(DEAP) and engage in DET planning activities

Define DET infrastructure requirements, review existing tools and software packages for suitability for incorporation, design ensemble modules, and define priorities for module development

Complete basic development of two modules, including preliminary testing to assure functionality and portability

Establish formal test and evaluation protocol for DET and demonstrate by applying to an available ensemble data set

Resource RequirementsCategory FTE Cost

Scientists 1.95 $439.2 K

Software Engineers 1.05 $183.2 K

Travel $9 K

Workshop / Meetings $20 K

Laurie Carson

DTC & NEMS NOAA Environmental Modeling System

DTC NEMS MotivationNCEP will use the NEMS framework for their

NWP (Numerical Weather Prediction) capabilities GFS NMM-B FIM

DTC facilitates operational and research collaborations with the goal of accelerating the transfer of new science and technology from research into operations

WRF-NMM WPP (WRF Post Processor) GSI HWRF

DTC NEMS GoalsNear TermDefine community software goals for the

NEMS modeling framework and operational model configurations

Gain expertise in the NEMS framework, by participating in and assisting the on-going development, with a community support focus

Long TermSupport the operational NEMS to the

community; Provide research results and capabilities to operational forecast systems

Proposed Activities for FY 2010Participate in the Global Interoperability

ProgramAttended kick-off meeting Nov 5-6, 2009http://gip.noaa.gov/

Build a foundation of DTC expertise with the NEMS software

Proposed Activities for FY 2010Define community software goals for NEMS

Collaborate with EMC (Environmental Modeling Center) to:

Hold a working group meetingDTC transition process for moving from WRF software

infrastructure to NEMS-based packagesTiming of the transitionPotential overlap between WRF and NEMS with respect to

community supportMaintaining a connection with the broad research

community.

Draft a code management planDTC NEMS operations to research (O2R)

transition plan

Anticipated Major Accomplishments for FY 2010: DTC NEMSDefine DTC NEMS community software

goals and timelineDevelop DTC expertise in NEMS softwareEstablish DTC NEMS operations to

research (O2R) transition plan

Resource RequirementsStaff Category FTE Cost

Scientists 0.1 $16.2 K

Software Engineers 0.75 $138.9 K

Travel $18 K