interactive verification program training for awips ob8.2 hank herr office of hydrologic development...
TRANSCRIPT
![Page 1: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/1.jpg)
Interactive Verification ProgramInteractive Verification ProgramTraining for AWIPS OB8.2Training for AWIPS OB8.2
Hank HerrHank Herr
Office of Hydrologic DevelopmentOffice of Hydrologic Development
RFC Verification Workshop, 08/15/2007
![Page 2: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/2.jpg)
22
OverviewOverview
IntroductionIntroduction IVP TerminologyIVP Terminology Software ComponentsSoftware Components Database UsageDatabase Usage Output FilesOutput Files DesignDesign DemonstrationDemonstration TipsTips Future PossibilitiesFuture Possibilities ExercisesExercises DiscussionDiscussion
![Page 3: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/3.jpg)
33
Original IVP written around 2000-2001 for old HP based verify softwareOriginal IVP written around 2000-2001 for old HP based verify software Verification moved to archive database for OB4Verification moved to archive database for OB4
– IVP Batch Program software writtenIVP Batch Program software written– Old IVP GUI still ran, but used pairs files created during batch processing; Old IVP GUI still ran, but used pairs files created during batch processing;
it did not access the archive database directlyit did not access the archive database directly New IVP GUI written for OB7.2New IVP GUI written for OB7.2
– Provided ability to build verification products via GUI, modify components Provided ability to build verification products via GUI, modify components of a chart and save modifications, create batch file to reconstruct graphics of a chart and save modifications, create batch file to reconstruct graphics via IVP Batch Programvia IVP Batch Program
– Accessed archive database directlyAccessed archive database directly– Limitations:Limitations:
Designed for stage dataDesigned for stage data Accessed only three data tables of the archive databaseAccessed only three data tables of the archive database Limited number of statisticsLimited number of statistics No quality control of pairsNo quality control of pairs Inflexibility in building batch files from GUIInflexibility in building batch files from GUI JClass Desktop Views used to render charts (limited charting capabilities; JClass Desktop Views used to render charts (limited charting capabilities;
upgrade cost too much money)upgrade cost too much money)
IntroductionIntroductionHistory of IVPHistory of IVP
![Page 4: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/4.jpg)
44
IntroductionIntroductionNew for OB8.2New for OB8.2
Designed for verification of stage, flow, temperature, and precipitation Designed for verification of stage, flow, temperature, and precipitation forecastsforecasts– Can do other data types as long as data is stored within accessible tables Can do other data types as long as data is stored within accessible tables – Accesses 10 data tables of the archive databaseAccesses 10 data tables of the archive database
Uses all data tables except pempsep, pemrsep, and pairedvaluesUses all data tables except pempsep, pemrsep, and pairedvalues Additional statisticsAdditional statistics
– Moments, CDF Plot, ROC, etc.Moments, CDF Plot, ROC, etc. Quality control of pairs via GUIQuality control of pairs via GUI
– View pairs in a time series plot, in addition to existing scatter plotView pairs in a time series plot, in addition to existing scatter plot– Mark pairs as ‘bad’ so that verification ignores themMark pairs as ‘bad’ so that verification ignores them– Change forecast or observed values within a pair and save changes to Change forecast or observed values within a pair and save changes to
databasedatabase More flexible batch file constructionMore flexible batch file construction
– Batch file can be built that will construct current chart for other Batch file can be built that will construct current chart for other combinations of locations and analysis periodscombinations of locations and analysis periods
Chart Director used for renderingChart Director used for rendering– Relatively quick, memory efficient chart renderingRelatively quick, memory efficient chart rendering– Used previously for HydroGen, so a run-time license already existsUsed previously for HydroGen, so a run-time license already exists
![Page 5: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/5.jpg)
55
IVP TerminologyIVP Terminology
Verification LocationVerification Location– A forecast point or location and a data type for which verification is A forecast point or location and a data type for which verification is
to be performedto be performed– Uniquely specified by location id (lid within database) and SHEF Uniquely specified by location id (lid within database) and SHEF
pedtsep code.pedtsep code.– Includes following information:Includes following information:
Forecast and observed categoriesForecast and observed categories Observation type (raw or processed)Observation type (raw or processed) Active flag (yes or no)Active flag (yes or no) National flag (yes or no)National flag (yes or no) Sensor preferences (ordered list of observed type source, representing Sensor preferences (ordered list of observed type source, representing
preferences for which observed values to pair with a forecast)preferences for which observed values to pair with a forecast)
![Page 6: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/6.jpg)
66
IVP TerminologyIVP Terminology
Verification GroupVerification Group– A set of locations and parameters that define the data to load and A set of locations and parameters that define the data to load and
other parameters of verificationother parameters of verification Locations must have the same number of categories and must be of Locations must have the same number of categories and must be of
the same type: stage (height), flow (discharge), temperature, the same type: stage (height), flow (discharge), temperature, precipitationprecipitation
– Includes following parameters:Includes following parameters: Analysis period and intervalsAnalysis period and intervals Lead time period and intervalsLead time period and intervals Issuance time-of-day (0-24 hours) period and intervalsIssuance time-of-day (0-24 hours) period and intervals Restrictions on location response time, active flag, and SHEF Restrictions on location response time, active flag, and SHEF
components (forecast type source, physical element, duration, and components (forecast type source, physical element, duration, and extremum)extremum)
Whether or not to compute statistics independently for each lid or Whether or not to compute statistics independently for each lid or forecast type sourceforecast type source
![Page 7: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/7.jpg)
77
IVP TerminologyIVP Terminology
Forecast-Observed Data PairForecast-Observed Data Pair– A forecast value paired with an observed valueA forecast value paired with an observed value
Observed value can be “raw” or “processed” depending on if the value Observed value can be “raw” or “processed” depending on if the value was posted to the database by the raw or processed SHEF decoderwas posted to the database by the raw or processed SHEF decoder
Each forecast can have one raw pair and one processed pair Each forecast can have one raw pair and one processed pair associated with itassociated with it
– ““Best” candidate observation is chosen for each forecast based on Best” candidate observation is chosen for each forecast based on following criteria:following criteria: SHEF qualifier code (G, M, P, V, S, Z, T, F, and Q, in order)SHEF qualifier code (G, M, P, V, S, Z, T, F, and Q, in order) sensor preference (observation type source) ranksensor preference (observation type source) rank difference between forecast validtime and observation obstimedifference between forecast validtime and observation obstime
– A pair can be “good” or “bad”A pair can be “good” or “bad” Bad pairs are not used in verification computationsBad pairs are not used in verification computations
![Page 8: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/8.jpg)
88
Software ComponentsSoftware Components
Vfyruninfo Editor GUIVfyruninfo Editor GUI– Manage vfyruninfo tableManage vfyruninfo table
IVPIVP– Data quality control and define verification products graphicallyData quality control and define verification products graphically
IVP Batch ProgramIVP Batch Program– Build pairs and verification products via a batch languageBuild pairs and verification products via a batch language
IVP Batch BuilderIVP Batch Builder– Edit IVP Batch Program batch files graphicallyEdit IVP Batch Program batch files graphically
Verify Pairs IngestorVerify Pairs Ingestor– Ingest pairs directly from text files of specific format into the Ingest pairs directly from text files of specific format into the
vfypairs or vfyprocpairs tablesvfypairs or vfyprocpairs tables del_bad_pairsdel_bad_pairs
– Removes bad pairs from the vfypairs and vfyprocpairs tableRemoves bad pairs from the vfypairs and vfyprocpairs table
![Page 9: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/9.jpg)
99
Database UsageDatabase Usage
Verification Location InformationVerification Location Information– Vfyruninfo tableVfyruninfo table
Defines locations for which verification can be performed.Defines locations for which verification can be performed. Individual entries for each verification location and sensor Individual entries for each verification location and sensor
preference component (observed type source)preference component (observed type source) Edited via Vfyruninfo Editor GUIEdited via Vfyruninfo Editor GUI
– Location table Location table Information used in building labels, includes RFC name, full Information used in building labels, includes RFC name, full
location name, etc.location name, etc.
– Rivercrit table Rivercrit table Flood stages and flows (if location is of appropriate data type).Flood stages and flows (if location is of appropriate data type).
![Page 10: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/10.jpg)
1010
Database UsageDatabase Usage
Building PairsBuilding Pairs– Tables of data for pairing:Tables of data for pairing:
Forecasts: pedfsep, pehfsep, peqfsep, pedcsepForecasts: pedfsep, pehfsep, peqfsep, pedcsep Raw Observations: pecrsep, pedrsep, peoosepRaw Observations: pecrsep, pedrsep, peoosep Processed Observations: pedpsep, pehpsep, peqpsep, peoosepProcessed Observations: pedpsep, pehpsep, peqpsep, peoosep
– Vaddadjust tableVaddadjust table Adjust observations prior to pairing with a forecastAdjust observations prior to pairing with a forecast Only affects forecast-observed pairsOnly affects forecast-observed pairs Is not dated; there can be only one entry per locationIs not dated; there can be only one entry per location
– Sensok tableSensok table Allows for specifying time ranges during which sensors should Allows for specifying time ranges during which sensors should
not be used (i.e. observations should not be used for pairing)not be used (i.e. observations should not be used for pairing)
![Page 11: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/11.jpg)
1111
Database UsageDatabase Usage
Storing PairsStoring Pairs– Vfypairs tableVfypairs table
Stores pairs constructed using raw observationsStores pairs constructed using raw observations Field quality_code defines if the pair is good or badField quality_code defines if the pair is good or bad Can be very large (millions of records)Can be very large (millions of records)
– Vfyprocpairs tableVfyprocpairs table Stores pairs constructed using processed observationsStores pairs constructed using processed observations Field quality_code defines if the pair is good or badField quality_code defines if the pair is good or bad Can be very large, but will likely be a fraction of the size of Can be very large, but will likely be a fraction of the size of
vfypairsvfypairs
– A forecast can have up to two pairs built for it: one A forecast can have up to two pairs built for it: one stored in vfypairs and one stored in vfyprocpairsstored in vfypairs and one stored in vfyprocpairs Only one of the two pairs can be analyzed at one timeOnly one of the two pairs can be analyzed at one time
![Page 12: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/12.jpg)
1212
Output FilesOutput Files
Output Statistics File (batch only)Output Statistics File (batch only)– Verification statistics output in an ASCII tabular formatVerification statistics output in an ASCII tabular format
Pairs FilePairs File– Pairs used to compute statisticsPairs used to compute statistics– ASCII tabular formatASCII tabular format
Output Image File (.png, .jpg)Output Image File (.png, .jpg)– PNG or JPG files displaying a plot of verification PNG or JPG files displaying a plot of verification
statisticsstatistics Image Data File (.dat)Image Data File (.dat)
– Statistic values comprising an output image plotStatistic values comprising an output image plot– ASCII tabular formatASCII tabular format
![Page 13: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/13.jpg)
1313
DesignDesignFoundationFoundation
Archive Database
Archive database provides the following:Archive database provides the following:– Forecasts and observationsForecasts and observations
– Verification location informationVerification location information
– Forecast-observed data pairsForecast-observed data pairs
User
![Page 14: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/14.jpg)
1414
DesignDesignVfyruninfo EditorVfyruninfo Editor
Archive Database
Vfyruninfo Editor
User
Vfyruninfo rows
Vfyruninfo Editor is used to specify verification Vfyruninfo Editor is used to specify verification locations and saves them to the vfyruninfo locations and saves them to the vfyruninfo table of the archive databasetable of the archive database
![Page 15: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/15.jpg)
1515
PairingProcessor
Archive Database
User
pairs
Design Design IVP Batch ProgramIVP Batch Program
Vfyruninfo Editor
Vfyruninfo rows
forecasts,observations,location info
IVP Batch Program provides the user the IVP Batch Program provides the user the ability to:ability to:
– Pair forecasts with observed values according Pair forecasts with observed values according to user specified parametersto user specified parameters
parameters
Batch
Batch file
![Page 16: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/16.jpg)
1616
DesignDesignIVP Batch ProgramIVP Batch Program
VerificationProcessor
PairingProcessor
Archive Database
User
pairs
ASCII
pairs, location info
forecasts,observations,location info
Vfyruninfo Editor
Vfyruninfo rows
pairs, statistics
IVP Batch Program provides the user the IVP Batch Program provides the user the ability to:ability to:
– Calculate statistics and output results to tabular Calculate statistics and output results to tabular ASCII filesASCII files
parameters
Batch
Batch file
![Page 17: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/17.jpg)
1717
DesignDesignIVP Batch ProgramIVP Batch Program
VerificationProcessor
PairingProcessor
Archive Database
User
ASCII
pairs
GraphicsProcessor
Graphics
statisticsparams
Vfyruninfo Editor
Vfyruninfo rows
forecasts,observations,location info
pairs, location info
pairs, statistics
IVP Batch Program provides the user the IVP Batch Program provides the user the ability to:ability to:
– Generate user customizable graphical Generate user customizable graphical verification products displaying statisticsverification products displaying statistics
parameters
Batch
Batch file
![Page 18: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/18.jpg)
1818
params
VerificationProcessor
PairingProcessor
Archive Database
User
GUIGraphics
ASCII
pairs
pairs, statistics
DesignDesignIVP GUIIVP GUI
Vfyruninfo Editor
Vfyruninfo rows
forecasts,observations,location info
pairs, location info
IVP provides the user the ability to:IVP provides the user the ability to:– Specify, build, and view graphicsSpecify, build, and view graphics
GraphicsProcessor
statisticsparams
parameters
Batch
Batch file
![Page 19: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/19.jpg)
1919
params
VerificationProcessor
PairingProcessor
Archive Database
User
GUIGraphics
ASCII
pairs
pairs, statistics
DesignDesignIVP GUIIVP GUI
Vfyruninfo Editor
Vfyruninfo rows
forecasts,observations,location info
pairs, location info
IVP provides the user the ability to:IVP provides the user the ability to:– Modify forecast-observed data pairsModify forecast-observed data pairs
GraphicsProcessor
statisticsparams
pairs
parameters
Batch
Batch file
![Page 20: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/20.jpg)
2020
params
VerificationProcessor
PairingProcessor
Archive Database
User
GUIGraphics
ASCII
pairs
parameters
pairs, statistics
DesignDesignIVP GUIIVP GUI
Batch
Vfyruninfo Editor
Vfyruninfo rows
forecasts,observations,location info
pairs, location info
IVP provides the user the ability to:IVP provides the user the ability to:– Create batch files to produce graphics via IVP Create batch files to produce graphics via IVP
Batch ProgramBatch Program
GraphicsProcessor
statisticsparams
pairs
Batch file batchtokens
![Page 21: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/21.jpg)
2121
DesignDesignIVP Window LayoutIVP Window Layout
1. Load Pairs 2. View and QC Pairs
3. Define Products4. View Products
![Page 22: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/22.jpg)
2222
DemonstrationDemonstration
Feature availability:Feature availability:– Both OB7.2 and OB8.2Both OB7.2 and OB8.2– OB8.2 only*OB8.2 only*
![Page 23: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/23.jpg)
2323
Start Vfyruninfo EditorStart Vfyruninfo Editor Filter ingestfilter table contents*Filter ingestfilter table contents* Select locations from ingestfilter table*Select locations from ingestfilter table* Create locations from scratchCreate locations from scratch Specify location parametersSpecify location parameters
– Active*Active*– National*National*– Response timeResponse time– Sensor preferencesSensor preferences
Save to archive databaseSave to archive database
DemonstrationDemonstrationVfyruninfo EditorVfyruninfo Editor
ref: Vfyruninfo Editor User’s Manual
![Page 24: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/24.jpg)
2424
Commands:Commands:– Time window: START_TIME, END_TIMETime window: START_TIME, END_TIME– Specify locations: LOCATION, Specify locations: LOCATION, DUR*DUR*, , EXTREMUM*EXTREMUM*, ,
FCST_TS*FCST_TS*, , PE*PE*– Pairing window: PAIRING_WINDOWPairing window: PAIRING_WINDOW– Observation type: Observation type: OBS_TYPE (replaces OBS_TABLE)*OBS_TYPE (replaces OBS_TABLE)*– Persistence flag: PERSISTENCEPersistence flag: PERSISTENCE
Action:Action:– Build the pairs: BUILD_PAIRSBuild the pairs: BUILD_PAIRS
DemonstrationDemonstrationPairing Batch FilePairing Batch File
ref: IVP Batch Program User’s Manual for Pairing
![Page 25: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/25.jpg)
2525
Example (buildpairs_template.bat):Example (buildpairs_template.bat):
DemonstrationDemonstrationPairing Batch FilePairing Batch File
# Pair all locations. Use default settings of # ALL for all other location specifying commandslocation = ALL
# Build pairs for a two week window prior to # current system timestart_time = "* - 14 days"end_time = "*"
# Parameters pairing_window = 1obs_type = RAW
build_pairs = true
ref: IVP Batch Program User’s Manual for Pairing
![Page 26: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/26.jpg)
2626
Commands:Commands:– Time Window: START_TIME, END_TIMETime Window: START_TIME, END_TIME– Specify Locations: Specify Locations: DUR*DUR*, , EXTREMUM*EXTREMUM*, FCST_TS, PE, FCST_TS, PE
Action:Action:– Compute statistics: NATLSTATSCompute statistics: NATLSTATS
Recommendation*:Recommendation*:– Set the national flag for all locations that are part of the Set the national flag for all locations that are part of the
national program to ‘Y’ (use Vfyruninfo Editor)national program to ‘Y’ (use Vfyruninfo Editor)– Give the NATLSTATS action a value of NATIONALGive the NATLSTATS action a value of NATIONAL
DemonstrationDemonstrationNational Statistics Batch FileNational Statistics Batch File
ref: IVP Batch Program User’s Manual for Verification
![Page 27: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/27.jpg)
2727
Example (natlstats_template.bat):Example (natlstats_template.bat):
DemonstrationDemonstrationNational Statistics Batch FileNational Statistics Batch File
# Compute stats for September, 2003.START_TIME = “2003-09-01 00:00:00”END_TIME = “2003-09-30 23:59:59”
# Default Values are ‘ALL’# FCST_TS = ALL# PE = ALL# DUR = ALL# EXTREMUM = ALL
NATLSTATS = NATIONAL
ref: IVP Batch Program User’s Manual for Verification
![Page 28: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/28.jpg)
2828
Start IVPStart IVPgo vsys_scriptsgo vsys_scripts
ivpivp
Specify parameters of a verification groupSpecify parameters of a verification group– Analysis period and intervalsAnalysis period and intervals– Lead time period and intervalsLead time period and intervals– Issuance time period and intervals*Issuance time period and intervals*– Miscellaneous parametersMiscellaneous parameters
DemonstrationDemonstrationIVP: Verification Group ManagerIVP: Verification Group Manager
ref: IVP User’s Manual, Section 9
![Page 29: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/29.jpg)
2929
Specify verification locations that comprise a Specify verification locations that comprise a groupgroup– Filter available locations*Filter available locations*– Choose locationsChoose locations– Specify categoriesSpecify categories– Specify observation type (from which pairs table Specify observation type (from which pairs table
to acquire data)*to acquire data)*
DemonstrationDemonstrationIVP: Verification Location ManagerIVP: Verification Location Manager
ref: IVP User’s Manual, Section 13
![Page 30: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/30.jpg)
3030
Create IVP Data DisplayCreate IVP Data Display– Loading data progress windowLoading data progress window
Open the Verification Group Manager and Open the Verification Group Manager and Verification Location ManagerVerification Location Manager
Save the imageSave the image
DemonstrationDemonstrationIVP: IVP Data DisplayIVP: IVP Data Display
ref: IVP User’s Manual, Section 15
![Page 31: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/31.jpg)
3131
Examine data through two views:Examine data through two views:– Scatter Plot ViewScatter Plot View
Select a location*Select a location* – In OB7.2, use Show button in Verification Location ManagerIn OB7.2, use Show button in Verification Location Manager
Display info about a location*Display info about a location* Line x = yLine x = y
– Time Series View*Time Series View* Specify time series to viewSpecify time series to view Emphasize a time seriesEmphasize a time series
DemonstrationDemonstrationIVP: IVP Data DisplayIVP: IVP Data Display
ref: IVP User’s Manual, Sections 15.1, 15.2
![Page 32: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/32.jpg)
3232
Draw a selected region rectangleDraw a selected region rectangle Zoom and the navigator panelZoom and the navigator panel
– Zoom in on scatter viewZoom in on scatter view– Zoom in on time series view*Zoom in on time series view*
Panning through the time series by zoomingPanning through the time series by zooming
– Shift the zoomed region via the navigator panel Shift the zoomed region via the navigator panel and keyboard*and keyboard*
– Zoom outZoom out
DemonstrationDemonstrationIVP: IVP Data DisplayIVP: IVP Data Display
ref: IVP User’s Manual, Sections 15.6, 15.7
![Page 33: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/33.jpg)
3333
View Pairs via the Verify Pairs ViewerView Pairs via the Verify Pairs Viewer– Select pairsSelect pairs– Sort tableSort table– Create a pairs fileCreate a pairs file
WARNING: Do not view very large WARNING: Do not view very large (30,000+) sets of pairs… the table rendering (30,000+) sets of pairs… the table rendering may be too slowmay be too slow
DemonstrationDemonstrationIVP: IVP Data DisplayIVP: IVP Data Display
ref: IVP User’s Manual, Section 16
![Page 34: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/34.jpg)
3434
Mark pairs as good or bad via IVP Data Display*Mark pairs as good or bad via IVP Data Display* Edit pairs via the Verify Pairs Editor*Edit pairs via the Verify Pairs Editor*
– Select pairsSelect pairs– Mark pairs as good or badMark pairs as good or bad– Edit forecast or observed value manuallyEdit forecast or observed value manually– Focus on a pairFocus on a pair– Edit forecast or observed value graphicallyEdit forecast or observed value graphically
Scatter viewScatter view Time series viewTime series view
Change the mouse mode*Change the mouse mode*
WARNING: Do not select large sets (1000+) of pairs for WARNING: Do not select large sets (1000+) of pairs for editing… the table rendering speed may be too slowediting… the table rendering speed may be too slow
DemonstrationDemonstrationIVP: IVP Data DisplayIVP: IVP Data Display
ref: IVP User’s Manual, Sections 15.11, 17
![Page 35: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/35.jpg)
3535
Open the Plot Definition ManagerOpen the Plot Definition Manager Select statistics Select statistics
– IVP Statistic Chooser ManagerIVP Statistic Chooser Manager Select plot typeSelect plot type Select x-axis variableSelect x-axis variable Select comparison variable*Select comparison variable* Edit statistics parameters*Edit statistics parameters* Edit group parameters*Edit group parameters*
– Spawns a window nearly identical to Verification Group ManagerSpawns a window nearly identical to Verification Group Manager
Note how variable selection affects analysis, lead time, and issuance Note how variable selection affects analysis, lead time, and issuance time-of-day intervalstime-of-day intervals
Note how variable selection affects category choice boxesNote how variable selection affects category choice boxes
DemonstrationDemonstrationIVP: Verification Plot Definition ManagerIVP: Verification Plot Definition Manager
ref: IVP User’s Manual, Sections 18, 19
![Page 36: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/36.jpg)
3636
ErrorsErrors Categorical StatisticsCategorical Statistics Quantiles and ExtremesQuantiles and Extremes
– Computed relative to whichever variable whose category used is Computed relative to whichever variable whose category used is set to “Do Not Use”set to “Do Not Use”
Moments*Moments* Sample SizeSample Size
– The number of pairs used in the computations The number of pairs used in the computations ––or–or–
– Number of pairs within the category (if x-axis or comparison Number of pairs within the category (if x-axis or comparison variable is forecast or observed categories)variable is forecast or observed categories)
Misc.*Misc.* Special Plots*Special Plots*
DemonstrationDemonstrationIVP: The StatisticsIVP: The Statistics
ref: IVP Batch Builder User’s Manual, Appendix B
![Page 37: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/37.jpg)
3737
Categorical Statistics use 3x3 Contingency TableCategorical Statistics use 3x3 Contingency Table
Examples:Examples:– HFAR = D/(D + E + F)HFAR = D/(D + E + F)– TFAR = (D + F)/(D + E + F)TFAR = (D + F)/(D + E + F)– CSI = (A+B+D+E)/(A+B+D+E+G+H+C+F) CSI = (A+B+D+E)/(A+B+D+E+G+H+C+F)
In a 2x2 contingency tableIn a 2x2 contingency table– HFAR is always 0 for the lowermost categoryHFAR is always 0 for the lowermost category– TFAR and HFAR are equivalent for uppermost categoryTFAR and HFAR are equivalent for uppermost category
POD + UFR + OFR = 1 in all casesPOD + UFR + OFR = 1 in all cases
DemonstrationDemonstrationIVP: The StatisticsIVP: The Statistics
ref: IVP Batch Builder User’s Manual, Appendix B
Observed was...
Forecast was... below within above
below A B C
within D E F
above G H I
![Page 38: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/38.jpg)
3838
DemonstrationDemonstrationIVP: Verification Plot Definition ManagerIVP: Verification Plot Definition Manager
x-axis
primary y-axis
comparisonvariable
secondary y-axis
Statisticshort name(matches outputfiles)
![Page 39: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/39.jpg)
3939
Choose categories usedChoose categories used– Disabled if x-axis or comparison variable is forecast or Disabled if x-axis or comparison variable is forecast or
observed categoriesobserved categories– Select “Do Not Use”Select “Do Not Use”
Only one of the two choice boxes can be “Do Not Use”Only one of the two choice boxes can be “Do Not Use”
– Impact of selections on how statistics are computedImpact of selections on how statistics are computed Category used restricts pairs used in computations to only Category used restricts pairs used in computations to only
those within the categorythose within the category Some statistics are calculated relative to the variable not used Some statistics are calculated relative to the variable not used
to categorize data (e.g. quantiles, to categorize data (e.g. quantiles, cdf plot*cdf plot*, , pdf plot*pdf plot*))
– Special case: ROC plot*Special case: ROC plot* Category used defines observation thresholdCategory used defines observation threshold
DemonstrationDemonstrationIVP: Verification Plot Definition ManagerIVP: Verification Plot Definition Manager
ref: IVP User’s Manual, Section 18.7
![Page 40: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/40.jpg)
4040
Create IVP Statistic DisplayCreate IVP Statistic Display– Data is always loaded from memory even if the Data is always loaded from memory even if the
overridden group parameters imply additional data is overridden group parameters imply additional data is needed (such as a wider analysis interval, for example)needed (such as a wider analysis interval, for example)
– Allows for relatively quick plot generationAllows for relatively quick plot generation
Save the imageSave the image View the data via the IVP Statistics Data ViewerView the data via the IVP Statistics Data Viewer
– Create an ASCII format image data fileCreate an ASCII format image data file
DemonstrationDemonstrationIVP: IVP Statistic DisplayIVP: IVP Statistic Display
ref: IVP User’s Manual, Section 21
![Page 41: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/41.jpg)
4141
Create a batch fileCreate a batch file– Change the analysis period*Change the analysis period*– Change the included locations*Change the included locations*– Create other groups for which to generate Create other groups for which to generate
products*products*– Specify output and template file namesSpecify output and template file names
Replacement strings are needed if images are to be Replacement strings are needed if images are to be produced for multiple groups in order to avoid produced for multiple groups in order to avoid overwriting output files*overwriting output files*
– Specify batch file nameSpecify batch file name
DemonstrationDemonstrationIVP: IVP Statistics DisplayIVP: IVP Statistics Display
ref: IVP User’s Manual, Section 23
![Page 42: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/42.jpg)
4242
Open up the Chart Property ManagerOpen up the Chart Property Manager– Click on menu item or tool bar itemClick on menu item or tool bar item– Click on plot component*Click on plot component*
Change labelsChange labels Change axis limits or chart dimensionsChange axis limits or chart dimensions Change legend componentsChange legend components Change x-axis labels (for IVP Statistic Display)Change x-axis labels (for IVP Statistic Display) Create a template fileCreate a template file
– Don’t modify manually unless absolutely necessaryDon’t modify manually unless absolutely necessary Apply changes from a template fileApply changes from a template file
DemonstrationDemonstrationIVP: Chart Property ManagerIVP: Chart Property Manager
ref: IVP User’s Manual, Sections 24, Appendix D
![Page 43: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/43.jpg)
4343
DemonstrationDemonstrationIVP: Chart Property ManagerIVP: Chart Property Manager
X-Axis Title
Y-Axis 1Title
Legend
Y-Axis 2Title
LegendTitle
Tick Mark Font
Plot Title
![Page 44: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/44.jpg)
4444
Start IVP Batch BuilderStart IVP Batch Builder Load batch fileLoad batch file Insert, append, delete command linesInsert, append, delete command lines Edit command linesEdit command lines Comment and uncomment command linesComment and uncomment command lines Check for errorsCheck for errors Run batch fileRun batch file
DemonstrationDemonstrationIVP Batch Builder and the IVP Batch ProgramIVP Batch Builder and the IVP Batch Program
ref: IVP Batch Builder User’s Manual
![Page 45: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/45.jpg)
4545
Create one batch file to generate products Create one batch file to generate products for many locationsfor many locations– @FILE Command to refer to other files@FILE Command to refer to other files– Replacement strings to avoid overwriting output Replacement strings to avoid overwriting output
filesfiles
DemonstrationDemonstrationIVP Batch Builder and the IVP Batch ProgramIVP Batch Builder and the IVP Batch Program
ref: IVP Batch Builder User’s Manual
![Page 46: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/46.jpg)
4646
Organize output by using subdirectories of $Organize output by using subdirectories of $(vsys_output)*(vsys_output)*– If producing graphics for each location, use If producing graphics for each location, use
@ONELID or @ONELIDSHEF to define @ONELID or @ONELIDSHEF to define directories: directories:
OUTPUT_FILE = “@ONELID/output.txt,c”OUTPUT_FILE = “@ONELID/output.txt,c”– If producing graphics for time periods, use If producing graphics for time periods, use
@STARTTIME or @ENDTIME:@STARTTIME or @ENDTIME:OUTPUT_FILE = OUTPUT_FILE =
“@STARTTIME{[CCYY]}/output.txt,c”“@STARTTIME{[CCYY]}/output.txt,c”
DemonstrationDemonstrationIVP Batch Builder and the IVP Batch ProgramIVP Batch Builder and the IVP Batch Program
ref: IVP Batch Builder User’s Manual
![Page 47: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/47.jpg)
4747
To minimize time spent querying database, load all needed To minimize time spent querying database, load all needed pairs at oncepairs at once– Only specify the overall analysis period and choose the locations Only specify the overall analysis period and choose the locations
within the Verification Group Manager and Verification Location within the Verification Group Manager and Verification Location ManagerManager Specify other parameters via the Plot Definition version of the Specify other parameters via the Plot Definition version of the
Verification Group Manager (click on Edit Group Parameters in the Plot Verification Group Manager (click on Edit Group Parameters in the Plot Definition Manager)Definition Manager)
– Important when generating many productsImportant when generating many products To minimize the chart rendering time for the IVP Data To minimize the chart rendering time for the IVP Data
Display, keep the number of pairs smallDisplay, keep the number of pairs small– Do one location at a time for a couple yearsDo one location at a time for a couple years– Do several locations at a time for one monthDo several locations at a time for one month– Important when quality controlling pairs dataImportant when quality controlling pairs data
See the Memory Limitations Tips later for recommended See the Memory Limitations Tips later for recommended numbers of pairs to loadnumbers of pairs to load
Tips:Tips:Verification Group Manager UsageVerification Group Manager Usage
![Page 48: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/48.jpg)
4848
Use OBS_TYPE “RAW” for paired observations Use OBS_TYPE “RAW” for paired observations if…if…– Observed data was posted to archive database via raw Observed data was posted to archive database via raw
SHEF decoder, SHEF decoder, ANDAND– First letter of type source is ‘R’.First letter of type source is ‘R’.
Use OBS_TYPE “PROCESSED” if…Use OBS_TYPE “PROCESSED” if…– Observed data was posted to archive database via Observed data was posted to archive database via
processed SHEF decoder, processed SHEF decoder, OROR– First letter of type source is not ‘R’First letter of type source is not ‘R’
Example:Example:– If processed observations were posted via raw SHEF If processed observations were posted via raw SHEF
decoder (to peoosep table), then specify PROCESSEDdecoder (to peoosep table), then specify PROCESSED
Tips:Tips:Deciding Between PROCESSED or RAWDeciding Between PROCESSED or RAW
![Page 49: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/49.jpg)
4949
Specifies parameters of graphical user interfacesSpecifies parameters of graphical user interfaces– FontsFonts– Window SizesWindow Sizes– ColorsColors– OthersOthers
LocationLocation– Default:Default:
$(get_apps_defaults vsys_dir)/app-defaults/*_SYSTEM_FILE.txt$(get_apps_defaults vsys_dir)/app-defaults/*_SYSTEM_FILE.txt
– Directory can be overridden by exporting environment variable Directory can be overridden by exporting environment variable SYSTEM_SETTINGS_FILE_DIRSYSTEM_SETTINGS_FILE_DIR Can allow for user specific system settingsCan allow for user specific system settings
FormatFormat– ESPADP/IVP batch language standard “<token> = <value>” formatESPADP/IVP batch language standard “<token> = <value>” format
Tips:Tips:System Settings FilesSystem Settings Files
ref: IVP User’s Manual, IVP Batch Builder User’s Manual, Vfyruninfo Editor User’s Manual, Appendix A
![Page 50: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/50.jpg)
5050
Tips: Tips: Memory LimitationsMemory Limitations
Recommended number of pairs:Recommended number of pairs:
– Limits can be changed by using the ivp and ivpbatch script –m and –s Limits can be changed by using the ivp and ivpbatch script –m and –s options (run “ivp –hh” or “ivpbatch –hh” to read about these options)options (run “ivp –hh” or “ivpbatch –hh” to read about these options)
– Limits above are for IVP GUI; IVP Batch Program can deal with more Limits above are for IVP GUI; IVP Batch Program can deal with more pairs because not all pairs need to be stored in memory at one timepairs because not all pairs need to be stored in memory at one time
If IVP GUI runs out of memory, an error message will pop-up and If IVP GUI runs out of memory, an error message will pop-up and the pairs will be discarded (in most cases)the pairs will be discarded (in most cases)– The software may crash in other casesThe software may crash in other cases
Verify Pairs Viewer window is a memory hogVerify Pairs Viewer window is a memory hog– Add about 25% to the recommended value if not using the ViewerAdd about 25% to the recommended value if not using the Viewer
ref: IVP User’s Manual, IVP Batch Builder User’s Manual, Vfyruninfo Editor User’s Manual, Appendix A
System pairs QC product generation Approx. MaximumAX 20,000 20,000 40-60,000LX 100,000 400,000 600-800,000
Recommendation
![Page 51: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/51.jpg)
5151
Debug problemsDebug problems– Export vsys_debug token prior to execution to see additional outputExport vsys_debug token prior to execution to see additional output
Useful levels are 0 (no output), 1, 20, 40, 60, 80, 100, 120, 128.Useful levels are 0 (no output), 1, 20, 40, 60, 80, 100, 120, 128. ““export vsys_debug=20” to see additional progress outputexport vsys_debug=20” to see additional progress output ““export vsys_debug=40” to see database query where clauses and export vsys_debug=40” to see database query where clauses and
timing results for chart renderingtiming results for chart rendering ““export vsys_debug=128” to see everything (lots and lots of output)export vsys_debug=128” to see everything (lots and lots of output)
PerformancePerformance– If IVP Data Display chart rendering is slow…If IVP Data Display chart rendering is slow…
Reduce number of pairs viewedReduce number of pairs viewed Zoom in on regions (points outside current zoom area are not Zoom in on regions (points outside current zoom area are not
rendered)rendered) Run on faster machines (port code to LX and execute software there)Run on faster machines (port code to LX and execute software there)
– If database query speed is slow…If database query speed is slow… Turn off SHEF decoders for a short time (if you can)Turn off SHEF decoders for a short time (if you can) Reduce number of locations or width of analysis periodReduce number of locations or width of analysis period
Tips:Tips:Debugging and PerformanceDebugging and Performance
ref: IVP User’s Manual, Section 25
![Page 52: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/52.jpg)
5252
Future PossibilitiesFuture Possibilities Confidence IntervalsConfidence Intervals
– Algorithms need to be developedAlgorithms need to be developed– Ability to produce CI values and plot them needs to be addedAbility to produce CI values and plot them needs to be added
GUI redesignGUI redesign– Reduce the number of windowsReduce the number of windows– Reorganize window so that the Verification Group Manager does not need Reorganize window so that the Verification Group Manager does not need
to be in two placesto be in two places Add ability to handle multiple comparison variablesAdd ability to handle multiple comparison variables Add ability to modify other properties of a chartAdd ability to modify other properties of a chart
– Tick spacing, line style (solid, dashed, etc), line width, etc.Tick spacing, line style (solid, dashed, etc), line width, etc.– Requires additions to Chart Property ManagerRequires additions to Chart Property Manager
Create baseline batch files requiring no changes in order to produce Create baseline batch files requiring no changes in order to produce standard graphics standard graphics – Standard graphics need to be determinedStandard graphics need to be determined
Add ability to put changed forecasts and observed values back into Add ability to put changed forecasts and observed values back into original data tables and rebuild all affected pairs on-the-flyoriginal data tables and rebuild all affected pairs on-the-fly– IVP would become a more powerful, general QC tool for archive dataIVP would become a more powerful, general QC tool for archive data
![Page 53: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/53.jpg)
5353
DocumentationDocumentation
Available on the webAvailable on the web
http://www.nws.noaa.gov/oh/hrl/verification/verification_doc_ob82.phphttp://www.nws.noaa.gov/oh/hrl/verification/verification_doc_ob82.php
![Page 54: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/54.jpg)
5454
ExercisesExercises
Described in hand-outsDescribed in hand-outs– The data sets are very large, so while the pairs are The data sets are very large, so while the pairs are
loading, discuss the plots that are needed in order to loading, discuss the plots that are needed in order to answer the exercise questionsanswer the exercise questions What statistics are to be displayed?What statistics are to be displayed? What are the x-axis and comparison variables?What are the x-axis and comparison variables? Are there any observed or forecast category restrictions?Are there any observed or forecast category restrictions?
Group assignmentsGroup assignments
Group LX Machine Exercise 1 Location Exercise 2 Location1 lx1 BLKO2 TIFM72 lx3 GLOO2 WTTO23 lx4 MORA4 TIFM74 lx5 QUAO2 WTTO25 lx6 WTTO2 TIFM7
![Page 55: Interactive Verification Program Training for AWIPS OB8.2 Hank Herr Office of Hydrologic Development RFC Verification Workshop, 08/15/2007](https://reader036.vdocument.in/reader036/viewer/2022062500/5697bfef1a28abf838cba51c/html5/thumbnails/55.jpg)
5555
DiscussionDiscussion
Review exercisesReview exercises Pre-workshop questionsPre-workshop questions
– Have they been answered?Have they been answered?
General comments on softwareGeneral comments on software