11 th national planning applications conference topic: statewide modeling validation measures and...

11th National Planning Applications Conference

Topic: Statewide Modeling Validation Measures and Issues

Authors: Dave Powers, Anne Reyner, Tom Williams, Paul Hershkowitz, Rob Bostrom

Date: May 9, 2007

Modeling Issues

• Statewide Modeling Objectives• Forecast major intercity volumes and roadway

deficiencies• Estimate urban external-internal and through

volumes• Freight transport growth and potential for

intermodal diversion• Evaluate major travel corridor improvements

– Urban bypass– Intercity corridor

Modeling Issues

• Off- Model Traffic• Urban intrazonal / local private vehicle• Rural local private vehicle• Freight intra-county • Commercial vehicles

Modeling Issues

• Observed Traffic Counts• Primary independent validation criterion• Vehicle class not universally available• Estimated traffic volumes• Geographically extrapolated traffic volumes

Modeling Issues

• Travel Demand Survey Data - Person• National Samples – not necessarily representative

at the state level• Not comparable to traffic counts• Add-on samples can expand usefulness• Only source of intercity trips

Modeling Issues

• Travel Demand Inventory - Freight• Inter-county, large vehicle only• Requires extensive review for reasonableness• Requires conversion of tonnage to vehicles• Requires estimation of empty vehicle movements• Comparison to counts problematic

Modeling Issues

• Data Checking – Traffic Counts• Focus on rural higher functional classes• Eliminate urban influence• Review spatial and temporal consistency• Eliminate inconsistent and/or estimated/transferred

counts

2003 vs 2006 Counts

Counts Comparision 2003 - 2006

-60%

-40%

-20%

0%

20%

40%

60%

10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000

2003 Count

Gro

wth

20

03

- 2

00

6

Modeling Issues

• Data Checking – Person Travel Surveys• Review sample size by household categories• Compare trip rates, distribution characteristics

across household categories• Supplement with similar state data if necessary

Modeling Issues

• Data Checking – Freight Flow Inventory• Compare flow origins/destinations with

development inventories• If available, compare multiple year data• Major flow desire lines - review with knowledgeable

state staff• Review load factors• Review implied empty backhaul factors• Assign and compare VMT and traffic on major

roadways

Modeling Issues

• Validation Reasonableness Tests – Person Travel

• Statistical correlation of model dependent and independent variables

• Household trip rates• Proportion of trips by purpose / length• Average trip length / trip length distribution• Compared against model development data

Modeling Issues

• Validation Objective Tests – Person Travel• VMT by major functional roadway class• Volumes across major rural screenlines• Volumes at major state boundary crossings• Volumes along Interstate and other major

roadways• RMSE and individual count deviation analysis• Identification and explanation of outliers

Validation Screenlines - Mississippi

Validation Screenlines - Kentucky

Modeling Issues• Validation Reasonableness Tests–Freight

• Statistical correlation of model dependent and independent variables

• Tonnage by commodity type, and internal/external orientation

• Replication of special generator origins/destinations

• Inter-district flows• Average trip length / trip length distribution• Load factors / empty backhaul rates• Compared against model development data

Modeling Issues

• Validation Objective Tests – Freight • VMT by major functional roadway class• Volumes across major rural screenlines• Volumes at major state boundary crossings• Volumes along Interstate and other major

roadways• RMSE and individual count deviation analysis• Identification and explanation of outliers

Percent Deviation for Validation Links - Mississippi

Figure VII-2Percent Deviation For Rural Links with Counts

MS Statewide 2000 Transportation ModelN=1751

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

- 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0

Traffic Counts (Thousands)

Per

cen

t D

evia

tio

n

Maximum Desirable Deviation

Percent Deviation for Validation Links - Kentucky

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

- 10.0 20.0 30.0 40.0 50.0 60.0 Counts (Thousands)

Per

cen

t D

evia

tio

n

Maximum Desirable Deviation

Modeling Issues

• Validation Expectations• Recognize limitations of source data• Focus on rural areas / higher functional classes• Quantitative comparisons will be less accurate

than experienced with urban models• Validation quality may vary depending on model

complexity, state development/road system complexity and traffic counting system

Modeling Issues

• Lessons Learned• Spend time and effort to fully understand

assumptions and estimation procedures present in traffic counts

• Spend time and effort to get input person and freight data as free from logical inconsistencies as possible

• Focus model calibration and objective validation tests on facility types/volume groups where a statewide model can be reasonably expected to perform accurately