presented to

presented byCambridge Systematics, Inc.

Comprehensive Validation of Activity-Based ModelsExperiences from Houston-Galveston Area Council’s Activity-Based Model Development

TRB Applications Conference

May 21, 2015

Martin MilkovitsArun KuppamDavid KurthThomas Rossi

Tour-Level Choices

Long-Term Choices

Stop/Trip-Level Choices

H-GAC ABM Structure

2

All Tour Stop Generation and Mode Choice

Tour Generation

Mandatory Tour Destination

and Time of Day

Auto Ownership, Work Location, School Location

Daily Activity Pattern(Including Work/School

Travel)Fully Joint Travel

Stop (Trip)-Level Destination, Time of Day, and Mode Choice

Individual Nonmandatory Travel

School Escorting Model

Joint Tour Destination and Time of Day

Individual Nonmandatory Tour Destination and Time

of Day

3

Validation Approach

Identify key areas of future analysis

Compile validation data

Assure correct model operation

Prevalidation

Population synthesis validation

Single-pass validation (TourCast)

Full-feedback validation

Sensitivity test

Elasticity test

Backcast test

Validation Tests

Lessons Learned

Don’t have eyes bigger than your stomach (for cross classifications)

» The greater the stratification

• The less available/statistically significant observed data

• The more difficult to effectively calibrate

» Avoid information overload

• Threshold screens

• Identify trends using heat-maps

• GIS representations

» Know when to trust the model

• Particularly with ordinal data (age, income, etc.)4

Lessons Learned (continued)

Need consistent expansion factors between synthesized population and validation survey summaries

5

Synthesized PopulationExpanded Survey (Prenormalized)

child1child2child3adult studentftwptwnwasenior

child1child2child3adult studentftwptwnwasenior

Step-wise/iterative process is most robust

» Single-pass before full-feedback

» Logsums create circular dependencies within single-pass

» Iterative adjustments to accommodate multiple stratifications

» Time-intensive process, particularly for models with many segments and/or alternatives

Sensitivity Test to check and demonstrate ABM advantages

» Peak period tolling

» Changing demographic6

Lessons Learned (continued)

Acknowledgements

7

Chris van Slyke, Chi-Ping Lam, Sharon Ju, Heng Wang, David Gao, Michael Onuogu

H-GAC

Vincent Sanders

Houston Metro

Andy Mullins

Texas Transportation Institute

Pat Coleman, Srikanth Neelisetty, Nagaruju Kashayi

AECOM

Vijay Mahal

HDR

8

Vehicle Availability

Stratified by

• County

• HH size

• HH income

• HH size/income

• Workers

• Workers/income

• Children

• Workers/children

• Accessibility

Expanded Household Survey

Model

Percentage Difference (model – survey)/survey

HHIncome HHSize

Total<$200

00

20,000-$39,99

9

$40,000-

$69,999

$70,000-

$99,999

>$100,000

Total 2.05 1.35 1.88 2.37 2.52 2.641 1.10 0.88 1.19 1.29 1.43 1.482 2.05 1.59 1.93 2.16 2.33 2.313 2.57 1.86 2.29 2.82 3.00 3.01

4+ 2.54 1.79 2.37 2.80 2.80 2.92

HH

Siz

e

HHIncome HHSize

Total<$200

00

20,000-$39,99

9

$40,000-

$69,999

$70,000-

$99,999

>$100,000

Total 2.05 1.20 1.73 2.15 2.40 2.591 1.13 0.85 1.13 1.33 1.39 1.472 2.12 1.51 1.88 2.15 2.27 2.393 2.23 1.43 1.88 2.30 2.47 2.65

4+ 2.59 1.61 2.15 2.66 2.85 3.00

HH

Siz

e

HHIncome HHSize

Total<$200

00

20,000-

$39,999

$40,000-

$69,999

$70,000-

$99,999

>$100,000

Total 0% -15% -15% -22% -12% -5%1 3% -3% -6% 4% -3% -1%2 7% -8% -5% -1% -6% 8%3 -34% -43% -41% -52% -53% -35%

4+ 5% -18% -22% -15% 5% 8%

HH

Siz

e

9

Aggregate-Level Results

Initial assignment results comparable (or better!) than current trip-based model after substantial calibration demonstrates value of comprehensive single-pass validationTDM ABM

Percent Difference RMSE Percent Difference RMSE

Freeway -10% 28% -9% 27%

Toll -14% 31% -11% 30%

Principal Arterial 3% 65% 3% 64%

Other Arterial 6% 79% 6% 80%

Collector 11% 121% 11% 121%

All 1% 71% 2% 71%