survey design with a focus on model validity

Conducted on behalf of:

Conducted by:

In Association With:PTV DataSource

GeoStatsPB

Chandra BhatMark Bradley

Mary Kay ChristopherKeith Lawton

Survey Design with aFocus on Model Validity

11th National Transportation Planning Applications ConferenceMay 6-10, 2007, Daytona Beach, Florida

Stacey Bricka | NuStats | 512-306-9065 | [email protected] Kermit Wies | CMAP | 312-386-8820 | [email protected]


Conducted by:


GeoStatsPB



Overview of Presentation

Study Environment

Design Approach

Findings


Conducted by:


GeoStatsPB



Study Environment

Geography 8 Counties 4,000 square miles

Population 2,940,007 households

Diversity Residents Languages Travel Modes Political Environment

Changing Technology Telephone Service Travel Demand Modeling

McHenry Lake

Kane

Cook

DuPage

Will

Kendall

Grundy


Conducted by:


GeoStatsPB



Inventory Objectives

Document demographic and travel behavior characteristics of the region

Support update of the regional travel demand model

Serve as inputs for the development of a new regional travel demand model

Model Validity


Conducted by:


GeoStatsPB



Design Phase Objectives

1. Confirm inventory contents

2. Validate budget assumptions Establish “typical” conditions Identify unique conditions, remedies, and

associated costs Evaluate most effective survey

administration options

3. Vet recommendations


Conducted by:


GeoStatsPB



Design Approach

White Papers

Passive Recruitme

nt

Traditional Pilot

Peer Review

Community Outreach

Final Study Design


Conducted by:


GeoStatsPB



Passive Recruitment

Objectives Low-Cost Recruitment Mode Contact Non-Telephone HH

Action Address-based Sample Mailed 1000 Letters with Brochure and

Mail/Web Recruitment Survey

50% received $2 incentive


Conducted by:


GeoStatsPB



Passive Recruitment

Results 53 HH returned surveys (6%)

• 37% were non-telephone households

• 83% received an incentive

• No statistical differences from phone recruits

38 non-participating HH debriefed

• Only 28% recalled packet

• 39% said would have participated if we called them


Conducted by:


GeoStatsPB



Passive Recruitment

Full Study Advance Mailings Included in Full Study

• Mail to Active and Passive Sample

• Capture of Cell-Only HH

• Lower-cost Recruitment

Mailed 10,000 letters to Date

• 225 passive recruits (~10% of all recruits) 4% non-landline HH 1-person/1-worker HH Higher reported use of Auto


Conducted by:


GeoStatsPB



Community Outreach

Objective Identify and ameliorate participation issues

Action Held 4 community group meetings


Conducted by:


GeoStatsPB



Community Outreach

Results Identified Design Improvements Identified Activities to Increase Participation

Full Study Design – Reflected Community Group Input Lower Participation among Target Groups Phase 1: Advance Mailings, Incentives, Tailored

Interviewing, Multiplicity Sampling Phase 2: Community Groups Phase 3: In-Person Interviewing


Conducted by:


GeoStatsPB



Traditional Pilot

Objectives Establish “typical”

conditions Evaluate respondent

reaction

Action Three Pilot Test Areas Three Diary Types Two Survey Lengths Incentives Test


Conducted by:


GeoStatsPB



Traditional Pilot

1-day Place 2-day Place 1-day Activity

Respondent Reaction

64% 48% 49%

Incentives Impact

-- 0.5% 43%

Trip Capture

4.58 3.97 4.10

Interview Lengths

30.6 min 37.0 min 33.7 min


Conducted by:


GeoStatsPB



White Papers

Objectives Recommend Final Study Design Recommend Inventory Elements

Action Identified Primary Data Elements for Pilot Analyzed Pilot Data and Secondary Data

Presented & Discussed Recommendations


Conducted by:


GeoStatsPB



White Papers

Results Inventory Contents Sampling Efficient Data Collection Maximizing Participation

www.nustats.com/chicago


Conducted by:


GeoStatsPB



White Papers

Full Study Identified data elements

• Support Model Update

• Develop New Model

Sampling based on density and transit access Phone/Mail/Web Options


Conducted by:


GeoStatsPB



Peer Review Panel

Objectives Vet Design Recommendations National and Local “Experts”

Action Presented Pilot Results and White Papers Obtained Local Stakeholder Input

Full Day Discussion of Each Design Element


Conducted by:


GeoStatsPB



Final Project Design

11,600 HH

Mix of 1-day and 2-day HH

Year-long Data Collection Effort

300 HH 7-day GPS Component

1,000 SP Follow-up Surveys


Conducted by:


GeoStatsPB



Findings

In-depth Design Phase Was Worth It If Budget Constrained, Secondary Data

Helpful

Peer Review Panel Input Was Invaluable Mix of Local and National

Focus on Model Validity Driving Question for Evaluation Identify Data Elements for Current and Future

Model Updates


Conducted by:


GeoStatsPB



THANK YOU!

Questions or Comments?

survey design with a focus on model validity

Documents