selectivity in the german mobility panel tobias kuhnimhof institute for transport studies,...

Selectivity in the German Mobility Panel

Tobias Kuhnimhof Institute for Transport Studies, University of Karlsruhe

Paris, May 20th, 2005

Institute for Transport Studies, University of KarlsruheMay 20th 2005 - 2

Overview

The German Mobility Panel

MOP-recruitment, non-response and assumptions about selectivity

Additional sources of information to analyse selectivity in the MOP

Some possibilities to analyse selectivity issues

Some findings

– Trustworthiness of CATI-Data

– Who drops-out and who takes part

– The advantages of recruiting households

Conclusions


The MOP-survey: 7-day mobility diary in 3 consecutive years since 1994

7 days Longitudinal survey of individual mobility behavior

The German Mobility Panel

Year 1

3 years Survey of individual developments, transitions, causes and effects

Year 2

Year 3


The multi-stage recruitment process:

Commercial market research CATI

MOP-recruitment CATI

Written declaration of participation

Participation: 1-Week-mobility-diary report, mailing of documents

recruitment (mailing of documents etc.)

+ 1-week-report

+ 3-year-survey

= high participant burden

There is plenty possibility & it is understandable

not to participate in the MOP

Recruitment, Nonresponse and Assumptions about Selectivity

The MOP recruitment: motivating relialable participants


Recruitment, Nonresponse and Assumptions about Selectivity

4.6

6.5

12.9

22.1

44.9

52.5

100

0 10 20 30 40 50 60 70 80 90 100

% Remaining Phonenumbers, Adresses, Participants

MOP-Participation

Returning of documents

Recruitment-CATI

Stratified sampling

Agreement to 2. contact

Initial CATI

Tel.-Nrs. after neutral drop-out

Drop-out in the different stages in the 2002 MOP-recruitment

High respondent burden

High drop-out

Strong selective bias?


Mobility in Population

P

Business trips?

High incomes ?

Stress due to job, education, children?

Large leisure activity spectrum?

Not mobile due to incapacity (e.g. sick)?

Not interested in mobility issues

Common Assumptions about selective error in mobility surveys

MOP-Recruitment, Non-response and Assumptions about Selectivity


MOP-Participation


Recruitment-CATI

Stratified sampling


Initial CATI

Tel.-Nrs. after neutral drop-out

OBJECTIVE: Get to know the drop-outs

BUT: preserve the survey’s continuity

Mobility interview

Enquiry of reasons for declining

Documenta

tion o

f entir

e dro

p-out

Documentation of calling attempts (max. 12)



Core-element of selectivity analysis: Mobility interview in commercial market research CATI

- individual and household socio-economics

- role of the person in the household

- interviewee’s daily obligations (childcare, work etc.)

- individual mobility on a test day (“yesterday”)

- characterizing information about the person’s general mobility behavior

- proxy-data on other household members

Data available about all CATI-interviewees (MOP-participants & drop-outs):



MOP-Participation


Recruitment-CATI

Stratified sampling


Initial CATI

Some possibilities to analyse the selectivity issues

Analysees to understand selective bias

Modeling participation: P(participation)=f(mobility, socio-demografics)

Logit 4

(Sampling Drop-out)

Global drop out logit

Logit 3

Logit 2

Logit 1


All MOP-Participants (1.-,+ 2.-,+ 3.-time participants)

Respondent groups and data availability:

CATI-Interviewees

CATI and recruitment:

First-time MOP-Participants

(CATI-Interviewees + other household members)

CATI-Interviewees, who participate in the MOP

Possibilities to compare mobility data of - different sources (survey methods) - and different samples

Some possibilities to analyse the selectivity issues


Findings: Trustworthiness of CATI-data

Are CATI-responses useful to characterize mobility behavior?

Intrapersonal comparison of general mobility information (CATI) and reported mobility (MOP)


Are CATI-responses useful to characterize mobility behavior?

- in most cases: YES

- the stronger the questions relate to routines, the more reliable is the answer

- active persons tend to overestimate themselves

- less active persons tend to underestimate themselves

- “Extrapolation of normal weekday” – Example: use of travel modes

Daily use of car = car use on 5,4 days per week Daily use of PT = PT use on 4,5 days per week



Differences in cross-sectional mobility figures in CATI and mobility diary data

Comparision of test-day cross-sectional data of CATI vs. MOP



Methodological Differences: The MOP-Report is more exact than the CATI

CATI MOP P > |t|

Estimate of methodological

difference

Share of mobile persons - total 87.50% 91.90% 0.008 ~4.3%

Share of mobile persons – under 60 years 89.9 93.50% 0.06 ~3.6%

Share of mobile persons – over 60 years 82.7 88.60% 0.06 ~6.1%

Trips per mobile person and day 3.07 3.57 < 0.0001 ~0.5

Travel time per mobile person and day [Min] 87.6 85.5 0.68 ~0

Test day data of persons who participated in both surveys*:

*No Fridays, Saturdays, exclusion of daily trips that exceed 6



What is the impact of the sample differences due to selectivity on mobility figures?

Comparison of cross-sectional data CATI – MOP: After accounting for methodological differences the selective error can be estimated

Findings: Who drops out and who takes part


CATI test day data vs. MOP test day data of all persons in both surveys*:

*No Fridays, Saturdays, exclusion of daily trips that exceed 6, appropriate weighting procedures applied in order to account for socio-demographic differences

CATI MOP

Estimate of methodological

difference

Estimate of difference due

to selectivity

Share of mobile persons – total 83.90% 91.70% ~4.3% ~3.5%

Share of mobile persons – test persons under 60 years 88.10% 93.20% ~3.6% ~1.5%

Share of mobile persons – test persons over 60 years 73.60% 88.00% ~6.1% ~8.3%

Trips per mobile person and day 3.09 3.53 ~0.5 0

Travel time per mobile person and day [Min] 80.83 81.58 ~0 0

Drop-Out of Non-trippers

Findings: Who drops out and who takes part


Who drops out and who takes part

Socio-economic aspects of Selectivity

Monthly Household Income

0 0.5 1 1.5 2

<750 Euro (Ref.)

750 - 1500 Euro

1500 - 2000 Euro

2000 - 3000 Euro

3000 - 4000 Euro

4000 - 5000 Euro

> 5000 Euro

Age of Interviewee

0 0.5 1 1.5 2

14-17

18-25

26-35

36-50 (Ref.)

51-60

61-70

>=71

Significant, P<0.1 P<0.2 Not significantP<0.3

Example: Odds ratios of 2 out of 15 variables in participation logit model

“Middle class bias”


Who drops out and who takes part

Explanations for increasing share of mobile persons

Example: Odds ratios of 2 out of 15 variables in participation logit model

“Mobility interest bias”

Distance to work place, place of education etc.

0 0.5 1 1.5 2

0-5 KM

5-20 KM (Ref.)

20-40 KM

>40 KM

Weekday Evening Leisure Activities

0 0.5 1 1.5 2

active(daily)

normal(Ref.)

less active(<2x /

Woche)

Significant, P<0.1 P<0.2 Not significantP<0.3


The advantages of surveying households

CATI-Interviewees

CATI and recruitment:

First-time MOP-Participants

(CATI-Interviewees + other household members)

CATI-Interviewees, who participate in the MOP

Unemployment rate

Distance to work place

4.2% 14.7 Km

2.7% 17.1 Km

3.7 % 14.8 Km

Participation of other household member can counter-balance selectivity impacts


Conclusions and Recommendations

• Socio-economic selective effects dominate drop-out• “Middle Class Bias” good education, good income, middle aged• “Mobility Interest Bias” drop out of non-trippers (particularly elderly,

permanent incapacity?)

Findings on Selectivity

Other Findings

• CATI-data is useful but has to be interpreted• Surveying households counter-balances selectivity

Recommendations

• Balanced recruitment of different mobility styles is vital• Survey households not individuals• Don’t trade data quality for a high response rate

selectivity in the german mobility panel tobias kuhnimhof institute for transport studies,...

Documents

mobility data

transport studies

catiinterviewees cati

selectivity folie

reported mobility mop

data folie

mop recruitment

mobility figures