centraal bureau voor de statistiek challenges of redesigning household surveys and maintaining...
TRANSCRIPT
![Page 1: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/1.jpg)
Centraal Bureau voor de Statistiek
Challenges of redesigning household surveys and maintaining output quality
Menno CuppenPaul van der LaanWim van Nunspeet
Statistics Netherlands
Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
May 6, 2010
![Page 2: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/2.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Outline
• Driving forces
• Approach
• Maintaining quality
• Results and perspectives
• Conclusions
![Page 3: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/3.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Driving forces
• Efficiency
• Quality
• Flexibility
Constraints:
• (European) quality / user satisfaction
• Solution for discontinuities
![Page 4: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/4.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Approach
Means:
• More use of registrative data
• More use of CAWI and mixed mode designs
• Integration of surveys in one survey system
• “Re-use” of respondents
• Implementation of advanced estimation
techniques
![Page 5: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/5.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Survey Model
Optimal use of these means in integrated design for household surveys:
• Core module and topic modules
• Well-balanced use of mixed mode (CAWI - PAPI - CATI - CAPI)
![Page 6: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/6.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Core Questionnaire
• Household composition• Labour status• Health• Religion• Voting behaviour• Occupation in employment• Economic sector in employment• Highest level of education completed• Screening, telephone numbers, e-mail adresses
![Page 7: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/7.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Standard mixed mode strategy
CAWI (+ PAPI)
CATI CAPI
If tel.nr. available If not tel.nr. available
25 – 35 %
± 45% ± 35%
55 - 65% 35 - 45%
Refusal No contact
± 5%
![Page 8: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/8.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Mode effects
Design principle: Unimode questionnaire design
• Selection effects
• Context effects
• Satisficing
• Interviewer effects
![Page 9: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/9.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Maintaining output quality
• Explain discontinuities for core
variables
• Quality indicators (framework)
• Pilots and experiments
• Parallel designs
• Time series modelling
![Page 10: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/10.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Quality indicators
• Relevance
• Accuracy
• Timeliness
• Accessibility
• Comparability
• Coherence
![Page 11: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/11.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Results
2009:
• Core questionnaire in Health Survey
• LFS sample size reduction by using data from unemployment offices
• Structural time series estimators for monthly unemployment
• Small area estimators for sub-regional unemployment statistics
![Page 12: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/12.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Results
2010:
• Core questionnaire in LFS
• Mixed mode in LFS (CAPI – CATI)
• Mixed mode in Health Survey (CAWI – CATI – CAPI)
• Topic modules after Health Survey
2011/2012:
• CAWI in LFS
• More topic modules after core questionnaire
![Page 13: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/13.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Parallel designs
LFS [1] CATI
LFS [2] CATI
LFS [3] CATI
LFS [4] CATI
LFS [1] CAPI
2010
Q1 Q2 Q3 Q4
New design
![Page 14: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/14.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Parallel designs
LFS [3] CATI
LFS [4] CATI
LFS [5] CATI
LFS [1] CATI
LFS [2] CATI
LFS [3] CATI
LFS [4] CATI
LFS [1] CAPI
2010
Q1 Q2 Q3 Q4
LFS [2] CATINew design
Old design
![Page 15: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/15.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Parallel designs
LFS [1] CAPI
LFS [2] CATI
LFS [3] CATI
LFS [4] CATI
LFS [5] CATI
LFS [1] CATI
LFS [2] CATI
LFS [3] CATI
LFS [4] CATI
LFS [1] CAPI
2010
Q1 Q2 Q3 Q4
LFS [2] CATINew design
Old design, parallel run
Old design
![Page 16: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/16.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Issues
• Unpredictibility CAWI results and impact on fieldwork planning in a mixed mode design
• Changing geographical distribution CAPI-samples
• Mode effects (methodology & best practices)
• Survey integration
• Creating uni-mode questionnaires
• Households/LFS and CAWI
• Flexibility in operation (methodology – survey case management system)
![Page 17: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/17.jpg)
Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands
Conclusions
• Quality can be maintained
• Efficiency targets met
• Flexible in meeting new national and
European needs
![Page 18: Centraal Bureau voor de Statistiek Challenges of redesigning household surveys and maintaining output quality Menno Cuppen Paul van der Laan Wim van Nunspeet](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f065503460f94c1b655/html5/thumbnails/18.jpg)
Centraal Bureau voor de Statistiek
Challenges of redesigning household surveys and maintaining output quality
Menno Cuppen May 6, 2010
Statistics Netherlands
Challenges of redesigning household surveys and maintaining output qualityStatistics Netherlands