2011 census 2007 census test – emerging findings garnett compton, ons updated 4 september 2007...
TRANSCRIPT
![Page 1: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/1.jpg)
2011 Census2007 Census Test – emerging findings
Garnett Compton, ONS
Updated 4 September 2007
BSPS – 12 September 2007
![Page 2: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/2.jpg)
Session Aim
Aim:
• To share emerging findings from the 2007
Census Test• Outline plan for publishing 2007 Census Test
evaluation• Present a brief update on other 2011 Census
developments
![Page 3: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/3.jpg)
Contents
• Test background• Results:
• Address register development• Post-out/hand delivery• Income question
• Key findings of 2007 Test• Further evaluation and publication• Did you know ….
![Page 4: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/4.jpg)
2007 Census Test - Objectives
Test objectives:• Assess the effect on response of:
– Inclusion of an income question; and– the use of post-out to deliver questionnaires.
• Assess the feasibility of major innovations in proposed 2011 Census operational procedures:
– Outsourcing of field staff recruitment, pay and training;– development of an operational intelligence system to
enable individual questionnaires to be tracked; and,– development of an address list and address checking
procedures.
![Page 5: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/5.jpg)
2007 Census Test - Sample sizes by LA and ETC
0
5000
10000
15000
20000
25000
30000
35000
40000
Cam
den
L'pool
Stoke
Bath &
NE
S
Car'shire
5
4
3
2
1
ETC
![Page 6: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/6.jpg)
2007 Census Test – High-level Design
• Address checking– Conducted in all Test areas during Sept and October– Split discretionary and full contact methods– Used controlled errors
• Delivery – 50% Post-out, 50% hand delivery– 50% questionnaires income, 50% no income – For hand delivery 3 attempts at contact over 2 week period
• Collection/Follow-up– Central post-back– 23 May – 22 June– 3 attempts everywhere– reminder letter to all outstanding addresses as at 31 May
![Page 7: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/7.jpg)
2007 Census Test – Key constraints
Key constraints affecting evaluation of the Test:
• Voluntary– Relied on public’s good will to complete a return
• Publicity• Sample – skewed to harder to enumerate areas• Follow-up
– Fixed number of follow-up attempts everywhere
![Page 8: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/8.jpg)
Results
![Page 9: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/9.jpg)
Household* return rates by LA
0%
10%
20%
30%
40%
50%
60%
70%
Cam
den
L'pool
Stoke
Bath &
NE
S
Car'shire
All areas
* As at 15 July
![Page 10: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/10.jpg)
Household* return rates by ETC
0%
10%
20%
30%
40%
50%
60%
70%
ETC* As at 15 July
![Page 11: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/11.jpg)
Address register development
Results
![Page 12: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/12.jpg)
Key results
• Address checking (AC) found about 12% new addresses– Time consuming to validate and update AR
• 4.4% (4,400) of questionnaires undelivered– Of which 17% were added by address checkers– Reasons include vacant properties, non-existing
addresses, duplicate addresses.
• 1,200 new addresses found during enumeration– Of which, over 50% found at follow-up.– Nearly 70% of new addresses were sub-premise
addresses – suggest existed at time of AC.
![Page 13: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/13.jpg)
Early conclusions
• Suggests we need to plan to do a 100% address check in 2011 no matter what the delivery design.
• Plan is for a rolling address check over a longer period of time (4-6 months).
– enables better quality address checking; and,– enables more time to update the final address register
with Address check findings.
• Consider a re-address check shortly before the Census in a small %’age of areas
![Page 14: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/14.jpg)
Early conclusions cont …
• Main enumeration and controlled errors didn't identify all missing addresses – need to improve methods and guidance
• Issues around whether information with address suppliers can be shared;
• Criteria for deciding which address products to use as a base under consideration.
![Page 15: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/15.jpg)
Delivery Method
Results
![Page 16: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/16.jpg)
Household return rates* by delivery method by ETC
0%
10%
20%
30%
40%
50%
60%
70%
80%
1 2 3 4 5 All areasETC
Post-out Hand delivery * As at 15 July
![Page 17: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/17.jpg)
Success rates* at follow-up by delivery method by ETC
* As at 15 July
ETC Post-out Hand delivery
1 38.7% 39.9%
2 29.1% 29.4%
3 24.2% 26.1%
4 19.1% 18.4%
5 14.9% 16.3%
Overall 23.9% 24.3%
![Page 18: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/18.jpg)
Delivery method – estimated cost savings
Estimated cost savings between 100% post-out and 100% hand delivery
Initial return* rate difference
(%’age points)
Estimated savings
5 £28m - £35m
6 £25m – £33m
10 £6m - £21m
15 -£18m - £1m
* At the start of follow-up – 23 May
![Page 19: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/19.jpg)
Address register coverage
New addresses found during the 2007 Census Test by delivery method
During delivery During follow-up Total
Delivery method
No. %’age No. %’age No. %’age
Post-out 9 0.02% 478 0.94% 487 0.95%
Hand delivery 540 1.06% 181 0.36% 721 1.42%
Total 549 0.54% 659 0.65% 1,208 1.19%
![Page 20: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/20.jpg)
Delivery method - conclusions
• Post-out has an impact on return rates, minimal impact which can be addressed through additional follow-up.
• A post-out methodology will provide significant savings (£25-£33m) to invest in targeted follow-up and community liaison.
• Improvements identified for the address register and follow-up procedures suggest that the levels of AR undercoverage will be small and manageable.
Decision: In E&W, post-out will be the primary means (at least 85%) of delivering questionnaires in 2011.
![Page 21: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/21.jpg)
Income – Setting the scene
Including income depends on:• results from the Test; and,• consultation on other topics and relative priority of
income in relation to other demands.
Further analysis required as follows:• Quality and accuracy of responses to income
question• Item imputation rates• Public perception• Other data sources
![Page 22: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/22.jpg)
Household return rates* by income/no income question by ETC
0%
10%
20%
30%
40%
50%
60%
70%
1 2 3 4 5 All areas
Income No Income * As at 15 July
![Page 23: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/23.jpg)
2007 Census Test - Key findings
Delivery method• In E&W, post-out will be the primary method (at
least 85%) of delivering questionnaires in 2011.
Address register development• Address checking will be required for 2011,
currently planning 100% for E&W.
Outsourcing recruitment, training and pay:• Worked well in the Test and are considering
outsourcing for 2011;
![Page 24: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/24.jpg)
Other key findings:
Recruitment:• More difficult than expected in some areas, mostly delivery
enumerators.• Good MI to identify and manage problems early.Training:• Some development issues with e-learning but overall well
received/effective.Pay:• Much simpler system defined thereby avoiding some of the
problems experienced in 2001.• Hourly pay worked well, provided good control and
flexibility; travel expenses still considered cumbersome. Refinement for 2011 required.
![Page 25: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/25.jpg)
Other key findings:
LA Liaison:• Principles and benefits of LA Liaison proven.• Variable engagement across the LAs – some more engaged than
others • Going forward we need to consider:
– methods for achieving more consistency across LAs; – Making it simpler for ONS and LAs– Achieving Chief Exec buy-in.
Follow-up:• Transfer of information held centrally to field staff worked well within the
constraints of the Test.• A good start to follow-up is imperative – need to review start dates.• Organisation and management of field staff worked well, but more
development required on doorstep interaction to convince respondent to respond.
![Page 26: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/26.jpg)
2007 Census Test Evaluation– planned publications
Publication
Statistical Evaluation of the 2007 Census Test
2007 Census Test - Evaluation of the delivery method
2007 Census Test - Evaluation of the Income question
Evaluation of 2007 Census Test – Summary Report
![Page 27: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/27.jpg)
Did you know?
Some other key Census developments:
• Rehearsal - Spring 2009• Route A – contract to be let in January 2008• Route C – start procurement in January 2008• Finalising questionnaire – Spring 2008• White Paper – Autumn 2008
![Page 28: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/28.jpg)
Questions ????
![Page 29: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/29.jpg)
Household return rates* by delivery method by LA
0%
10%
20%
30%
40%
50%
60%
70%
Post-out Hand delivery * As at 15 July
![Page 30: 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007](https://reader036.vdocument.in/reader036/viewer/2022062301/56649ea95503460f94bad233/html5/thumbnails/30.jpg)
Household return rates* by income/no income question by LA
0%
10%
20%
30%
40%
50%
60%
70%
Cam
den
L'pool
Stoke
Bath
Carm
arthen
All areas
Income No Income * As at 15 July