analysis of post-out 2007 census test analysis of post-out garnett compton field design manager,...

2007 Census Test Analysis of post-outAnalysis of post-out

Garnett ComptonGarnett Compton

Field Design Manager, 2011 CensusField Design Manager, 2011 Census

1. Setting the scene – Post-out and the 2007 Census Test

2. Differences in response

3. Quality of the address register

4. Costs

5. Other quality and operational impacts

6. Conclusions and questions

Overview

Why consider Post-out?

• To reduce serious risks experienced in 2001, in particular the failure to recruit a large number of enumerators.

• To provide savings to invest in improving response.

• Because of the limited success of making contact at delivery.

2007 Census Test – High-level Design

• Address checking

- Conducted in all Test areas during Sept and October

- Split discretionary and full contact methods• Delivery

- 50% Post-out, 50% hand delivery

- For hand delivery 3 attempts at contact over 2 week period• Collection/Follow-up

- Central post-back- 23 May – 22 June- 3 attempts everywhere

- Reminder letter to all outstanding addresses as at 31 May

2007 Census Test – Some caveats ….

• Voluntary:

Relied on public’s good will to complete a return.

• Publicity:

Pre-delivery information cards.

• Sample:

Skewed to harder to enumerate areas.

• Follow-up:

Fixed number of follow-up attempts everywhere.

Household response rates by delivery method and ETC

ETC Hand delivery

Post-out Difference P value

Total 53.4% 50.6% 2.8% <1%

1 66.9% 63.4% 3.6% 1%

2 55.7% 51.2% 4.5% 4%

3 47.8% 44.7% 3.1% 9%

4 36.8% 37.0% -0.2% 54%

5 33.8% 29.3% 4.5% 1%

Success rates at follow-up by delivery method by ETC

ETC Hand delivery

Post-out

1 37.1% 35.4%

2 27.0% 26.9%

3 23.6% 22.2%

4 16.5% 17.6%

5 14.9% 13.6%

Overall 26.0% 25.8%

Address register coverage

• 680 (1.3%) new addresses found during enumeration in hand delivery areas

– Nearly 70% of new addresses were sub-premise addresses – suggest existed at time of AC.

– About 20% found already existing/available latest update.

– About 1/6 found in hand delivery areas during follow-up

Comparative costs

Developed a cost model with three key parameters:

• Percentage mix of delivery method;• Differences in initial return rates (i.e. amount

of follow-up); and• Success rates at each follow-up visit.

Estimated Cost Savings

Estimated cost savings between 100% post-out and 100% hand delivery

Initial return* rate difference

(%’age points)

Estimated savings

5 £28m - £35m

6 £25m – £33m

10 £6m - £21m

15 -£18m - £1m

* At the start of follow-up – 23 May

Quality and operational impacts

• No difference in under/over count between two methods

• No difference in number failing 2 of 4 rule• No large difference in age/sex distributions

between delivery methods• 50% more calls to the contact centre in post-

out areas• CTES:

– No difference in views on “junk” mail– Small difference recognised as “official” mail

Conclusions

• Post-out has an impact on return rates, but not success at follow-up. Post-out requires more follow-up to obtain same overall response rate.

• Differences in return rates are not affected by the hard to count characteristics of an area (i.e. the ETC).

• No significant differences in response quality• A post-out methodology will allow savings to invest in

targeted follow-up and community liaison.• The levels of AR undercoverage will be small with

minimal, but manageable, impact on the overall quality.

• Some operational impacts but manageable through design and development

Conclusions cont ….

Therefore:• Post-out will be the primary means of

delivering questionnaires in 2011;• Approximately 95% of England and Wales

Supporting post-out for 2011

• Revised addressing strategy• Targeted address checking• Working with the postal service provider• Publicity• Operational Intelligence (questionnaire

tracking)

Thank you

Any Questions?