1

Weighting on National Statistics Household Surveys

Jeremy BartonOffice for National Statistics

2

Outline of talk

• The ONS surveys

• Why should we weight?

• The weighting process

• When should we use weights?

3

ONS social surveys

• Labour Force Survey (LFS)

• General Household Survey (GHS)

• Expenditure & Food Survey (EFS)

• Family Resources Survey (FRS)

• Omnibus Survey (OMN)

4

Labour Force Survey

• Quarterly panel survey (c. 56K hh per qtr)

• HHs stay in survey for 5 qtrs

• require estimates of:

– totals (e.g. employment)

– rates (e.g. unemployment)

• interview: all hh members

• Local boosts for annual estimates

5

General Household Survey

• 9,000 hhs per annum

• housing, consumer durables, employment, health, family structure, pensions, education

• also ad hoc trailers, e.g. drinking

• interview: all hh members

6

Expenditure & Food Survey

• Merger of Food and Expenditure surveys

• 7,000 HHs in UK

• 14 day expenditure diary

• Expenditure and income

• Food consumption and nutrient intake

• Interview: all hh members

7

Family Resources Survey

• For DWP

• 25,000 HHs per year

• Income, benefits, pensions, savings

• Fieldwork shared by ONS and NatCen

• HHs, individuals, benefit units

• Interview: all hh members

8

Omnibus Survey

• Interview: 1 adult per hh

• 1,800 adults per month

• Core questionnaire and modules

• Covers a great range of different topics

9

Why should we weight?

• Adjust for unequal selection probabilities

• Adjust for nonresponse

• Adjust our sample to match known population totals

10

Probability weights

• Weight 1/(prob of selection)

• Boost samples

– EFS in NI, weight = GB weight /4

– more common in ad hocs

11

Probability weights

• Subsampling of units

• Omnibus (1 adult per hh)

– weight = # adults in hh

• FRS (Multi-household addresses)

12

Nonresponse weights

13

Nonresponse weights

• Sample-based nonresponse methods

• Split set sample into weighting classes

• Estimate weighted response rates in each class

• New weight is 1/RR

14

Nonresponse weights

• Response rates different in each weighting class

• Means for major survey variables must differ between each class

• Means for major survey variables must be same for R and NR within each class

15

Nonresponse weights

• GHS and EFS

• Based on Census-link studies 1991

• Target nonresponse in specific demographic groups

• Sampling frame information

• Interviewer observations

16

EFS NR weighting classesGroup Characteristics Response

rateWeight

1 or 2 adults No children

1 London or Met. area 67% 1.49 Non-met. area

2 Scotland /North 78% 1.283 South East 64% 1.564 Other non-met 73% 1.375 With children 77% 1.30

3 or more adults No children

6 London 42% 2.387 Not London 58% 1.728 With children 68% 1.47

17

Population weights

Sample % Population %MalesUnder 16 14456 11% 5,987,000 10%16-64 39107 30% 18,538,000 32%65+ 9332 7% 3,902,000 7%

FemalesUnder 16 13777 11% 5,705,000 10%16-59 38733 30% 17,589,000 30%65+ 15012 12% 6,617,000 11%

All 130417 100% 58,337,000 100%

LFS Mar-May 2003

18

Population weights

• Produce population totals of estimates

• Reduce nonresponse bias further

• Improve precision (reduce SEs)

• Comparability across surveys

• a.k.a. calibration, post-stratification

19

LFS Population weights

• LFS - Individual level weights

– raking to 3 controls:

• 5 yr age group by sex within region

• Local Authority

• Single years 16-24 by sex

– population projections

20

LFS Population weights

• LFS HH level weights

– Same weight each hh member (Lemaitre/Dufour)

– software: Calmar

– bounded weights

– Age group 5 yrs and single years 16-24 by sex and region

21

GHS/EFS Pop. weights

• HH-level weights

• Pre-weighted by NR/prob weights

• Calibrate to 5-year age groups by sex and to region

• Pop estimates excl. communal establishments

22

FRS Population weights

• Calibration to:

– Age group, sex, marital status

– Lone Parents

– Families

– Tenure Type

– Council Tax band

– Region

23

The weighting process

Unweighted Probabilityweight (NI)

NR weight Calibrationweight

COUNTRY % % % %England 79.8 84.3 84.9 83.8Scotland 4.8 5.0 4.9 5.0Wales 8.3 8.8 8.3 8.7NorthernIreland

7.1 1.9 1.9 2.5

Total 100 100 100 100

EFS 2001/02

24

When to use weights

• Always (whenever you can)

• Problems with presentation /interpretation

– estimates / sample sizes / SEs

• NR & probability weights tend to increase variances

25

When to use weights

• Stat packages (e.g. SPSS) don’t always deal with weighting correctly

– Scale weights to average 1

• Stata/SAS survey estimation procedures

• Calibration tends to reduce variances

26

Conclusions

• Weights combination of probability, NR, calibration

• Required for unbiased estimation

• May require specialist software for correct hypothesis testing

27

Current Issues

• Use of 2001 Census data– Census-linked NR studies– Change in Pop. Controls (back-weighting)

• Integrated survey (CPS)

• LFS: – Attrition weighting– Local LFS– Number of controls

28

References

• Weighting for non-response, Elliot, D. NM17• Grossing Up - when and how, Butcher, B. SMB 14• The presentation of weighted data in survey report tables, Elliot, D.

SMB 38• Using weights in regression analysis: A comparison between SPSS

and STATA packages, Insalaco, F. SMB 45• Developing a weighting and grossing system for the GHS, Barton, J.

SMB 49

• Evaluation nonresponse on household surveys, Foster, K. GSS

Methodology Series 8. • Report of the Task Force on Weighting and Estimation, Elliot, D. GSS

Methodology Series 16.