1

Measuring Quality Issues Associated with Internal

Migration EstimatesJoanne Clements, Amir Islam, Ruth Fulton & Jane Naylor

Demographics Methods Centreand Centre for Demography

2

Outline

• Background

• Internal Migration Quality Issues

• Research methods

• Findings

• Issues arising

• Next Steps

3

Project

Improve understanding,

measurement and reporting of

the quality of population

estimates

4

Context

• Debate about amount of uncertainty in

population estimates

• Improving Migration and Population Statistics

(IMPS) Project – Quality strand

• ‘ONS should flag the level of reliability of

individual local authority population estimates’

(UK Statistics Authority)

• Leading new international research

5

Key Methodology Points

• Map out the procedures and data sources used to derive population estimates

• Identify associated quality issues

• Attempt to quantify uncertainty using statistical theory & empirical evidence instead of expert opinion

• Combine individual measures of uncertainty by simulating potential errors in the data

6

Key Methodology Points

• Map out the procedures and data sources used to derive population estimates

• Identify associated quality issues

• Attempt to quantify uncertainty using statistical theory & empirical evidence instead of expert opinion

• Combine individual measures of uncertainty by simulating potential errors in the data

7

Key Methodology Points

• Map out the procedures and data sources used to derive population estimates

• Identify associated quality issues

• Attempt to quantify uncertainty using statistical theory & empirical evidence instead of expert opinion

• Combine individual measures of uncertainty by simulating potential errors in the data

8

Key Methodology Points

• Map out the procedures and data sources used to derive population estimates

• Identify associated quality issues

• Attempt to quantify uncertainty using statistical theory & empirical evidence instead of expert opinion

• Combine individual measures of uncertainty by simulating potential errors in the data

9

Progress

• Initial work proved feasibility of simulation methodology

• Focus now on sources of error with greatest impact; internal and international migration

• Currently focussing on internal migration

10

Internal Migration Methodology

• Individual moves captured from GP re-registration data

• Annual (end July) download of patient registers

• Moves identified from changes with previous year’s download.

• Local authority moves constrained to information provided by NHS Central Register

11

Key Internal Migration Quality Issues

Source LA for out-flowsto NI and Scotland

Census and 2001 Patient

Registers

Constraining GP register data to

NHSCR data

Time

Lags

Double counting of School boarders

Not registered

at mid-year

13

Research Methods

• A review of relevant literature.

• Local authority level data analysis

• Review any internal quality assurance.

• Sensitivity analysis

14

Re-registration Time Lag Research

• Comparison of mid-2001 internal migration estimates with 2001 Census migration estimates

• Sex ratios• Propensity to migrate

• Comparison with other data sources• Investigating ‘bumps’ in population age

profiles that sustain over time

15

Birmingham Population Age Profile

16

Provisional Time Lag Findings:Sex Ratios

• Evidence of late-registration of young male migrants

• Geographic variation in sex ratio differences and therefore time lags

Source Migrant Sex Ratio

15-29 years

Census 2001 0.915

Internal Migration 00/01

0.765

17

Provisional Time Lag Findings:Propensity to Migrate

• GP List inflation invalidates analysis to compare Census and internal migration propensities

• Instead, comparing migrant counts for similar populations to identify possible time lags

• Census doesn’t always produce higher LA internal migration estimates

18

Provisional Time Lag Findings:Other Data Sources

• Limited other data sources with which to compare with – No major differences with comparator data sources

• Evidence from survey data of significant late registration (Median 4 months)

19

Provisional Time Lag Findings:Age Profiles

• Some LAs do have age profile bumps that sustain (particularly young adults ages)

• Patterns vary again geographically• Possibly due to:

Imbalance between in and out migrants in LAs with higher education institutions (Males especially)

Increases in International immigrants (young males again)

20

Provisional Time Lag Findings:Summary

• Evidence of Age-Sex Specific Time lags in re-registration.

• Evidence that these vary geographically.• Unclear how much year on year time lags

cancel each other out.• Next Step is to produce an potential error

distribution

21

School Boarder Research

• LAs with largest school boarder populations chosen to identify possible double counting

• Comparing age profile changes in school boarders with LA internal migration estimates

22

Provisional School Boarder Findings

• Similar patterns between school boarder arrivals and internal in-migration

• Therefore, strong evidence of double counting

• Difficult to estimate accurately due to data issues

• Limited impact, for most LAs, on all age internal migration estimates

23

Challenge: Deriving Error Distributions For Each Quality Issue

• Lack of suitable data

• Conflicting evidence

• Somewhat subjective choice of error bounds

- Bias towards larger errors?

- Sensitivity Testing

- Constraining

- Correlation

- User Feedback

24

Challenge: Interpretation of Findings

• In reality, there is uncertainty in these measures of uncertainty, as…

– Only as good as the error assumptions made for each issue

• Therefore exact findings are misleading

• Present approximate indicators

25

Reporting and Future Work

• Short update on progress – August 2009

• Detailed papers on internal migration findings

- November 2009

- 2010

• Potential further work:

- international migration

- quantifying impact of methodological

changes on quality of estimates