quality issues in the 2001 census ludi simpson, cathie marsh centre for census and survey research
TRANSCRIPT
![Page 1: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/1.jpg)
Quality issues in the 2001 Census
Ludi Simpson, Cathie Marsh Centre for Census and Survey Research
![Page 2: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/2.jpg)
Quality for which purposes?
Macro administration and planning Fine geography, timeliness, constant questions
Policy evaluation, research Cross-tabulated questions
Micro administration Identified individuals
![Page 3: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/3.jpg)
Major changes for the2001 census
Content and design of the questionnaire New questions
Health, lowest floor level, caring for others, relationship between residents, religion
Students at term-time address, no visitors Fieldwork
Pre-printed list of addresses Contracts for major operations Post-back Focus on hard to count areas
![Page 4: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/4.jpg)
More major changes for the2001 census
Processing and validation All forms scanned All responses fully coded and processed – no 10% tables Redesigned census coverage survey (CCS)
impute the whole population before output: a One Number Census.
Output Simpler cross-tabulations and key statistics UK wide Standard area statistics free on the internet Census Access Project
All standard area statistics; migration and commuting; vector boundaries. Not included: commissioned output; SARs.
Confidentiality - disclosure control
![Page 5: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/5.jpg)
Plan
The quality of enumeration, and adjustments before publication
Differential undercount
Disclosure control: small cell adjustment
One Number Census
![Page 6: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/6.jpg)
“The key to our whole strategy has been to try to minimise the amount of differential undercount”
ONC Steering Group, and ONS evidence to Treasury Committee
Courtesy of Mt Meagre cosmetic stones
![Page 7: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/7.jpg)
Procedures affecting differential undercount
Community Liaison programme Pre-listed addresses Double EDs for enumerators in easier areas Post-back Checks on returned forms Centralised form production Local enumerator loyalty
![Page 8: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/8.jpg)
1991 % non-response after ONC2001, total persons
1614121086420
20
01
% n
on
-re
spo
nse
ON
C2
00
1,
tota
l pe
rso
ns
40
30
20
10
0
SuffolkForest H
South StCherw ell
Richmond
CeredigiSlough U
Manchest
Westmins
Wandsw or
Tow er HaSouthw ar
Redbridg
New hamLew isham
Lambeth
Kensingt
Islingto
Haringey
Hammersm
Hackney
Camden
Brent
City of
More in 2001
Less in 2001
1991 %
2001
%Census non-response 1991 and 2001
![Page 9: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/9.jpg)
Person non-response
(1) Differential between LADs1991 2001
Mean of LADs 2.7% 5.2%Range 0% - 14% 1% - 36%Standard Deviation 1.9% 4.6%Inter Quartile Range 1.1% 3.2%SD/mean 0.72 0.88
(2) Differential between types of peopleBiases severe, but not as marked as in 1991:
Young men or young people generally Private renters, unemployed, not-White
![Page 10: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/10.jpg)
Item non-response: missing, invalid, inconsistent responses
Under 1%: age, sex, marital status 1-5%: 15 variables 5-10%: religion, provision of care, qualifications,
employment status, supervisor status, industry, workplace address, hours worked, travel to work, number of rooms
> 10%: Professional quals 17%, company size 14%
LAD variation: Wokingham, Eastleigh, Hart, E Dorset best, Manchester, Blackburn and 5 London Boroughs worst.
“Biases … were in the same direction as those present in the 1991 Census, but were less marked.” (Edit-imputation evaluation report)
![Page 11: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/11.jpg)
Conclusions on non-response
Post-back problems jeopardised quality Neither levelling up of response rates
nor levelling down Lower response than 1991 Wider geographical differences than
1991 More of all types of people missed Future preparation for next census
![Page 12: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/12.jpg)
Measures to prevent disclosure have an impact on data quality
Measures to prevent disclosure: Thresholds for census areas
OAs 40 hh and 100 residents Broad output categories
No SARs sub-regional geography; no large households
Imputed records not distinguishable Record swapping between areas July 2002: Adjust small numbers to eliminate all 1s and 2s
Table totals the sum of internal cells Different tables, different totals
![Page 13: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/13.jpg)
Impact on small area indicators Unemployment rate from ST028
GrassingtonALL PEOPLE
Economically Active
Employee: Part-time
Employee: Full-time
Self-employed (with employees): p/t
Self-employed (with employees): f/t
Self-empl (without employees): p/t
Self-empl (without employees): f/t
Economically Active: Unemployed
ALL PEOPLE 1,102 729 148 293 15 71 42 120 2116 13 3 0 0 0 0 0 0 017 10 4 0 0 0 0 0 0 018 12 9 0 0 0 0 0 0 319 13 13 0 7 0 0 0 3 0
20 to 24 42 39 8 25 0 0 0 0 3 25 to 29 45 39 4 23 3 0 3 6 0 30 to 34 63 58 13 26 0 5 6 5 3 35 to 39 100 90 24 36 0 10 3 17 0 40 to 44 93 81 14 35 3 3 9 17 0 45 to 49 103 97 17 47 3 13 3 8 6 50 to 54 136 115 23 50 0 17 3 19 3 55 to 59 133 97 24 28 0 12 6 27 0 60 to 64 111 40 14 10 0 5 4 7 0 65 to 69 136 38 7 6 6 3 5 8 3 70 to 74 92 6 0 0 0 3 0 3 0
Males 526 375 22 169 3 55 14 94 916 8 3 0 0 0 0 0 0 017 3 0 0 0 0 0 0 0 018 9 6 0 0 0 0 0 0 3
(Sum of 30 cells) divided by (Sum of 390 cells) 21 ± 4 divided by 729 ± 11 Rate: 2.9% is likely to be in error by 0.6%
![Page 14: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/14.jpg)
3020100-10-20-30
120
100
80
60
40
20
0
-20
K0209LT4
K0209D
K0208LT4
K0208D
K0207LT4
K0207D
K0206LT4
K0206D
K0205LT4
K0205D
K0204LT4
K0204D
K0203LT4
K0203D
K0202LT4
K0202D
Num
ber
of O
A v
alue
s of
0 o
r 3
Difference between Ward value and sum of OA values
Impact of aggregating small area adjustment: 8,850 wards, 16 age groups (KS02)
Impact of aggregating
![Page 15: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/15.jpg)
Impact of small area adjustment: advice for users
Substitute 1.5 for 0s and 3s? No! Average error=sqrt(no. of 0s and 3s) *0.8
5% of time error more than twice this. Beware percentages based on small rounded totals Statistical analyses are affected: measurement error Aggregate a minimal number of areas or cells
Key statistics better than Standard Tables Use Univariate tables for denominators 1 ward is better than the sum of its OAs 1 ward minus 2 OAs is better than 18 OAs
Impact of adjustment is only worse than in 1991 if 0s and 3s are one third of summed values
![Page 16: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/16.jpg)
Variability of Tabular Counts by Processes
0.0002% 0.03%AGE Data Capture
0.91%
Overall standard error 1.08% Edit and Imputation
0.001%SEX One Number Census
0.91%
Overall standard error 1.07% Disclosure Control(Small Cell Adjustment)
MARITALSTATUS
Overallstandard error 1.23%
0.02% 0.01%
0.91%ETHNIC
Overall standard error 1.35%
Note that there is no small cell ajdustment variability for sex and marital status as there were no small counts for these variables
0.41%
0.16%
0.16% 0.91%0.16%
0.14%
![Page 17: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/17.jpg)
Measuring census error
What is the probable distance of the 2001 ONC from the truth?
For future Census planners: for which populations is the ONC expected to have greater error than the census enumeration and other alternatives?
Population size; variable; undercount
![Page 18: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/18.jpg)
How accurate is the census population?
Root mean square error, %
Enumeration
Census + absent
households
One Number Census
E&W 6.2% 0.1%
LAD 10.4% 0.74%
Ward
Output Area
![Page 19: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/19.jpg)
Statistics of a complete population versus Speedy delivery of results
Compromise: a firm output prospectus Achievement of data release
Standard area statistics: Feb 03 – Sept 03 (3-4mths delay) Origin-Destination statistics: Jan 04?
Documentation, software Public availability of key statistics via NeSS/SCROL…
![Page 20: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/20.jpg)
Enumerated
Census enumeration and the population, Great Britain
51
52
53
54
55
56
57
58
59
1981 1991 2001
Mil
lio
ns
Estimated by CVS/CCS
Enumerated
Rolled on from 1981
2001 ONC.
![Page 21: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/21.jpg)
Additions to enumerated 48,843,000 Evidence Judgements
Dual system estimate of undercount
3,199,000 Census Coverage Survey Medium
Revised household estimate
230,000 Addresspoint with LFS Medium
Census day to June 30 43,000 Births, deaths, migration Light
Revised persons estimate 193,000 Longitudinal Study Medium
Further revised persons estimate
? Address matching Mcr, Westr
Medium
Unmonitored int’l migration 81-91
351,000 2001 census Heavy
Unmonitored int’l migration 91-01
c111,000 Improved 91-98 Light
c85,000 Visitor switchers Medium
c108,000 Migrant switchers Heavy
Unexplained difference with rolled forward MYE 2001
291,000 2001 census
Oct
ober
200
203
-04
![Page 22: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/22.jpg)
Sub-national
Best ever sub-national estimation procedures for census year Methods agreed by users before census
Concerns: Insufficient information about quality assurance Administrative comparisons not acted upon Unconvincing geography of undercount
![Page 23: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/23.jpg)
QA of local population: comparators Students, armed forces, prisoners Local pre-census population estimates and
administrative records Child benefit, pensioner, births, school census,
adjusted GP patients “calculate a range of plausible values for the
number of people of each sex within five-year age groups in each geographical area” ONC Guide
Diagnostic range up to double that of the comparators
![Page 24: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/24.jpg)
![Page 25: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/25.jpg)
![Page 26: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/26.jpg)
Composition of undercount: ratio
of undercount rates
Change in population
sex ratio
Type of District:
Under-count
rate, all persons
Male 20-34/ Female
20-34
All 20-34/ All other
20-34 M/F, ONC2001-MYE2000
Inner London 22% 1.11 1.41 -.126
Outer London 10% 1.15 1.95 -.146
Principal metropolitan cities 9% 1.30 1.64 -.131
Large cities 7% 1.30 1.90 -.146
Small cities 6% 1.36 1.91 -.125
Resort, port and retirement 5% 1.25 2.21 -.073
Other metroplitan Districts 5% 1.25 1.75 -.091
New towns 5% 1.35 1.76 -.063
Industrial areas 4% 1.30 1.85 -.074
Urban and mixed urban-rural 4% 1.35 2.30 -.032
Remoter, mainly rural 4% 1.42 2.13 -.028
England and Wales 6% 1.22 2.06 -0.086
![Page 27: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/27.jpg)
Where did population estimates fall?
A few Districts in each region account for most of change
Districts with transient populations – students, armed forces, seasonal labour, immigration
ONS now focusing on outliers Manchester and Westminster address-matching
Were address lists complete?
![Page 28: Quality issues in the 2001 Census Ludi Simpson, Cathie Marsh Centre for Census and Survey Research](https://reader033.vdocument.in/reader033/viewer/2022051014/5515f5e4550346d46f8b561d/html5/thumbnails/28.jpg)
Conclusions Wide-spread undercount – new types of people Impact on output reduced by ONC imputation? Uncertain population total
Young men, children Multi-source error for small areas: advice for users
Future priorities Fieldwork: improved management Output: firm timetable, pre-release documentation. Information: users are the Census’ best friends Third way: validated administrative records International migration: beyond interim revisions Residence definitions: avoid ‘legal population’ Why were people missed?