2011 census: content workshop
DESCRIPTION
2011 Census: Content Workshop. Office for National Statistics February 2008. Welcome. Glen Watson Census. Purpose. Provide update on funding for fourth page of individual questions Allow key users another chance to comment on ONS priorities for topics - PowerPoint PPT PresentationTRANSCRIPT
Purpose
• Provide update on funding for fourth page of individual questions
• Allow key users another chance to comment on ONS priorities for topics
• Focus on the new topics/questions that ONS proposes for inclusion
• NB: Presentation focuses on Census in England & Wales
Agenda
• 10.10 Introduction• 10.40 Question testing • 11.00 Second residences and visitors• 11.40 Coffee• 11.55 Migration• 12.35 National identity• 13.00 Lunch• 13.45 Income• 14.15 UK views on content• 14.30 Prioritisation of topics• 16.00 Close
Census provides statistics on:
• Population units: – people and housing &– key demographics (age, sex, marital status, ethnicity)
• Population structures: – households, families
• More detailed characteristics :– eg religion, labour market status, industry, qualifications,
health/disability; etc
• Key requirement to ‘Get the Count Right’– 2011 Census aim : “maximise overall response rates
and minimise non-response in specific areas and among particular population subgroups”
Key lessons from 2001 Census
• Need to gather information on more than just usual residents
– Include visitors
• Need to understand coverage– Comparisons census vs admin data
• Need to improve enumeration of Communal Establishments
Mobile population & complex lifestyles
• More people at more than one address– Weekday residences for work– Holiday / weekend homes– Children of divorced parents
• International migration– More ‘short term visitors to UK’ – resident or not?
• Plus familiar issues of students, armed
forces, prisons, hospitals, hotels, hostels• Risk of undercount, overcount, or wrong location• Need to count right people, in right place
– and be able to demonstrate this
Work done so far – 2005 consultation
• Formal 3 month consultation (May – Aug)• Publication March 2006• Over 2000 responses from 500 users• Scoring of user requirements• Evaluation of ONS considerations & other reasons for collection
• Allocation of topics to one of three categories
Evaluation of topics
Category 1 Topics likely to be included
Question testing to refine questions
Category 2 Topics under consideration
Investigation of alternative sources
Question design & testing
Category 3 Topics not to be included
Topics will not be considered further
Criteria
• Strength of user need– Resource allocation– Policy use– Research
• Small geographies• Alternative sources• Multi-variate analysis• UK comparability• Continuity with 2001
• ONS considerations– Data quality– Public acceptability– Respondent burden– Operational concerns
(eg costs, space)
• Other reasons for collection
– Operational purposes– Use for coding
Key requirement: Robust estimate of population count
Summary of user requirements from 2005
• Most 2001 topics• New topics including:
- income- language- second residences- national identity
• Little expressed requirement for number of employees & hours worked
• More than 3 pages of questions!• Difficult trade-offs to be made
Work done so far – topic groups
• Eight topic groups• Topic experts from within ONS (except DIUS for
qualifications)• UK-wide membership• ONS Harmonisation• Methodology• Census
Topic-specific consultation
• Further specific consultation with users to better understand needs/priorities:
– Ethnicity, identity, language & religion– Population definitions – PDWG– Migration– Demographics– Housing– Income – NS-SEC– Disability
Other consultation and question testing
Cognitive question testing 2005 - 2008
Census Advisory Groups May & Dec 2006, Nov 2007
Statistical Heads of Profession Sept 2006, June & Nov 2007
Open meetings (London, Cardiff, Sheffield)
March 2007
Postal Test April 2007
Focus groups April/May 2007
2007 Census Test May 2007
Census Test Evaluation Survey May 2007
Omnibus Survey testing of new questions Sept - Dec 2007
Fourth page of individual questions
• Would allow inclusion of more topics • Research carried out on effects of longer
questionnaires• Split sample postal test 2007 – 3 vs 4 pages • Comparison with other censuses around world• Cost analysis of fourth page
Changes since 2005 evaluation
• Increased requirement for information on migration– Inter-Departmental Taskforce on Migration 2006– Migration consultation
• Planned Disability Survey• Question testing carried out on category 2 topics• 2007 Test and Postal Test• Priorities for content of 3 & 4 pages discussed:
– Open meetings March 2007– Advisory Groups– Heads of Profession
Why pre-test questionnaires?
• Find out if questionnaires meet survey objectives
• Gain an estimate of reliability and validity of answers
• Identify sources of potential non-sampling error
What is cognitive interviewing?
A face-to-face interview to test questions or a questionnaire, which aims:
• to understand how the respondent fulfils the task of answering questions
• to detect any actions or understandings that are not what the designer intended
Identifies problems
• Uncovering ‘hidden error’• Understanding the question• Remembering or recalling the information• Selecting a response• Reactions to sensitive questions
Features of cognitive testing
• Observation• Think aloud• Concurrent probes• Retrospective probes• Paraphrasing• Vignettes• Card sorting tasks
Cognitive testing programme
• Phase 1: question development & testing for 2007 Census Test
• Oct 2004 to Apr 2006– The questions tested were developed using:
• 2001 Population Bases and Definitions questions
• ONS’ harmonised questions • new user requirements
• A range of respondents were interviewed to ensure all criteria of interest were covered. In total: – 56 interviews – 2 focus groups
Cognitive testing programme
• Phase 2: development & testing for Rehearsal• Jan 2007 to Aug 2008
– New format testing questions in context of the whole questionnaire
• Five waves of cognitive testing:– Wave 1: 23 respondents (Nov 2006 - Jan 2007) – Wave 2: 69 respondents (Apr & May 2007) – Wave 3: 77 respondents (Jul – Sept 2007)– Wave 4: 27 respondents (Oct & Nov 2007) – Wave 5 is under way
• Whole-of-questionnaire testing (Apr – Jun 2008)
Ethnicity focus groups
• Aims:– explore public opinions surrounding
classifications of ethnic groups– unpack attitudes towards colour labels, in
particular ‘Black’ – understandings of the term ‘ethnicity’– inform development of ethnicity question
• 12 focus groups, 109 participants– Caribbean, African, Any other black
background, mixed background– London, Birmingham, Manchester, April & May
2007• Report due soon
Questionnaire design principles
• Navigation & layout– In West, people start reading at top left, after
that, we guide them with elements of layout– Essential to have a clear path for eye to follow
through the questionnaire– Clear numbering– Consistent even columns & vertical layout– Text set left and ragged right to provide patterns
for eye to grab hold of
Questionnaire design principles
• Minimise clutter– Text, symbols, graphics should be kept to a
minimum, as every mark on the page must be processed by respondent
– Maximise white space to make it easier to read and more appealing
– Keep respondent relaxed
140,000 households 140,000 households
Background
• Information on second residences has not been collected in any previous census
• Information on visitors was collected in the 1991 Census, but not in a structured way in 2001
• Enumeration base for 2011 is ‘usual residents and visitors’ – some information will be collected from everyone at their Census night address.
Second residences consultation
• Over 80 responses received from a range of users– including DCLG, DfT, Welsh Assembly Government, Local
Authorities, Demographics User Group, Market Research Society
• Uses include: – inform service provision
• plan health, waste and public transport services
– resource allocation• provide information on where people are likely to use resources
– understand the housing market • dynamics of housing market and profiling housing demand
– inform housing affordability and development policies• impact of second residences on availability of affordable housing
• Category 2 – more work required to develop suitable questions and assess respondent burden
Second residences in coverage assessment
• Duplicate returns from different locations:– e.g student counted at both term-time and parents’
address, people with second residences for the working week, children of divorced parents
– matching process to search for duplicates– use second residences information to help matching
process
Second residences in processing of outputs
• Census estimate of the population using the Mid-Year Population Estimate base:
– definitional differences between where people are counted
– use second residences information to estimate where people spend the majority of their time and adjust for difference
• Alternative population bases:– e.g. weekday population, out of term population
Second residences in 2007 postal test
• 2.5% of respondents said they had a second residence• A further 1% of respondents said they had a second
residence outside the UK
• Of those who said they had a second residence:– 87% entered the address
• 69% of those entered the full postcode
• 11% of postcodes were half completed
• 20% of postcodes were left blank
• Of those that entered an address, the highest frequency of location was London (13%), followed by West Sussex (9%)
Second residence question
A number of versions of this question have been tested.
An additional question on length of time spent at address was tested, but respondents found it difficult to answer.
The current question has proved successful in testing.
Visitors consultation
• Over 40 responses received from a range of users– including DCLG, DfT, DfES, Welsh Assembly Government, Local
Authorities
• Uses include: – inform service provision
• measure the total pressure on services such as transport and health
– resource allocation• essential for DCLG’s daytime net inflow and foreign visitors indicators
• DCLG still use visitor information from 1991 Census as not collected in 2001
• Category 1
Visitors in communal establishments
• Visitor information will not be collected from visitors in communal establishments
– too complex– could jeopardise the count of usual residents
• This means that there will not be a full visitor count– only visitors in households– no persons present output base
• Primary purpose of collecting visitor information is to get the best usual residents estimate
Visitor information for Quality Assurance
• A sample of visitors will be matched back to their usual residence
– check whether they were missed where they usually live
• In CCS areas, will also be matched to the CCS in the households they were visiting
– check whether they were mis-classified
• This will provide a source of data to assess the quality of, and potentially adjust, the census population estimates
– important when defending the census results
• In non-CCS areas, the mis-classification rate will be used in the QA
• A full match may be done in certain LAs if the QA suggests concerns
Visitors in the 2001 Census
• From a sample of 7 Enumeration Areas:– 9.5% of people recorded as visitors were actually usual
residents– 1.5% were of no fixed abode and should have been
recorded as usual residents
– 17% of visitors were overseas visitors– 67% of visitors were from the UK
• 20% of these UK visitors were missed at their usual residence
– 4.5% of visitors did not have an address recorded
• It is estimated there will be around 2.1 million visitors on Census night in 2011
Visitor questions
In testing, respondents who had visitors generally had a good understanding of who should be included in this question.
When the visitors questions were located in the household section of the form, many respondents filled in the questions for usual residents.
Locating the questions on the back page of the form has proved more successful.
ONS External Consultation; Preliminary Work
• May 2005 ONS published consultation document ‘The 2011 Census: Initial view on content for England and Wales’
• Following on, ONS published an information paper ‘The 2011 Census: Assessment of initial user requirements on content for England and Wales’
Key Migration Requirements Noted
• Address One Year Ago
• Country of Birth
• Clear need for additional migration data
Specific Migration Consultation
• In October 2006 a consultation document on external user needs sent to data users
• 61 responses were received from:
– 12 Central Government Departments
– 34 Local Government
– 10 Academics
– 5 other groups/organisations
October 2006 Consultation: Key Migration Issues Noted
• Address One Year Ago
• Country of Birth
• Citizenship
• Date of Arrival
Oct 2006 Consultation: Issues
• Address One Year Ago – Identifies internal migrants and international in-migrants
in previous year
• Country of Birth
– Continuity with previous censuses essential
– To profile the large numbers of migrants from EU ‘Accession’ countries
– House of Commons Library said CoB a ‘poor proxy’ for Citizenship; increased ‘blurring’ between CoB & Citizenship.
Oct 2006 Consultation: Issues
• Citizenship
• Has never been included in UK censuses
• Is most easily defined by passport eligibility
• The National Statistics Quality Review (NSQR) on International Migration Statistics recommended inclusion of a Citizenship question in future censuses
Citizenship: Potential Stated Uses
• Defines right of movement, eligibility to vote and access to employment/welfare benefits
• Collection would allow citizens of other EU countries to be measured at local level
• UK is one of only three OECD countries not collecting Census information on citizenship
Oct 2006 Consultation: Issues
• Date of Arrival• Date of Arrival for those born overseas not included in past
UK censuses
• NSQR on International Migration Statistics (2003) recommended a question on year of (last) entry;
• Year of entry is included in the LFS, but data at local level are limited.
Oct 2006 Consultation: Issues (cont.)
• Current CoB question makes it impossible to differentiate migrants living in UK for many years from recent arrivals
• Date of Arrival for those born overseas would allow reconciliation of stock/flow estimates of migration
• For those who arrived in UK in previous year, month of arrival & intended length of stay should be collected
• Inter-Departmental Taskforce on Migration highlighted importance of accurate data on short-term migrants for local areas
Task Force on Migration Statistics
• Set up by National Statistician to improve migration estimates
• Necessary due to current importance of migration in population change
• Key recommendations in December 2006 included:– More information about migrants in the country– Use linkage for better information– Provide better indicators of migrant numbers– More coherent reporting
Census 2011: ONSCD Requirements;
• Key Priority is ‘capture’ of the ‘12 month+’ usually resident population
• Enumeration Base of Usual Residents Plus Visitors agreed in February 2005
• In the last two years much public discussion of short-term migration from ‘Accession’ countries
Census 2011: ONSCD Requirements (cont.)
• Short-term migrants not uniformly distributed: many LAs concerned about insufficient data;
• ONSCD concerned to:– meet more recent requirements for information
on short-term migration– ensure Census provides an appropriate base
for mid-year estimation/projection
Specific ONSCD Requirements
Requirements for Flexible Base Estimation
Residence Bases:
– Usually resident population length of stay 12 months+
– Short-term migrants temporarily resident in the UK intending to stay for:
1-3 months 3-12 months 6-12 months
ONSCD: Required Residence Bases
• Priority is headcount of the usually resident population (i.e. those people intending to be resident for 12 months+)
• Headcount estimates required of short-term migrants from the Census
ONSCD: Required Residence Bases (cont.)
• Year/month of arrival and address 12 months ago only provide information on length of stay up to census day
• Intended total length of stay would allow distinctions to be drawn
• Intended length of stay questions are currently being developed
• Implication = all international visitors > 1 month complete all individual questions
Migration – date of entry
This question was based upon the question used in the Labour Force Survey (LFS).
Generally respondents were able to answer this question well and many were quite certain that the date written was correct.
Migration - citizenship
This question was based upon the question used in the General Household Survey (GHS).
Respondents during testing showed a general understanding of the concept of passport entitlement.
Migration – intention to stay
Respondents certain of their circumstances or visa entitlements generally found this question easy to answer
Migration – intention to stay
This question has since been revised during testing to take into account the 1 month usual residence rule; it is therefore still being tested.
Background
• The national identity question has never been asked in a census and was first introduced as part of the 2001 Labour Force Survey (LFS)
Numerous groups expressed dissatisfaction with the ethnic group question in the 2001 Census as it did not provide a tick-box for respondents to identify themselves as ‘Welsh’, ‘Scottish’ or ‘English’ but only allowed for ‘British’
As a result of campaigning, ONS have given a public commitment that National Identity will be asked in the 2011 Census
Majority of most ethnic minority groups identify with one or more GB national identity
Proportion who consider their identity to be British, English, Scottish or Welsh: by ethnic group, 2004, GB
Ethnic Group, National Identity, Religion and Language Consultation
• 2011 Census Stakeholders Consultation 2006/07: Ethnic Group, National Identity, Religion and Language (EILR), England and Wales
• Provided a detailed picture of user needs to help decide which questions should be asked in the 2011 Census.
• 606 responses to the consultation.– central and devolved government– expert, community and special interest groups– Local and regional government– local Service providers
User need
Majority of respondents to the 2006/07 Census Stakeholders Consultation stated they needed a national identity question:
• 58 per cent of respondents in England and Wales
• 75 per cent of respondents in Wales
Meets user requirements by:
• Allowing previously unavailable responses• Improving public and political acceptance of the
questionnaire• Encouraging response rates and improving quality
of information from the ethnic group question• Providing a better understanding of local
populations and communities• Providing a measure of community cohesion• Measuring how people from different areas of the
UK perceive their national identity
Users require the outputs to effect a variety of outcomes
• Grant allocation by central government• Local resource allocation and service provision• Meeting statutory obligations under legislation on
race relations and equal opportunities• Informing policy development and monitoring• Enabling monitoring and regulatory bodies to
function effectively• Product marketing and delivery for private
organisations
Cognitive testing
• Tendency to interpret ‘national identity’ in terms of legal status, often referring to passports and nationality
• Others interpreted “what do you consider your national identity to be” more subjectively referring to how they feel, language, birthplace, residence
• Some overlap with ethnicity in how people understand concept
Combining national identity and ethnicity would not work in England and Wales
A single ethnic group question which included concepts of national identity would not work in England and Wales:
• Operational constraints• Methodology• Clarity• Acceptability• Analysis
National Identity – version 1
• Tested in cognitive testing wave 1-3 question.
• Based on LFS question
• Some respondents do not appear to notice the instruction ‘Tick all boxes that apply’
National Identity – wave 3 conclusions
• Ideas and concepts of national identity centre largely around legal status and country of birth, with some respondents expanding this idea to include affiliation and culture also. – “Proud of being Irish, where you’re born and raised.”– “What I perceive to be my nation, what I affiliate myself
with.”• Evidence suggests respondents are reading all answer
options and are more likely to tick a specific nation i.e. English, Scottish, Welsh or Northern Irish instead of British despite explicitly expressing their ‘Britishness’. – “I do often consider writing British but because the
option for Welsh was there it was easy because I would consider myself Welsh before British and probably not really perceive being British as a national identity.”
National Identity – version 2
• Wording changed to emphasise affiliation
• “Tick all that apply” instruction removed due to space limitations, assuming respondents will infer this from new wording
National Identity – version 3
Wording made clearer
Outstanding issues to improve question clarity:
• Alternative wording: “What do you consider your national identity to be?” allows room for a “Tick all that apply” instruction
• Examples for the “Other” may assist respondents
• Ordering of “British” tick box
Income
• Census topic consultation• Topic-specific consultation• Question testing prior to 2007 Test• Findings from 2007 Test• Findings from Test Evaluation Survey• Findings prior to 2001• Why not household income?
Census topic consultation
• Over 100 responses (2005)• Wide range of potential uses
– Measure of deprivation– Indicator of social exclusion– Estimation of local market sizes– Measure of affluence
• User need score – category 1• ONS category 2
– More work required to develop suitable question to collect accurate information & assess public acceptability
• Consultation on priorities (2007)– More demand for other 4th page topics eg carers
Topic-specific consultation
• November 2005• 30 responses• Preference for:
- gross income- closed question (7-10 bands)- individual income to be collected
• Sources of income not supported• Users supported inclusion of income – but not at
expense of poor response rate or poor quality answers
Income question testing prior to 2007 Test
• Pre 2007 cognitive testing• Banded vs exact income• Gross vs net income• Individual vs household• Current vs time period• Sources of income
Findings from 2007 Census Test
• 2.7% significant difference in response rates between income and non-income areas
• Note: Estimated £2 million cost for follow-up per 1% drop in response rate
• Non-response to level of income high (9%) • Non-response higher for unemployed, ethnic
minorities, less educated• No sig. adverse public reaction notified through
contact centre or field staff• No sig. difference in requests for individual forms• No sig. difference in household under-coverage
2007 Census Test – key constraints
Key constraints affecting income evaluation:• Voluntary
– relied on public’s good will to complete a return– 46.1% response rate
• Publicity and media coverage– limited
Findings from Test Evaluation Survey
• 16% refused to answer income questions• 15% unhappy about answering 1 or more questions
– 58% unhappy about income level– 48% unhappy about income sources– 9% unhappy about ethnicity– 2% unhappy about qualifications
• Comments from respondents– “should be left to Revenue, nothing to do with ONS”– “invasion of privacy”– “irrelevant to concept of Census”– “intrusive”
• 64% retest reliability for income level• 77% retest reliability for sources of income
Findings prior to 2001
• Cognitive interviews 1995-97• 1997 Test
– 2.8% difference in response rates but not statistically significant
– 22% item non-response for income
• 1999 Rehearsal
Why not household income?
• Public acceptability– Respondents unhappy about answering on behalf of
others
• Quality of data– Respondents not including everyone in household– Respondents don’t know income of others– Household income better derived from individual
incomes
• Limited space on housing page• International experience
– Other countries collect individual income
UK views on content
• No final decisions on content from any country• Final consultations taking place• Each country subject to views of respective
parliaments/Ministers• Further question testing to be carried out eg intended
length of stay
Possible UK differences for 4 page content
Topic (new) England & Wales
Scotland NI
Second residences Intended length of stay
National identity *
Month/year of entry into UK
Month/year of entry for those who have lived outside NI for continuous period of 1 year or more
Nature of disability/illness Voluntary work *partially incorporated within ethnicity question
Possible UK differences for 4 page content
Topic (2001) England & Wales
Scotland NI
Hours worked
Part-time/full-time work
Study place address
Transport to study place address
Religion brought up in
Housing questions
• General UK agreement• GROS considering a household income question• NISRA considering accessibility for disabled
question• ONS considering number of bedrooms question
Table exercise
• Prioritise questions into low vs medium vs high priority
• Construct questionnaire for four pages• Construct questionnaire for three pages• Feedback views to group
– Where agreement on priority for topics– Where disagreement on priority for topics– What has been fitted on three pages of questions– What has been fitted on four pages of questions
Timetable
Decision on funding for fourth page March 2008
Topics for Census agreed by RGs April 2008
Cognitive question testing April – June 2008
Postal Test and Omnibus Survey April – June 2008
Census Advisory groups April/May 2008
White Paper published with topics Autumn 2008
Questionnaires for 2009 Rehearsal agreed Autumn 2008
Census Advisory groups / roadshows Autumn 2008
Census Rehearsal Autumn 2009
Questionnaires for 2011 Census agreed Autumn 2009
Census questionnaires approved by Parliament Spring 2010
Census day Spring 2011
Contact details
Census Questionnaire Design and Content
Office for National Statistics
Segensworth Rd
Titchfield
Hampshire
PO15 5RR