tracking procedures era

Emma Hooper, Daniel Lewis The Application of Selective Editing to and Claire Dobbins the ONS Monthly Business Survey

SMB 68 3/11

1

The Application of Selective Editing to the ONS Monthly Business Survey

Emma Hooper, Daniel Lewis and Claire Dobbins

Methodology Directorate, ONS

Summary

This is an updated version of a paper by Emma Hooper and Daniel Lewis, originally presented at the Q2010 conference in Helsinki.

When running surveys it is important to detect and correct errors in the response data in order to maintain the quality of results. The UK’s Office for National Statistics (ONS) is undertaking improvements to its data editing strategy across business surveys through the introduction of selective editing.

The selective editing approach that the ONS has taken with its short-term business surveys is to use an estimate-related score function to create item scores for key variables. These item scores are combined into a unit score, thresholds are then constructed for the unit score to determine whether a unit will be manually edited or not. Various quality indicators such as bias, change rates and savings are used throughout this process in order to choose an appropriate selective editing approach.

Two existing ONS short-term business surveys have recently been combined into a new survey called the Monthly Business Survey (MBS). The development of MBS was an opportunity to improve the survey process using selective editing. This survey is presented as an example of how selective editing has been implemented at ONS, and various quality indicators are presented showing the impact of selective editing on the published estimates and the savings that can be achieved over the traditional edit rule approach.

1. Introduction

For any survey it is important to detect and correct errors in the response data in order to maintain the quality of results. The UK’s Office for National Statistics (ONS) has been undertaking improvements to its data editing strategy across business surveys through the introduction of a new selective editing approach for its short-term business surveys. The recent development of the Monthly Business Survey was an opportunity to improve the survey process by applying the new ONS selective editing methodology to this survey.

Data editing is an expensive part of the survey process in terms of processing time, operating costs, and burden on respondents. ONS is hoping to reduce these costs through focussing editing efforts onto those records with the largest impact on the published estimates, while still maintaining the quality of those estimates.

The selective editing approach that ONS has taken with its short-term business surveys is to use an estimate-related score function (Hedlin 2003) to create item scores for key variables. These item scores are combined into a unit score, thresholds


SMB 68 3/11

2

are then constructed for the unit score to determine whether a unit will be manually edited or not. Various quality indicators such as bias, change rates and savings are used throughout this process in order to choose appropriate threshold values.

This paper outlines the process undertaken in order to apply selective editing to the ONS Monthly Business Survey (MBS). Section 2 introduces MBS, while section 3 explains the current use of selective editing at ONS. Section 4 outlines the selective editing methodology specifically for MBS, and section 5 presents some quality indicators of the impact of selective editing. An explanation of the system used for implementation and some of its current limitations is provided in section 6, followed by some post implementation plans in section 7.

2. Monthly Business Survey

MBS was launched in January 2010 with its first publication “Turnover and Orders in Production and Service Industries”, released in March 2010 (ONS 2010). It brings together existing short-term surveys that cover different sectors of the economy. The existing surveys that have moved to MBS are the Monthly Inquiry into the Distribution of Services Sector (MIDSS) and the Monthly Production Inquiry (MPI); these surveys are now processed as one survey. Other short term surveys – the Retail Sales Inquiry (RSI) and the Construction Statistics surveys, have been rebranded as MBS as well, but are processed separately for now. The benefits of bringing these surveys together include: harmonised methodology and consistent processes; a reduction in burden of businesses through less frequent collection of some variables and dropping some questions entirely; and a reduced number of surveys and systems to maintain in the long term.

As part of the aim to harmonise the methodology, the existing editing processes were reviewed and it was decided that MBS would be a good candidate for using the new selective editing methodology. References to MBS in the remaining paper do not include the RSI or Construction Statistics surveys.

The MBS questionnaire is sent out to approximately 30,000 businesses every month. The data feed into the Index of Services, the Index of Production and consequently the output measure of Gross Domestic Product. The main variables collected are Turnover, Total number of employees and Employees by male / female, part-time / full-time. The variables are collected monthly, except for the employment variables, which are only collected quarterly for a sub-sample of the units. Additional variables are collected on some specific questionnaires sent to a small number of industries.

3. Selective editing at ONS

Selective editing was implemented in MIDSS in August 2001; however, this was a slightly different approach to the selective editing methods discussed in this paper (Underwood 2001). The historic selective editing method for MIDSS was for returned questionnaires to pass sequentially through: automatic editing; edit rules; and selective editing score calculation. All edit rule failures that involved missing or inconsistent values in the key variables were checked, as were any edit rule failures incurred by new respondents to the survey. If a return still had edit failures remaining after these checks, then this return would pass through to the score calculation stage. Only those returns with an estimate-related item score greater than a set threshold


SMB 68 3/11

3

would have their remaining edit rule failures checked. Following the MIDSS implementation, the same selective editing method was implemented for MPI.

In 2008 a project was set up at ONS to fundamentally review the editing processes for ONS business surveys. A result of this project was the development of a new selective editing methodology for ONS short-term business surveys through collaboration between the ONS Methodology Directorate and Pedro Luis do Nascimento Silva of Southampton University.

RSI was used to develop the new ONS selective editing method; using the key variables Total turnover and Total employment (Silva 2009). How this methodology has been applied to MBS is described in section 4.

4. Outline of selective editing methodology for MBS

In order to complete the analysis of the current micro editing approach and specify a selective editing methodology for MBS, 24 periods of edited and unedited data were used for the period January 2008 to December 2009. The large number of periods used should ensure the robustness of the results.

4.1 Key variables and domains

The first step in developing a selective editing methodology for MBS was to identify key variables and domains. Following discussions with the MBS results team, they identified four key variables – Turnover, Export turnover, New orders (all collected monthly), and Total employment (collected quarterly). Export turnover and New orders are only collected from units that were in industries previously covered by MPI. Additionally, some industries that were previously in MIDSS have Commission and Sales collected instead of Turnover. In this situation Commission and Sales were summed to give a Turnover value. This is also the case for some industries which collect Sales and Invoices; these were summed to give Total turnover.

The key domains identified are UK National Accounts industry input/output groups; these are roughly equivalent to 2 or 3 digit UK Standard Industrial Classifications 2007 (SIC07). SIC07 is consistent with the latest version of NACE (the European industrial activity classification) at the 4 digit level. For MBS there are around 80 input/output groups in scope.

4.2 Item and unit score functions

Estimate-related score functions focus on large contributions to the key survey estimates, in this case Turnover, Export turnover, New orders, and Total employment by input/output group. An item score is calculated for each key variable, for each unit. The item score used is

1

ˆ100 ˆ

t t ti ij ijt

ij tjd

a z yscore

T −

−= × (1)


SMB 68 3/11

4

1

is the sample design weight for unit at time

is the unedited value for variable , unit at time

ˆ is a predicted value for variable , unit at time ˆ is the previous perio

tij

tij

tij

tjd

a i t

z j i t

y j i t

T − d's total estimate for variable in domain .j d

The sample design weight is used because calibration weights tend not to be available until later in the survey process. The process for selective editing should be that the item scores get calculated after batch take-on, so that editing can begin as soon as the first batch of data is returned and processed.

The predicted values used for MBS are previous edited value where available (this is the value from the previous month, except for Total employment where it is the value from the previous quarter). If this is not available then a corresponding register value for the current period is used for the variables Turnover and Total employment. For Export turnover and New orders, a register value is not available, so a pseudo-imputed value is used instead. The predicted value is thus defined

1

1

* 1

if available for unit

, for Turnover or Employment

, for E

ˆ _ if is not available

if is not available

tij

t t tij i ij

t tij ij

iy

y Selection x y

y y

−

−

−

=

xport turnover or New orders.

⎧⎪⎪⎨⎪⎪⎩

(2)

1

*

is the edited value for variable in the previous period for unit

_ is the corresponding register value for Turnover or Employment for unit

tij

ti

tij

y j i

Selection x i

y

−

is the current period pseudo-imputed value for Exports or New orders for unit .i The pseudo-imputed value used for Export turnover and New orders when there is no available previous value, is defined as:

* 1 _t t tij SIC iy CLink Selection to−= × (3)

1 1

111

1

_

timpclass tn

ijt

it iSIC t

impclass

ySelection toCLink

n

− −

−=−

−=∑

(4)

1

_ is the current period register value for Turnover for unit

is the constructed value link (a trimmed ratio of averages) from the previous period

ti

tSIC

Selection to i

CLink −

1

for the 4-digit SIC07 corresponding to this period for unit is the number of units left in the imputation class after trimming, in the previous

per

timpclass

in −

iod.

We are not able to use 1_ t

iSelection to − in the calculation of * ˆ tijy because the

corresponding register turnover value is not be held in the processing system for an enterprise that was entering the survey for the first time this period. We are currently restricted to only using variables that are available in this system.


SMB 68 3/11

5

We then want a single selective editing score for each unit; we call this the unit score. Only those enterprises with a unit score above a preset threshold are selected for manual micro editing. The size of the item scores give an indication as to which variables have contributed most to the unit score and caused it to fail editing. However, the principle used at ONS is that all variables on the questionnaire should be validated if the questionnaire has been selected for manual micro editing. Those enterprises that have a unit score below the threshold will not have their returned questionnaires edited any further, unless they are later identified in macro editing.

For the original RSI selective editing study we combined the item scores into a unit score using the unified global score function for selective editing presented in (Hedlin 2008). The method is based on the Minkowski distance function and is defined as:

1

1

( )p

t ti ij

j

u scoreλ

λ

−

=

⎛ ⎞= ⎜ ⎟⎝ ⎠∑ (5)

is the number of item scores for unit

1 is equivalent to sum of the item scores (or, equivalently, the mean)2 is equivalent to the Euclidean distance

Large values of would be roughly equ

p iλλ

λ

==

ivalent to the maximum item score.

It was found in the RSI study that there was no strong evidence to support choosing one value for lambda over the others that were tested. Limitations with the processing system meant that we were restricted to using the mean or maximum of scores. For RSI the decision was made to use the average of scores. For MBS we first used the mean of scores and found this to perform satisfactorily, so have not tested the maximum of scores. Choosing the mean of scores means that each key variable has an equal influence on the unit score. The maximum number of item scores that a unit in the MBS could have is four – in an end of quarter month where Turnover, Total employment, Export turnover and New orders were collected.

4.3 Setting of thresholds

We were concerned with three main quality indicators when we set the threshold for a domain. These are Relative Bias (RB), Absolute Relative Bias (ARB) and Savings, defined in formulae 6-8. The bias measures are indicators of the bias that would remain in the domain estimate if the enterprises with a unit score below the threshold were not edited. We make the assumption that the historic domain estimates after micro and macro editing are the ‘true’ figures. Note that in practice MBS data were already subject to a more conservative selective editing method, as described in section 3. Therefore, the bias measures are actually measuring the additional bias in introducing this new selective editing method.

We were interested in the relative bias because it gives us an indicator as to whether there is any systematic up or down movement in the level of bias. The absolute relative bias gives us a measure of the overall bias in the domain estimate as well as being an indicator of the potential bias in future survey rounds. The savings are measured by the relative change in the number of units that are currently edited and the number that will be edited under selective editing for the given threshold and domain.


SMB 68 3/11

6

( )ˆ ˆ( )t

t t t t t tjd ij ij ij i d jd

i s

RB w z y I u c T∈

= × − × <∑ (6)

( )ˆ ˆ| |

t

t t t t t tjd ij ij ij i d jd

i s

ARB w z y I u c T∈

= × − × <∑ (7)

is the sample at time

is the estimation weight for variable , unit at time

is equal to 1 if the unit score for unit at time is less than threshold for domain , else it is equ

t

tij

s t

w j i t

I i t c d al to 0.

t tt d dd t

d

trad selectSavingstrad−

= (8)

is the number of units failing at least one traditional edit rule at time in domain

is the number of units with a unit score above the threshold at time in domain .

td

td

trad t d

select t d The decision was made with the business area survey owners that we would aim to keep the estimated absolute relative bias for a domain estimate below 1%. With the bias at this level, there should be negligible difference between the domain level estimate based on selective editing and the domain level estimate based on the current MBS edit rules. Some bias outliers would be allowed over the periods of data, as these would be picked up at the macro editing stage if they had an unacceptable effect on the domain level estimates. A range of thresholds were looked at for each domain, and a threshold was selected that balanced the need to control the level of bias while also aiming to make positive savings in the number of units that would be edited. Examples of the types of graphs used to help inform the choice of thresholds follow in graphs 1 and 2.

Each box plot in graphs 1 and 2 contains quality indicators for a number of periods. 23 periods of data are contained in each box plot. There is not a measure for the first period of the data, as we do not have previous values to use as the predicted value, so it is excluded from the analysis. The ten thresholds in the graphs were set to different values for the different domains. If none of the initial ten thresholds tried proved to have an acceptable level of bias and savings then a different group of threshold levels were tried. We can see in graph 2 that the median level of savings does not increase rapidly after the fourth threshold, and that the fourth threshold in graph 1 has an acceptable level of absolute relative bias which is below 1% of the domain level estimate. There does not appear to be many additional savings to be made by increasing the level of absolute relative bias, so in this situation the fourth threshold seems to be a suitable level for this domain and variable. We would then need to check the bias graphs for all of the other key variables to make sure they were at an acceptable level. We would also want to look at the choice of threshold on the overall level of savings for the survey, in addition to the domain level savings, and see whether the increased level of bias between thresholds 3 and 4 was worth the extra savings achieved.


SMB 68 3/11

7

Graph 1. Absolute relative bias for variable j, domain d, for 10 different thresholds

1 2 3 4 5 6 7 8 9 10

0. 000

0. 500

1. 000

1. 500

2. 000

AbsBias1

cut of f Graph 2. Savings for domain d, for 10 different thresholds

1 2 3 4 5 6 7 8 9 10

0. 00

25. 00

50. 00

75. 00

100. 00

RelSavings_Score1

cut of f For some of the domains the number of units in a given month can be a lot smaller than other domains, for example there could be less than 50 units. For these smaller domains there may not be many savings to be made over the current number of units being edited, in which case we are more concerned with controlling the estimated bias and editing a sensible percentage of units in the cell than making savings.

5. Quality indicators

As mentioned in section 4, we are interested in overall level impact as well as domain level impact. There are a number of quality indicators we could use to see how well the selective editing methodology and choice of parameters is performing in comparison to the current MBS editing strategy.

For MBS, the initial investigation showed median overall savings (see formula 9) of approximately 46% in the months January, February, April, May, July, August, October and November. The median overall savings in the months March, June, September and December were approximately 3%. On these months, the employment variables are also collected. It would not be possible to achieve many savings in these months whilst maintaining the given level of accuracy for the Total employment variable, as well as the other key variables which are collected on a monthly basis.


SMB 68 3/11

8

t t

tt

trad selectOverallSavingstrad−

= (9)

is the number of units failing at least one traditional edit rule at time for the chosen

thresholds is the number of units with a unit score above the threshold at time f

t

t

trad t

select t or the chosen thresholds.

A comparison between the estimated absolute relative bias left in the domain estimate after selective editing and the absolute relative bias left in the domain estimate after the current micro editing showed large improvements across MBS domains. We took the final macro edited values to be the true estimates. This means that many errors that are left after the current micro editing and which are being captured at the macro editing stage will instead be captured at the selective editing stage. In practice this could lead to savings in the survey area because many errors that are currently identified at the macro editing stage would already be checked and cleared. The method and parameters we are using for the MBS selective editing methodology are clearly performing well at identifying influential errors at the micro editing stage.

Further useful quality indicators are edit failure rates and edit change rates. These are defined as follows:

100

number of units being micro editedEdit failure ratetotal number of responding units

= × (10)

100

number of units with value changes as a result of micro editing

total number of responding unitsEdit change rate= × (11)

These give us a further indication of how well micro editing is performing, whether it is the selective editing approach or the current edit rule approach. One would hope that the edit failure rate is not too high, as all these units need to be checked which is a resource expense. Furthermore for efficiency reasons we would hope that the edit change rate was similar to the edit failure rate, because if time is spent checking a unit we would want to find an error in need of correction.

Graphs 3, 4, 5 and 6 give us evidence that the new selective editing methodology is more efficient than the previous edit rule based method used for micro editing.


SMB 68 3/11

9

Graph 3. Edit failure rates and edit change rates after using edit rules to identify units to edit from the old MPI domains

05

1015202530354045

2008

04

2008

06

2008

08

2008

10

2008

12

2009

02

2009

04

2009

06

2009

08

2009

10

2009

12

%

Edit failure rate Edit change rate

Graph 4. Edit failure rates and edit change rates after using selective editing to identify units to edit from the old MPI domains

-5

5

15

25

35

45

2008

04

2008

06

2008

08

2008

10

2008

12

2009

02

2009

04

2009

06

2009

08

2009

10

2009

12

%


Graph 5: Edit failure rates and edit change rates after using edit rules to identify units to edit from the old MIDSS domains

05

101520253035

2008

04

2008

06

2008

08

2008

10

2008

12

2009

02

2009

04

2009

06

2009

08

2009

10

2009

12

%



SMB 68 3/11

10

Graph 6: Edit failure rates and edit change rates after using selective editing to identify units to edit from the old MIDSS domains

05

101520253035

2008

04

2008

05

2008

06

2008

07

2008

08

2008

09

2008

10

2008

11

2008

12

2009

01

2009

02

2009

03

2009

04

2009

05

2009

06

2009

07

2009

08

2009

09

2009

10

2009

11

2009

12

%


The edit failure rate is smaller in Graph 4 than Graph 3, which reflects the savings we have made. Also, the gap between the edit failure rate and the edit change rate has decreased showing that the number of units changing as a result of micro editing is closer to the number of units that have been identified for editing.

In Graph 5, we can see that there were more edit failures in 2008 than there were in 2009. This was due to efficiencies that were implemented into MIDSS, such as telephone data entry (TDE) which reads back answers to respondents and gives them the opportunity to correct errors. When comparing Graphs 5 and 6, we can see that in the months January, February, April, May, July, August, October and November, we have consistently achieved a lower edit failure rate under selective editing. However, for March, June, September and December in 2009, we have a slighter higher edit failure rate under selective editing. This is due to incorporating the Total employment variable into the selective editing process, which has been discussed previously in this paper (see section 5).

6. Editing process, implementation and system limitations

MBS will still have some edit rule checks for valid dates or comments in the comment box, followed by automatic editing for the “thousand pounds error” and component checks. The thousand pounds error identifies where it is likely that the respondent did not round the returned figure to the closest thousand as required. If this error is identified then the automatic editing will divide the returned figure by 1000. Component checks are for employment questions where the female/male/part-time/full-time splits do not add up to the given total number of employees. Selective editing will be carried out after automatic editing, and macro editing will then take place before publication. The continued use of edit rule checks for date validation will impact on the level of savings presented in the previous section.

ONS uses an in-house built system called Common Software to store and process data for most short-term business surveys. A module has been developed to carry out selective editing, as specified in section 4, in the Common Software system. This module does have some limitations on what Methodology Directorate can specify for the selective editing methodology. These current limitations include:

• being restricted to specifying at most five key variables for which item scores will be calculated;


SMB 68 3/11

11

• being restricted to only using variables already available in Common Software for use in calculating the predicted values;

• being restricted to combining item scores into a unit score through either a maximum function or an average function;

• being unable to use current edit rules to calculate other types of selective editing score.

7. Future work and post implementation

The new selective editing methodology and associated thresholds were implemented into the Monthly Business Survey in August 2010. ONS plans to continue testing and, where appropriate, implementing selective editing methods for other short-term and some annual business surveys. This will lead to more efficient editing, resulting in a better quality editing process.

Post implementation, ONS will want to know how well the new selective editing methodology is performing. As there will be no micro edited data for the enterprises that have a score below the thresholds we will not have information to monitor the performance and review the thresholds. Evidence can be gathered though by selecting a small sample of those units that are not selected for micro editing; enterprises in this sample will then be edited. This will enable us to calculate a measure of the estimated bias that is left in the survey estimates by those units that are not selected for editing. We can then monitor this bias or adjust the thresholds accordingly.

References Hedlin D (2003) ‘Score Functions to Reduce Business Survey Editing at the UK Office for National Statistics’, Journal of Official Statistics 19, pp 177-199. Hedlin D (2008) ‘Local and global score functions in selective editing’, UNECE Work Session on Statistical Data Editing, Vienna, 21-23 April 2008. ONS (2010) ‘Changes to the survey outputs from MBS (formerly ETOD and DST)’, available at: http://www.statistics.gov.uk/articles/nojournal/Changes-survey-outputs-MBS.pdf Silva P L N (2009) ‘Investigating selective editing ideas towards improving editing in the UK Retail Sales Inquiry’, European Establishment Statistics Workshop, Stockholm, 7-9 September 2009. Underwood C (2001) ‘Implementing selective editing in a monthly business survey’, Economic Trends 577.

SMB 68 3/11

12

Tracking Procedures on the Employment, Retention and Advancement Survey

Kathryn Ashton and Martina Portanti

ONS, Social Survey Division

Summary

Longitudinal surveys are subject to attrition throughout their waves. This is because it can be difficult to locate respondents and ensure they are able to be contacted at each wave and individuals may be more likely to refuse an interview over subsequent waves of a survey. Locating respondents between waves of a longitudinal study is particularly complex in countries where a central population register does not exist. This is a problem for the United Kingdom. In recent years, the linkage of individual level data from administrative sources with details from longitudinal survey respondents has become increasingly common to help locate respondents at the present location.

This research presents the results of using both prospective and reactive tracking methods to locate individuals participating in Wave 3 of the Employment, Retention and Advancement (ERA) survey carried out by the Office for National Statistics (ONS). At Wave 2 of ERA more than 15 per cent of the sample could not be located. This was predicted to increase at Wave 3, which was carried out three years later. In order to improve the accuracy of contact details, ONS used a number of different tracking methods, including a telephone unit (TU) ‘keep-in-touch’ exercise and linkage to several administrative sources in an attempt to increase contact rates. We report on the results of these methods in terms of improvements to contact and response rates, and changes in the composition of the respondent group. We conclude with an assessment of the tracking procedures used and of their potential for a more extensive use as part of ONS longitudinal surveys’ tracking strategy.

1. Introduction

In the context of longitudinal surveys, the term attrition is normally used to refer to the loss of survey participants over time. Although attrition may occur for a number of different reasons, researchers tend to be particularly concerned with attrition due to non-response, i.e. loss of sample members at follow-up because they cannot be contacted or they refuse to continue participation (Watson and Wooden, 2004). Attrition due to non-response not only may decrease the power of longitudinal data analysis but it may also be selective, thus impacting on the generalisability of results to the target population.

The process that leads to attrition due to non-response can be divided into three conditional processes: failure to locate respondents; failure to contact respondents, having locating them; and failure to obtain cooperation from respondents, having contacting them (Lepkowski and Couper , 2002).

In particular, failure to locate respondents may be a major contributor to attrition in a longitudinal study. This was the case for the third Wave of the Employment

Kathryn Ashton and Martina Portanti Tracking Procedures on the Employment, Retention and Advancement Survey

SMB 68 3/11

13

Retention and Advancement survey (ERA), carried out by the Office for National Statistics between December 2008 and March 2010.

This paper describes the methods that were used on the ERA in order to track sample members. It illustrates the outcome of these methods in terms of improvements to contact and response rates and changes in the composition of the respondent group.

2. Employment Retention and Advancement Survey (Wave 3)

The Employment Retention and Advancement (ERA) programme was designed to test a method to improve job retention and advancement among low-income individuals and consisted of a combination of job coaching and financial incentives which were offered to participants once they were working. It was implemented between 2003 and 2004 in six pilot areas across the UK.

The ERA programme was implemented using a random assignment process. Individuals who volunteered to be included in the programme were randomly assigned either to the ERA programme or to a control group, which did not have access to the ERA services.

As part of the ERA policy evaluation, a large-scale longitudinal survey with a sample from both the ERA programme and control group members was carried out by ONS. The survey has so far followed the respondents for 5 years through 3 waves of interviews. Wave 1 was carried out between December 2004 and March 2006; Wave 2 from December 2005 to March 2006; and Wave 3 from December 2008 to February 2010. The ERA survey followed a mixed-mode design. All sample members were attempted to be contacted and interviewed by the Telephone Unit. Non-contacts and circumstantial refusals were then reissued to face-to-face interviewers to attempt conversion.

At Wave 2, around 15% of the sample issued to field could not be located. This percentage was expected to increase even further at Wave 3 for a number of reasons. Firstly, the time gap between the Wave 2 and 3 interviews was much longer than between the Wave 2 and Wave 1 interviews (3 years versus one year). Secondly, between Wave 2 and 3, funds were not available to carry out any keep-in-touch exercise to try and maintain respondents’ contact details up-to-date. Finally, the Wave 3 sample also included non respondents at previous waves. This means that the Wave 3 survey could have been the first time some sample members were contacted since their enrolment into the ERA programme five years before.

In Wave 3, both prospective and reactive techniques were used in an attempt to locate sample members.

3. Prospective tracking

Forward-tracing or prospective techniques are tracking methods that try to ensure that up-to-date contact details for all sample members are available by the start of the fieldwork period. Information is normally gained from the respondent themselves, either as recorded at the latest interview or by updating the contact details before the next wave occurs (Burgess, 1989; Couper and Ofstedal, 2009). These methods are relatively inexpensive and have proved successful, as the most useful source of information for tracking is often the participants themselves (Ribisl et al, 1996).


SMB 68 3/11

14

On ERA two forward-tracing methods were used. Postal and telephone keep in touch exercises were carried out before the start of the mainstage fieldwork to try and update sample members’ contact details.

3.1. Postal ‘Keep-in Touch’ Exercise (KITE)

Between May and June 2008, all Wave 3 sample members (5,688 individuals) were sent a pre-contact letter. The letter included a leaflet setting out the purpose of the survey, a postcard so that sample members could inform ONS of any changes to their contact details, and an ERA/ONS labelled key ring as an unconditional incentive.

Around 7% of the ERA sample returned the postcard to ONS with updated contact details.

3.2. Telephone Unit KITE

Each sample member was also contacted for a keep-in-touch interview by the ONS Telephone Unit (TU) around 3 months before their main stage interview. The aim of this short questionnaire was to ask respondents about their most up-to-date contact details, and also inform respondents they were to be contacted in 3 months time for the main stage interview.

Overall, the TU KITE, which was administered from September 2008 to September 2009 achieved a response rate of 52.4 per cent and 54 per cent contact rate. Table 4.1. shows the type and amount of information which was collected during the TU KITE.

Table 3.1. Information Collected at the TU KITE TU KITE outcome TU KITE Responders Total Sample Size

Update some information 35.1% 18.4%

Address Update at TU KITE 14.1% 7.4%

Telephone number Update at TU KITE 16.2% 8.5%

Both telephone number and address update at TU KITE 4.8% 2.5%

No new details update at TU KITE 64.9% 34.0%

Total TU KITE respondents 100.0% 52.4%

Total TU KITE non respondents 47.6%

Around 35 per cent of those individuals who completed a TU KITE interview updated at least one element of their contact information (address and/or phone number): 14.1 per cent updated their address details; 16.2 per cent gave details of a new telephone number; and 4.8 respondents updated both their telephone and address. This means that the TU KITE collected new contact details for around 18.4% of the sample that was later issued to the mainstage and confirmed existing details for 34% of the sample.


SMB 68 3/11

15

The outcome of the TU KITE was a strong predictor of the outcome at the mainstage interview, as shown in table 3.2.

Table 3.2. TU KITE and mainstage outcomes

TU KITE outcome

Respondent Non respondent

Response rate at mainstage 78% 20%

Proportion of cases reissued to field at mainstage 21% 70%

Contact rate among cases reissued to field 83% 51%

Response rate among cases reissued to field 59% 32%

Around 78% of those individuals who were successfully contacted for the TU KITE exercise responded to the mainstage interview. Among TU KITE non-respondents, only around 20% of the sample were respondents at the mainstage interview. TU KITE non-respondents were also less likely to be interviewed in the Telephone Unit and they had to be reissued to a face-to-face interviewer more often. Around 70% of the cases who did not respond to the TU KITE had to be reissued to a face-to-face interviewer against 20.5% of the TU KITE responding cases.

Once in field, TU KITE non-productive cases were again less likely to be contacted and interviewed. Only 51% of TU KITE non-responding cases who were re-issued to the face-to-face field force were successfully contacted and only 32% successfully interviewed. In the group of TU KITE respondents who were re-issued to face-to-face interviewers, around 83% of cases were successfully contacted (with around 59% being interviewed).

Overall, cases who did respond at the TU KITE proved to be less costly to be contacted during the mainstage, as they were more often interviewed by the telephone unit and when reissued to field, they were more likely to be contacted.

4. Reactive techniques

On completion of the ERA main stage field period, the survey achieved a response rate of 62.9 per cent and a non-contact rate of 22.3 per cent. The main reason for non-contacts was due to inaccurate and out-of-date contact details. In an effort to improve response, additional tracking was attempted after the end of the mainstage field period. Three administrative sources were used to try and update contact details for 1,010 sample members who did not respond to the mainstage ERA interview, in the vast majority of cases because they had moved since the last time they had been contacted and their current whereabouts were unknown.


SMB 68 3/11

16

4.1. Use of administrative sources

Administrative data sources are increasingly being used to track respondents between waves of longitudinal surveys. Access to administrative records in the UK is regulated by national legislation. Government departments have put in place specific data sharing agreements which often require explicit consent to be collected from individuals before their data can be shared across departments.

ERA participants provided, back in 2001, consent for data held by Inland Revenue (now HMRC) and DWP, to be accessed and used as part of the ERA evaluation project. On this basis, in 2010 ONS sought access to HMRC tax credit records and DWP benefit records to update ERA respondents contact details. HMRC refused access to their data as they deemed the consent for data sharing being too old; DWP, as joint data controller and sponsor of the scheme, could instead provide access to their benefit records.

DWP provided ONS with updated contact details, including addresses and telephone numbers, for all ERA sample members who were present on the DWP benefit record database. The data was passed confidentially and securely across the Government Secure Intranet. Once the updates had been received, it was simple to match the new contact details to the ERA sample using an encrypted national insurance number.

Additional contact details were then linked to the sample members using the Royal Mail National Change of Address (NCOA) Register, which is sourced from the Royal Mail Redirection Service. The NCOA register not only identifies that an individual has moved from their original address, but also offers details of where that individual has moved to. Access to the database is provided by a number of commercial organisations, which provide online facilities to search and match addresses against the NCOA database.

New telephone numbers were also attached to sample members using information sourced from the British Telecom Operator Services Information System (OSIS). The OSIS database is updated on a daily basis and it contains more than 26 million residential and business names, addresses, and telephone, although it does not include ex-directory and mobile telephone numbers. Access to the database was obtained through one of OSIS licenced commercial users.

Finally, the Electoral Register was searched for those cases where a new address could not be found either in the DWP records or in the NCOA register. ONS holds a copy of the full electoral register, listing names and addresses of all individuals in the UK who are entitled to vote. The search of the Electoral Register proved to be extremely complex and resource-intensive for several reasons. Firstly, the only information available on the electoral register for matching purposes were individuals’ names and address. This made it difficult to try and identify a correct match for people with common names and individuals whose name may have been misspelt. Secondly, cases had to be searched one by one, as no facility was available to automatically link a list of names and/or addresses to the register. Finally, the register is actually a collection of separate databases organised by each Local Authority (LA) within the UK. Each LA database had to be searched separately, as no facility existed to search a name or address across several LA databases.

For these reasons, and because ERA face to face fieldwork did not follow movers out of the areas of previous residence, ERA sample members were searched only on the electoral register of the Local Authority of their latest address. Firstly, the database of the LA where the respondent last lived in was searched to check whether the respondent was still resident at the most recent address held by ONS. If different


SMB 68 3/11

17

names were found at the address, then a search by name and surname was performed on the LA database. If one match was found, than the new electoral register address was assigned as an alternative address for that case. If more than one match was found or there was no match, no alternative address was assigned.

4.2. Quality of new contact information

After updating their contact details, cases were included in one of three mop-up exercises which took place in July 2009, October 2009 to March 2010 and January to March 2010. The mop-up exercises took place after the end of the mainstage fieldwork period as a final effort to try and obtain interviews from cases who were still uncooperative. The first mop-up exercise was implemented solely in the TU using updated telephone numbers from the administrative sources. The second and third mop-ups were designed in the same way as the main stage; all cases were sent to the TU with updated telephone numbers and all TU non-contacts were then re-issued to the face-to-face interviewers.

If an alternative address, different from that held by ONS and used during the mainstage fieldwork, was found in one of the administrative sources, this was used either as a primary or a secondary address during the mop-up fieldwork. If a case had been coded as a non-contact at mainstage because the address details were incorrect, then the address obtained from the administrative sources was used as the primary address and interviewers were instructed to call at that address only. For all other mainstage outcome codes (e.g. non-contact because nobody was at the address), the case was issued to the mop-up fieldwork with both the mainstage address and the alternative address obtained from the administrative sources. Interviewers were instructed to attempt contact first at the mainstage address, and if unsuccessful, then attempt contact at the new address, if in the same local area.

The quality of the additional contact details proved complex to analyse. If a full interview was obtained during the mop-up fieldwork, respondents up-to-date contact details were recorded at the end of the interview. This allowed us to check whether the contact details obtained from the administrative sources proved to be correct in field. If however the case was not productive, it was often impossible to establish whether interviewers had actually used the new contact details and whether these had proved to be correct or not.

Table 4.1 summarises the administrative sources uses, how many contact details were obtained through each of the sources and how many of these could be confirmed with certainty whether they were correct or not at the end of the mop-up exercise


SMB 68 3/11

18

Table 4.1. Contact details updates by administrative source used

New addresses

New telephone

numbers

Administrative source

Found

Confirmed incorrect

Confirmed correct

Found

Confirmed correct

DWP benefit records

588 (100%)

109 (18.5%) 88 (15%)

345 (100%)

60 (17.4%)

NCOA 36

(100%) 6 (16.7%) 4 (11.1%) Not applicable Electoral register

49 (100%)

10 (20.4%) 8 (16.3%) Not applicable

BT OSIS Not applicable26

(100%) 8 (30.8%)

All sources

673

(100%)

125

(18.6%)

100

(14.9%)

371

(100%)

68

5. Response rates and respondents characteristics

The three mop-up periods proved successful in improving both response and contact rates. The final response rate increased of five percentage points, from 62.9 per cent to 67.9 per cent, and the non-contact rate was reduced from 22.3 per cent to 17.8 per cent.

Although an increase in response rates is beneficial for the statistical power of the analysis, it does not provide any indication about whether the additional tracking helped and reduced the non-response bias. Non-response may be selective, in that the individuals who do not take part in survey may differ from those who remain in the sample. This is a concern for analysts as it impacts on the generalisability of results to the entire target population.

For longitudinal studies in particular, much research has shown how characteristics of individuals lost at follow-ups because they cannot be located tend to differ from those of respondents (Hawkes and Plewis, 2006; Grey et al, 1996). Additional tracking efforts may help to reduce attrition bias by ensuring hard-to-reach group can be located.

In order to investigate whether the additional tracking carried out at the end of the ERA mainstage field period helped and reduced potential attrition bias, we used pooled T-tests to compare the characteristics of those individuals who responded to ERA during the main stage with those of the individuals who were instead interviewed during the mop-up periods.

As shown in table 5.1, the proportion of single respondents interviewed during the mop-up exercises was significantly higher than the proportion recorded at the main stage. This agrees with the literature that single people are more likely not to be


SMB 68 3/11

19

located or contacted at a follow-up of a longitudinal survey (eg. Grey et al, 1996), and supports the assumption that single respondents may require more tracking effort to be located.

Table 5.1. Marital status of respondents from the main stage and mop fieldwork periods Marital status Main stage Mop-up

Single *** 45.2% 60.1%

Married and Living with Partner 10.5% 10.4%

Civil Partner 0.7% 0.3%

Married and Separated 6.9% 7.2%

Divorced *** 34.6% 20.4%

Widowed 2.0% 1.6%

Total 100.0% 100.0%

*** Significance level of less than 1%

The difference in proportions of divorcee respondents in the main stage sample and in the mop-up sample also shows a significant difference. The proportion of divorced respondents at the main stage was 34.6 per cent, whereas only 20.4 per cent of the ERX respondents were divorced.

Mop-up respondents were more likely to rent their home compared with mainstage respondents (table 5.2.). Overall 79.5 per cent of respondents at ERX indicated they rented, compared to 66% among mainstage respondents. This results also supports survey attrition literature, which finds that people who rent their home are more likely to move and change their contact details (Watson and Wooden, 2004), thus being more difficult to locate.

Table 5.2. Tenure of respondents from the main stage and ERX fieldwork periods Tenure Main stage Mop-up

Own it outright 4.4% 1.0%

Buying it with the help of a mortgage *** 28.6% 18.6%

Pay part rent and part mortgage (shared) 0.6% 0%

Rent it *** 66.3% 79.5%

Live here rent-free * 0.1% 1.0%

Total 100% 100%

*** Significance level of less than 1%; * Significance level of more than 1% but less than 5%


SMB 68 3/11

20

Our results support the theory that job instability appears related to non-contactability (Hawkes and Plewis, 2006; Branden et al, 1995). The proportion of respondents with seasonal, temporary or casual jobs increased significantly from 3.8 per cent at the main stage interview to 8.1 per cent after the mop-up exercises (table 5.3). Without the use of the reactive tracking procedures we can assume that this category of individuals would have been under-represented after the main stage fieldwork period.

Table 5.3. Nature of job of respondents from the main stage and ERX fieldwork periods Job Main Stage Mop-up

Permanent job 91.2% 86.9%

Seasonal, temporary or casual job 3.8% ***8.1%

Job done under contract or for a limited time 4.0% 3.2%

Some other type of job 1.0% 1.8%

Total 100% 100%

*** Significance level of less than 1%

There was no identifiable or significant difference between the number of male and female respondents before or after the reactive tracking procedures were used. This may have been a result of the majority of the ERA W3 sample being female, leaving little room to test the effect of tracking procedures on gender Also, analysis showed that there were no significant differences between mainstage and mop-up respondents in the proportion of respondents who had children under the age of 19.

6. Conclusions

Survey organisations have devised and implemented a number of systems to locate, contact and ensure continued cooperation from panel members in an effort to reduce attrition over the life of their longitudinal studies. In spite of the growing array of methods available, there is little systematic evidence about which ones works best for different attrition problems (Couper and Ofstedfal 2006; Fumagalli, Laurie and Lynn 2009) and their relative cost-effectiveness.

At Wave 3 of the ERA survey, it was expected that a significant proportion of sample members could not be located because of out-of-date contact details. For this reason, much effort was needed to carry out extensive tracking both before and during fieldwork. The survey did not include any experimental design to test the efficacy of the tracking methods used and therefore only limited conclusions can be drawn from the ERA experience for wider application.

Keep-in-touch exercises are extremely popular tracking methods and our results show how they can be strong predictors of the mainstage outcome. This highlights the potential of attempting additional tracking on KITE non-responding cases before the mainstage fieldwork commences. Although this was not the case for ERA, other


SMB 68 3/11

21

larger scale ONS longitudinal surveys like the Wealth and Assets Survey carry out administrative data linkage to find new contact details for respondents who could not be located at KITE before the start of the mainstage fieldwork.

The use of administrative sources to update contact details helped to improve both the level of response and the representativeness of the responding sample. Administrative data sources are increasingly being used to track respondents between waves of longitudinal surveys. A number of ONS surveys already make use of services provided by commercial organisations which allow updated addresses to be obtained from the NCOA register and telephone numbers from the BT OSIS database. Other administrative data sources are used less extensively on an ad-hoc basis as ONS often does not have an automatic and continuous right to access such records. An administrative data source which is not affected by access issue for ONS is the full electoral register. Although this would appear as an ideal data source for a more extensive use as part of surveys tracking efforts, it is currently used only sporadically for sample checks. The main obstacles to a wider use of the register for surveys processing is the lack of sophisticated searching and matching functionalities, which makes the use of the register extremely resource-intensive.

References

Branden L, Gritz R M and Pergamit M R (1995) ‘The Effect of Interview Length on Attrition in the National Longitudinal Survey of Youth’. No. NLS 95-28, US Department of Labour.

Couper M and Ofstedal M (2006) ‘Keeping in Contact with Mobile Sample Members’. Draft prepared for presentation at: MOLS 2006: Methodology of Longitudinal Surveys, University of Essex, Colchester, UK. July 2006.

Gray R, Campanelli P, Deepchand K and Prescott-Clarke P (1996) ‘Exploring Survey Non-Response: The Effect of Attrition on a Follow-up of the 1984-85 Health and Life Style Survey’. The Statistician, 45, pp 163-183.

Fitzgerald J, Gottschalk P and Moffitt R (1998) ‘An Analysis of Sample Attrition in Panel Data: the Michigan Panel Study of Income Dynamics’ Journal of Human Resources, 33, pp 251-299.

Fumagalli L, Laurie H and Lynn P. (2009) ‘Methods to Reduce Attrition in Longitudinal Surveys: An Experiment’ ESRC Survey Design and Measurement and Initiative Programme on Understanding Non-Response and Non-Response Bias. European Survey Research Association Conference Warsaw June 29th-July 3rd 2009.

Hawkes D, Plewis I (2006) ‘Modelling Non-Response in the National Child Development Study’ Journal of Royal Statistics Society A, 169, Part 3, pp 479-491.

Laurie H, Smith R and Scott, L (1999) ‘Strategies for Reducing Non-Response in a Longitudinal Panel Survey’ Journal of Official Statistics, 15:2, pp269-2.

Lepkowski J and Couper M (2002) ‘Non-Response in the Second Wave of Longitudinal Household Surveys’ in Groves R, Dillman D, Eltinge J, Little R (2002), ‘Survey Non Response’, Wiley Series in Survey Methodology.

McAllister R, Goe S and Edgar B (1973) ‘Tracking Respondents in Longitudinal Surveys: Some Preliminary Considerations’, The Public Opinion Quarterly, 47:3, pp 413-416.


SMB 68 3/11

22

Nicoletti C and Peracchi F (2002) ‘A Cross-Country Comparison of Survey Non-Participation in the ECHP’, ISER Working Papers No. 2002-32, Colchester: University of Essex.

Uhrig S (2008) ‘The Nature and Causes of Attrition in the British Household Panel Survey’, ISER Working Paper Series No.2008-5.

Watson N and Wooden M (2004) ‘Sample Attrition in the HILDA Survey’, Australian Journal of Labour Economics Vol. 7, No 2, pp 293-308.

SMB 68 3/11

23

Model-Based Small Area Estimates of Households in Poverty

Philip Clarke, Kevin McGrath, Nargis Rahman, Denise Britz do N. Silva and Alan Taylor

All authors from Methodology, ONS, except Denise Silva who is from IBGE, Brazil

Summary

The Office for National Statistics (ONS) has published household mean income estimates for local areas since 2004. Model-based estimates at small areas are now accepted as a part of established statistical outputs from ONS. These are used by many different users across central and local government, academics, research organisations and charities. Users have expressed the need for estimates targeted at more specific poverty measures, particularly those related to the poverty line defined as 60% of national median income. To meet this need, ONS implemented small area estimation methods based on a logistic mixed model of the Family Resources Survey (FRS) and data from administrative sources to estimate the proportion of households with income under the poverty threshold. The models are designed with a unit level response and area level covariates and allow for varying sampling and estimation geographies by using both fixed and random effects.

This paper presents the work for the 2007/08 estimates. A discussion of the statistical model used, model diagnostics and model validation is included. Details of planned further development are also given. These estimates were published as experimental statistics in November 2010 and are fully documented by a Regional Trends Article1, User Guide2, Validation Report3 and Technical Report4.

1. Introduction

Income information is needed at the small area level in order to help identify deprived and disadvantaged communities and to support work on social exclusion. This requirement was previously identified by Census User Groups who made a strong case for a question on income to be included in both the 2001 and 2011 Censuses. Although this need was recognised by the government, concerns were also expressed about the sensitivity of an income question, potential impacts on response rates and data privacy issues. As a result, a question on income has not been included in these Censuses. Instead, alternative methods for obtaining data on income at the small area level were identified and implemented leading to the use of small area estimation methods to produce local area income estimates.

ONS has published these household mean income model-based estimates at middle

1 Regional Trends Article 2 User Guide 3 Validation Report 4 Technical Report

Philip Clarke et al Model-Based Small Area Estimates of Households in Poverty

SMB 68 3/11

24

layer super output area5 (MSOA) based on data from the FRS6 and Households Below Average Income (HBAI)7 statistics for 2004/05 and 2007/08. This followed previous publication at ward level in 1998/99 and 2001/02. Four measures of mean income have been published each time – household total gross income, household net income, equivalised household income before housing costs (BHC) and equivalised household income after housing costs (AHC).

Users have welcomed the publication of model-based estimates of mean income and, as noted above, have also expressed the need for estimates of more specific poverty measures, such as the HBAI indicator of those below 60 per cent UK median household weekly income. There are two measures of poverty currently used by the Department for Work and Pensions (DWP), one based on income before housing costs are taken into consideration and the other after housing costs. These statistics are not available at local area level. DWP strongly supported the request from users for producing small area poverty estimates.

ONS has investigated the possibility of generating such estimates. The analysis was based on proportions of households (rather than persons) as poverty is considered to be a household characteristic shared by all individuals in a household. This approach also integrates with the methodology developed for the estimation of mean household income. This paper documents the work undertaken to produce 2007/08 MSOA-level estimates of the proportion of households in poverty for England and Wales, calculated based on equivalised household income after housing costs and produced using the same methodology developed by ONS to provide small area estimates of mean income (Longhurst et al, 2004 and 2005).

The development of the 2007/08 poverty estimates calculated on an AHC basis was conducted following initial work modelling both income variables (AHC and BHC) for time periods 2004/05, 2006/07, and 2007/08. The results of this initial work to develop separate small area estimation models for all six target variables (percentage of households below threshold income based on AHC and BHC for three time periods) indicated that 2004/05 models had a lower explanatory power (measured by between area variability explained) than models for 2007/08. The quality of the model-based estimates (measured in terms of confidence intervals and associated distinguishability - defined as non-overlapping confidence intervals between low poverty and high poverty areas) were much better for AHC in all time periods. In addition, the models indicated some instability regarding the selected covariates as only a few were consistent across the models.

A decision was taken to investigate model stability revising the variable selection procedure but concentrating efforts on the model to obtain poverty estimates for 2007/08 using AHC data, since the aim was to publish estimates for the most recently available survey data and AHC estimates had better properties than the BHC estimates. This paper, therefore, presents results for 2007/08 MSOA-level estimates of the proportion of households below 60% national median income after housing costs. The other datasets were either used at the model selection stage or at the validation stage.

The paper is structured as follows. Section 2 describes the general small area

5 Super Output Areas (SOAs) is a geographic hierarchy designed to improve the reporting of small area statistics in England and Wales. There are 3 layers in the hierarchy and 7194 middle layer super output areas (MSOAs). 6 FRS Information Page 7 HBAI Information Page


SMB 68 3/11

25

estimation methodology used by ONS to deal with income related variables and Section 3 describes its application to the problem of estimating proportions of households in poverty. Section 4 explains the model selection procedure and the fitted model is described in Section 5, with the model-based estimates being presented in Section 6. An assessment of the quality of the estimates and their validity is given in Section 7, with a final summary and description of future work presented in Section 8.

2. Small area modelling

The principal reasoning behind the need for small area estimation is that whilst surveys are designed to provide reliable estimates at national and sometimes regional levels, they are not typically designed to provide estimates at lower geographical levels (local authorities, wards, etc.). With the exception of the Labour Force Survey, most of the principal national household surveys in UK have clustered designs. This means that the sample is not distributed totally randomly across the nation but that certain areas are first selected as primary sampling units (PSUs) and then households are selected for interview from these. The PSUs in the FRS are postcode sectors.8,9 The PSUs are stratified by 27 regions and also by three other variables derived from the 2001 Census of Population. Stratifying ensures that the proportions of the sample falling into each group reflect those of the population. Within each region the postcode sectors are ranked and grouped into eight equal bands using the proportion of households where the household reference person (HRP) is in National Statistics Socio-economic Classification (NS-SEC) 1 to 3. Within each of these eight bands, the PSUs are ranked by the proportion of economically active adults aged 16-74 and formed into two further bands, resulting in 16 bands for each region. These are then ranked according to the proportion of economically active men aged 16-74 who are unemployed. This set of stratifiers is chosen to have a maximum effectiveness on the accuracy of two key variables: household income and housing costs (Department of Work and Pensions (2009) Family Resources Survey United Kingdom 2007-8). The problem for estimation at the small area level is that, irrespective of the total sample size, with clustering like this the inevitable result for areas such as MSOAs is that the vast majority will contain no sample respondents at all. Hence no direct survey estimates are possible. Also, where there are estimates for particular MSOAs, the sample sizes would be so small that the precision of the estimates would be low. Note that MSOAs and PSUs are of similar size in terms of number of households. Following some preliminary studies into small area estimation, ONS established the Small Area Estimation Programme (SAEP) in April 1998. SAEP methodology (ONS 2003, 2005) involves combining survey data (in this case income related variables) with other data that are available at the small area level and modelling their relationship. The small area level is usually an area for which direct survey estimates cannot be reliably produced. The area-level relationship between the survey variable and auxiliary variables (covariates) is estimated by regressing individual survey responses on area-level values of the covariates.

The basic aim of the SAEP methodology is the construction of a statistical model relating the observed value of the survey variable of interest (measured at individual, 8 A postcode is associated with each address in the UK and they are assigned by the Royal Mail. They are also a key means of providing locational references for statistical data. There are approximately 1.78 million postcode units in the UK. 9 A postcode sector is formed by a set of contiguous postcode units. There are roughly 11,600 in the UK.


SMB 68 3/11

26

household, or address-level) to the covariates that relate to the small area in which the survey observation is located. These covariates are generally average values or proportions relating to all individuals or households in the area, based on administrative or census data which have coverage in all areas being modelled. Once the covariates have been selected and the model has been fitted, the model parameters can be applied to the appropriate covariate values for each and every area, hence obtaining estimates of the target variable for all small areas.

While the model is constructed only on responses from sampled areas, the relationships identified by the model are assumed to apply nationally. As administrative and census covariates are known for all areas, not just those sampled, the fitted model can be used to obtain estimates and confidence intervals for every area. This is the basis of the synthetic estimation ONS has already used to produce the estimates of average (mean) income for MSOA for 2004/05 and 2007/08 that were published as experimental statistics.

3. Modelling the Poverty Indicator

This section describes how the general SAEP methodology has been applied to the specific problem of estimating poverty at the MSOA-level. The datasets (both survey and covariate) used in the modelling process are also described. The standard approach to measuring low income or poverty has been to look at how many people, households or families have an income that falls below some threshold. The threshold is commonly set at a particular fraction of mean or median income, calculated across the whole population. Sixty percent of the median is currently the most widely used threshold and is the definition used by the UK government as one measure of progress on its target to reduce the number of children in low income households. The European Statistical Office (Eurostat) and the countries belonging to the Organisation for Economic Co-operation and Development (OECD) also use ‘below 60% of the median’ as their low income definition. The Households Below Average Income (HBAI) report (DWP, HBAI Team, 2009), published annually by the Department of Work and Pensions (DWP), is the principal source of information on the size and characteristics of the low income population in Great Britain.

3.1 The Datasets

3.1.1. The Survey Data

The survey data used in this modelling exercise comes from the HBAI datasets that are prepared by DWP using data from the 2007/8 Family Resources Survey (FRS) (Sullivan et al, 2009). FRS was chosen as the source for survey data for this study since it is the survey with the largest sample that includes suitable questions on income.

The target parameter to be estimated is the proportion of households below 60% of national median income based on net weekly household equivalised income after housing costs. As the SAEP methodology uses household level responses, the survey variable to be modelled is a binary variable that indicates if the household has net weekly equivalised income after housing costs below a threshold defined as 60% of national median income. The threshold value for 2007/08 is £199 pw and corresponds to 60% of national median AHC equivalised net income as published by DWP10.

10 DWP Poverty Theshold Information 2007/08


SMB 68 3/11

27

Equivalised income means that the household income values have been adjusted to take into account varying household size and composition. Equivalisation is needed in order to make sensible income comparisons between households.

Over recent years, DWP have moved to use the OECD equivalisation scale throughout the main body of their HBAI publication rather than the alternative McClements equivalisation scale11. This change occurred from the 2005/06 publication and was prompted by the UK Government’s 2004 Spending Review, which stated that future child poverty measurements will report incomes Before Housing Costs and equivalised using the OECD scale.

FRS uses a geographically stratified clustered probability sample drawn from the Royal Mail’s small users Postcode Address File (PAF). The survey selects 1,848 postcode sectors with a probability of selection that is proportional to size. Each sector is known as a Primary Sampling Unit (PSU). Within each PSU, a sample of addresses is selected. In 2007/08, 24 addresses were selected per PSU. More information on the FRS methodology is contained within the FRS technical report (Sullivan et al (2009)).

FRS aims to interview all adults in a selected household. A household is defined as fully co-operating when it meets this requirement. In addition, to count as fully cooperating, there must be less than 13 'don't know' or 'refusal' answers to monetary amount questions in the benefit unit schedule (i.e. excluding the assets section of the questionnaire). Proxy interviews are accepted only under restricted circumstances. In 2007/08, for those households classed as fully co-operating, proxy responses were obtained for 16% of adults. In 2007/08 the final achieved sample size (for Great Britain) was 24,982 households after removal of ineligible households, those partially co-operating, refusals and non-contacts.

The requirement for this project was to produce MSOA-level estimates of proportion of households in poverty for England and Wales. For 2007/08 this resulted in a survey data file that contained 18,822 households from 1,524 primary sampling units. The additional sample loss was primarily due to unusable income data for responding households. The final survey data file for England and Wales contained cases in 3,083 different MSOAs out of a total of 7,194.

3.1.2. The Covariate Data Sets

The small area estimation methodology requires covariate data to be available on a geography compatible with MSOAs. A range of data sources were used in the modelling process that included variables considered to be related to the propensity of a household having income below a threshold. They were: Census 2001 data, DWP benefit claimant counts (August 2007), Valuation Office Agency Council Tax Bandings (2007), Her Majesty’s Revenue and Customs (HMRC), Child Tax Credit and Working Tax Credit (2006), Communities and Local Government (CLG), Change of ownership by dwelling price (2007) and regional indicators.

The covariates used for modelling poverty were the same for England and Wales.

3.2 The Statistical Model

Binary response models that take into account the fact that each individual household belongs to a specific area were developed for England and Wales. These models take 11 For further information regarding the differences between the OECD and McClements equivalisation scale please see this link DWP


SMB 68 3/11

28

as the response variable whether the households are in poverty or not (1 if the household is in poverty and 0 otherwise) and the area level covariates are the explanatory variables. The models relate this survey variable of interest (measured at household level) to the covariates that correspond to the small area in which the household is located. Once fitted, the models can be used to produce estimates of the target variable at the small area level, i.e. the models can be used to produce MSOA-level estimates of the proportion of households in poverty (i.e. percentage of households with net equivalised income AHC below 60% of national median income) and confidence intervals for the estimates calculated appropriately.

The sampling area in the survey is the PSU but the estimation area is the MSOA. As FRS uses a clustered sample design, PSUs and MSOAs can cross cut each other. This means that the area level variation in the model has to be measured using the PSU (as this is the area where the data are modelled). The model assumes that variation for MSOAs is similar to variation for PSUs as PSUs and MSOAs are of similar size in terms of households. This allows for the use of the variance associated with PSUs in error calculations relating to MSOAs. Inference from the model also takes account of the effect of clustering within the data. This assumption was assessed (Heady et al, 2003) using LFS data which showed that within PSU variability was similar to MSOAs variability.

The underlying model is a two-level model given by:

(1)

id id idy eπ= +

( ) jTdid ulogit ++= βXαπ

where

idy is the survey variable for household i for MSOA d, so idy is the poverty indicator for household i in MSOA d;

j is the sampling area (primary sampling unit);

idπ is the expected probability of idy ;

dX is a vector of values for MSOA d of a set of covariates;

ju is the area level residual for primary sampling unit j (sampling area), assumed to

have expectation 0 and variance 2uσ ;

and ide is within area residual for household i in MSOA d with expectation 0.

Models are fitted on the sample data and using covariates in areas for which a sample is present. As covariates are available for all areas, a synthetic estimator of the proportion of households with an income level below 60% of the national median can be produced for all areas from the fitted model parameters, given by

is distributed as Binomial (1, )id idy π


SMB 68 3/11

29

( )( )βX

βXˆˆexp

ˆˆexpˆTd

Tdsynth

d++

+=

α

απ1

(2)

where α̂ is the estimate of α and β̂ is the vector of estimated coefficients for the covariates.

The 95% confidence interval for an area prediction in the logit scale is given by

( ) ( )2ˆ ˆˆ 1.96 ( )T Td u d dVarα σ β+ ± +X X Xβ (3)

Under the SAEP methodology the area level variance, 2uσ , is added to the standard

error term to provide an approximate 95 per cent confidence limits. The anti logit of these limits are taken to produce a 95 per cent confidence interval for ˆ synth

dπ .

4. Developing the Models

The previous sections introduced the statistical model and the datasets used to produce model-based estimates of proportion of households below 60% of median income. This section describes the model fitting procedures. During the initial development of these models automatic variable selection methods were used. This analysis showed that these were inadequate to produce valid and stable models and therefore a more “controlled” model selection approach was implemented. This strategy is described in detail in the following.

The first method applied was to select significant covariates for inclusion in model (1) by applying an automatic stepwise selection procedure in a single level model approach by excluding the random effect term for area. All of the appropriate covariates (those expressed as percentages or proportions) were transformed onto the logit scale and both the transformed and original covariates were considered for inclusion in the model. In this case, the automatic model selection procedure was carried out using SPSS options for fitting single level logistic regression because these automatic selection routines are not available when fitting multi-level models in most of the statistical packages usually used to fit multilevel models, such as SAS, STATA and MLwiN. The selected variables were then used to fit a multilevel model for the binary response and non-significant variables were removed. With these significant covariates, interaction terms were created, tested for significance and included in the model where appropriate. Interaction terms were again selected using an automatic stepwise procedure on a single level model.

The model was fitted using the statistical software SPSS (the initial model selection stage) and STATA (for the multilevel modelling) with postcode sectors at the higher level and households at the lower level, as outlined in Section 3.2. Regional indicator terms were always included in the model (whether significant or not) so as to control for region differences and to reduce the amount of calibration that would be necessary for benchmarking the model-based estimates to the published HBAI estimates at regional level. This procedure was used to develop initial small area estimation models for the 3 time periods (2004/05, 2006/07 and 2007/08) separately. Two issues were found with this approach. Firstly, the explanatory power for the 2004/05 was lower than for the other the two years and, secondly; the models included different


SMB 68 3/11

30

sets of variables. Based on this, a decision was taken to use a controlled approach for model selection in which the relation between the response variable and the covariates in each auxiliary source was analysed separately.

Explanatory analysis indicated that no unique covariate in the available auxiliary datasets showed a high correlation with the response variable, whereas the covariates are highly correlated to each other. This is not an optimal case as this implies that the covariate set all convey similar information about the response variable and none of them present a well defined relationship. Therefore, the controlled variable selection procedure was an attempt to identify if it would be possible to find a set of variables that would do reasonably well for the 3 time periods in order to develop more stable models (in terms of selected covariates).

One of the problems with an automatic stepwise selection procedure is that it can only be implemented on a single level model as the procedure is not available for multilevel models. In order to validate the “automatic selected” model, a more controlled procedure based on the multilevel model fitting was carried out. The full set of covariates was first separated into two groups: untransformed covariates and logit transformed covariates. Then, within these two groups, covariates from different datasets (e.g. Census, DWP, HMRC, etc.) were fitted separately. Automatic selection was carried out separately within each dataset followed by a forward selection (using multilevel model fitting) that started with Census covariates and was followed by adding the significant variables from each auxiliary data set. This controlled procedure results in more stable models and ensures variable for specific domains of the covariate sets are included in the final model.

This procedure was carried out for the 2004/05 and 2007/08 data twice, one run including only untransformed proportions and the other considering only the logit transformed covariates. Although the selected variables were not the same for every model, all models included at least one Census covariate representing the same underlying dimensions/area characteristics: socio-economic classification or social grade, nationality/ethnicity of the resident population, labour market status, dwelling characteristics plus the proportion of the population in working age.

The conclusion from this work was that the models presented a type of stability in relation to which information was conveyed by the selected Census covariates. The same argument was valid for DWP covariates. Therefore, in order to choose the final model, the variables identified as good predictors for the 2007/08 poverty outcome were all included in a model and a controlled backward elimination carried out. The decision to use a covariate on its original scale (as a proportion) or as a (logit) transformed covariate was based on exploratory analysis and on a measure of explanatory power obtained by fitting a model including each of the variables at a time. A measure of model adequacy was used to compare competing models. This was calculated as the percentage of between area variability explained by the covariates in the model calculated as:

Percentage between area variability explained = 2

2

(full model)1 100 (null model)

u

u

σσ

⎛ ⎞− ×⎜ ⎟

⎝ ⎠ . (4)

In addition, the percentage of between area variability explained by one covariate in the presence of all covariates was also calculated as:


SMB 68 3/11

31

Percentage between area variability explained by one covariate = 2

2

(full model)1 100 (model excluding 1 covariate)

u

u

σσ

⎛ ⎞− ×⎜ ⎟

⎝ ⎠. (5)

The chosen model and corresponding adequacy measures are presented in Section 5.

5. The Fitted Model

The final model for the proportion of households below income threshold for 2007/08 is given below and in Table 1 which contains a key to the labels of the covariates. The covariates/auxiliary data have been grouped by source. This model contained main effects and three interaction terms. The key predictors are listed below:

• proportion of household reference persons aged 16-74 whose NS-SEC is managerial and professional

• proportion of household spaces that are detached, semi detached or terraced

• proportion of persons aged 16 to 59

• proportion of households that contain one person

• proportion of persons aged 16-74 that are full-time students

• proportion of persons claiming Disability Living Allowance: Mobility Award Higher

• proportion of persons aged over 60 claiming Pension Credit: Guaranteed Element Only

• families out of work receiving Child Tax Credit

• the Government Office Region indicators


SMB 68 3/11

32

Full details of fitted model coefficients are as follows:

( )idˆlogit π =

+ 1.6436 x Constant

+ 0.1058 x Lnphrpman

- 0.0989 x Lnphouse

- 5.026 x p16_59

+ 0.5129 x lnphhtype1

+ 0.1494 x Lnpftstud

- 0.3250 x Lndlamah

+ 0.3027 x Lnpcgeo

+ 0.0007 x Famoutct

+ 0.1140 x lnphrpman x lndlamah

- 2.7671 x p16_59 x lnphhtype1

- 0.2521 x lnphhtype1 x lndlamah

- 0.0307 x Northest

- 0.1876 x Northwst

- 0.2532 x York

- 0.0758 x Eastmid

- 0.2026 x Westmid

- 0.2560 x Eastern

- 0.3466 x London

- 0.2741 x Southest

- 0.2781 x Southwest


SMB 68 3/11

33

Table 1. Estimated coefficients and key to covariates included in the model for households in poverty 2007/08, AHC

Covariate Name

Label Source

Lnphrpman

Logit transformation of proportion of HRPs aged 16-74 whose NS-SEC is managerial and professional

Census

Lnphouse Logit transformation of proportion of household spaces that are detached, semi detached or terraced

Census

P16_59 Proportion of persons aged 16 to 59 Census

Lnphhtype1 Logit transformation of proportion of households that contain one person

Census

Lnpftstud Logit transformation of proportion of persons aged 16-74 that are full time students

Census

Lndlamah Logit transformation of the proportion of persons claiming Disability Living Allowance: Mobility Award Higher

DWP

Lnpcgeo Logit transformation of the proportion of persons aged over 60 claiming Pension Credit: Guarantee Element Only

DWP

Famoutct Families out of work receiving Child Tax Credit

HMRC

Northest Dummy variable indicating MSOA is in North-East region

Country/regional indicators

Northwst Dummy variable indicating MSOA is in North-West region


York Dummy variable indicating MSOA is in Yorkshire and the Humber region


Eastmid Dummy variable indicating MSOA is in East Midlands region


Westmid Dummy variable indicating MSOA is in West Midlands region


East Dummy variable indicating MSOA is in East of England region


London Dummy variable indicating MSOA is in London region



SMB 68 3/11

34

Covariate Name

Label Source

Southest Dummy variable indicating MSOA is in South-East GOR


Southwst

Dummy variable indicating MSOA is in South-West region


Lnphrpman x lndlamah

Interaction between lnphrpman and lndlamah

P16_59 x lnphhtype1 Interaction between p16_59 and lnphhtype1

Lnphhtype1 x Lndlamah

Interaction between lnphhtype1 and Lndlamah

With no covariates included in the model, the estimated standard residual area variance 2ˆuσ is 0.24 (0.025) compared with 0.025 (0.018) obtained when the significant covariates are included in the model. These covariates together, therefore, account for 89.5% of the total between area variance.

To understand the decomposition of the between area variance, the model can be fitted by including each covariate on its own. The covariates that account for most of the between area variability are persons claiming pensions credit (lnpcgeo - DWP) and families out of work receiving tax credit (famoutct - HMRC), with each accounting for 65% and 64%, respectively, of the between area variability. The covariate indexing persons claiming Disability Living Allowance (lnphrpman - HMRC) accounts for 58% of the between area variability, while the regional indicators account for 13%. Further details of the final model are given in the Technical Report12.

6. Guidance on Use

It is important to provide guidance on the use of the estimates as there are some limitations that users need to be aware of. The main limitation of estimates for small areas, those estimated directly from responses to surveys or model-based, is that they are subject to variability. Confidence intervals associated with the model-based estimates for each MSOA are estimated in order to make the precision of the estimates clear.

These MSOA level estimates can be aggregated to provide poverty estimates for larger geographical areas such as local authority level but their confidence intervals cannot be computed. The model-based estimates for different MSOAs are correlated because they are obtained from a single model in which the set of regression coefficients is estimated using data from all MSOAs. Therefore, to make comparisons between broader areas, the standard error of the aggregated estimate has to incorporate not only the variability of each estimate but also the correlation

12 Technical Report


SMB 68 3/11

35

between estimates. Currently, ONS does not have estimates of the required correlations. Hence, it is not possible to assess the precision of the aggregated estimates, though this issue will be considered in future work.

The model-based methodology has been developed to ensure that the MSOA estimates are constrained to direct survey estimates from the HBAI for GORs in England and the estimate for Wales. For example, the model-based estimates for the MSOAs in Wales when aggregated correspond to the HBAI estimate of the proportion of households in poverty for Wales. However, the model-based estimates will not be consistent with HBAI estimates of proportion of households in poverty for other geographies (for example, for LAs).

In common with any ranking procedure based on estimates, when ordering MSOAs by poverty level, care must be exercised in interpreting these ranks. One needs to take into account the variability of the estimates when using these figures. For example, the confidence interval around the lowest ranked MSOA (lowest level of poverty) suggests that the estimate lies among the group of MSOAs with the lowest poverty levels rather than being the MSOA with the very lowest poverty level.

In order to compare model-based estimates of two particular MSOAs, it is necessary to account for the correlation between the estimates. None overlapping confidence intervals may be taken as an indication that the MSOA estimates are statistically different. However, this evidence must be used with caution as this constitutes a naive procedure to account for the uncertainty when estimating the difference between two cross-sectional estimates.

Although these model-based estimates can be used to rank MSOAs by proportion of households in poverty, they cannot be used to make any inferences on the distribution of poverty across the MSOAs. The estimation procedure tends to shrink estimates towards the average level of poverty for the whole population, so model-based estimates at each end of the scale tend to be over or under-estimated. This is a natural outcome of using a model-based approach as the predictions put less weight on outlying observations and so they appear to “shrink” towards the average level.

Nevertheless, estimates can be used to make certain inferences; for example, the proportion of households in poverty in MSOA A is greater than the proportion of households in poverty in MSOA B (if the appropriate confidence intervals do not overlap). Given that model-based estimates are subject to limitations, some examples of appropriate and inappropriate uses for the estimates are provided in the associated User Guide13.

7. Model Diagnostics & Validation

A number of diagnostic checks have been used to assess the appropriateness of the models developed for producing MSOA-level estimates of poverty. The diagnostic checks employed are those developed by ONS for small area estimation (Heady et al (2003) and Brown et al (2001)). This analysis showed that in general the models were well specified and the assumptions are sound. This provides confidence in the accuracy of the estimates and the associated confidence intervals. In addition, the methodology used to produce the model-based estimates has undergone internal and

13 User Guide


SMB 68 3/11

36

external review. Full details of this analysis can be found in the associated Technical Report.

Analysis to compare the model–based estimates with other sources of poverty data was carried out to establish the plausibility of the model-based estimates. For example, as detailed in the Validation Report14, the small area estimates of mean income show a consistent relationship to the poverty estimates. Areas with high levels of household poverty are in areas with low levels of household income, and the two indicators show a strong negative relationship. The rank of the Index of Multiple Deprivation (income component) in England shows a positive relation to the rank of the model-based estimates of households in poverty. Both of these processes have ensured that the methodology and its application are valid, the models developed are the best possible for the data available and the model-based estimates are plausible.

8. Summary and Further Work

Previous work on developing models for both BHC and AHC income variables for the three time periods 2004/05, 2006/07 and 2007/08 showed that the best estimates could be obtained for the AHC model for 2007/08. As a result, a more detailed study of a model for this period was undertaken and presented in this paper. The model chosen explains 89.5% of total between area variability (compared with the intercept only null model) and distinguishability between MSOAs defined as non-overlapping confidence intervals between low poverty and high poverty areas was 24%. In conclusion, the analysis shows that the AHC model proposed is well specified, performing well in terms of explanatory power, estimate precision and distinguishability between areas. It also performs well in other time periods. It will now be important to consult with users on the value of these now experimental statistics.

Further work, on the 2007/08 AHC model, will include a more detailed examination of some specific areas comparing the poverty estimates with the mean income estimates. The BHC based models will also be further investigated, possibly by pooling two or three years of FRS data to see if this helps improve model performance for 2004/05 and 2006/07. A Local Authority level model will also be investigated as well as other alternatives modelling approaches such as the use of composite estimators. Pooling several years FRS data may also be useful for modelling specific household types, for example, households with children or pensioner households. Lastly, an investigation of modern methods of model selection, such as Bayesian Model Averaging will be carried out together with attempts to reduce the level of correlation in the predictors using principal components analysis.

References

Brown G, Chambers R, Heady P and Heasman D (2001) ‘Evaluation of Small Area Estimation Methods – An Application to Unemployment Estimates from the UK LFS’, Proceedings of Statistics Canada Symposium in 2001.

Heady P, Clarke P, Brown G, Ellis K, Heasman D, Hennell S, Longhurst J and Mitchell B (2003) ‘Small Area Estimation Project Report’, Model-Based Small Area

14 Validation Report


SMB 68 3/11

37

Estimation Series No.2, ONS Publication. http://www.statistics.gov.uk/methods_quality/downloads/small_area_est_report/saep1 _Prelims&ch1&2_v2.pdf

Department for Work and Pensions, HBAI team (2009) ‘Households Below Average Income 1994/5 – 2007/08’.http://research.dwp.gov.uk/asd/hbai/hbai2008/contents.asp

Department of Work and Pensions (2009) ‘Family Resources Survey United Kingdom 2007-8’. http://research.dwp.gov.uk/asd/frs/2007_08/frs_2007_08_report.pdf Longhurst J, Cruddas M, Goldring S and Mitchell B (2004) ‘Model-based Estimates of Income for Wards, 1998/99: Technical Report’, Published in Model-Based Small Area Estimation Series, ONS Publication.

Longhurst J, Cruddas M and Goldring S (2004) ‘Model-based Estimates of Income for Wards, 2001/02: Technical Report’, Published in Model-Based Small Area Estimation Series, ONS Publication.

Office for National Statistics (2003) http://www.statistics.gov.uk/geography

Office for National Statistics (2005) ‘Super Output Area’. http://www.statistics.gov.uk/geography/soa.asp

Sullivan J, Gibb P, Chung R, Snow J, Beckhelling J, Vekaria R and Lord C (2009) ‘Family Resources Survey: United Kingdom, 2007-08’. http://research.dwp.gov.uk/asd/frs/2007_08/frs_2007_08_report.pdf

Valuation Office Agency (2006) ‘2004 Council Tax data’. http://www.neighbourhood.statistics.gov.uk


SMB 68 3/11

38

Appendix 1: Percentage of households (OECD) with income (AHC) < 60% of median (2007-2008 MSOA estimates (calibrated))

Source: Office for National Statistics

© Crown copyright. All rights reserved 100019153, 2010

SMB 68 3/11

39

Pruning the Labour Force Survey: Removing Respondents Aged 75 and Above from the Waves 2-5 Interviews

Karl Ashworth, Matthew Greenaway and Paul Smith

Methodology Directorate, Office for National Statistics

Summary

This article describes a change introduced to the UK Labour Force Survey (LFS) in Quarter 3 of 2010, where a longitudinal sample component was excluded in order to reduce the cost of the survey. This article details the work undertaken to evaluate the impact of this exclusion and to ensure key labour market statistics are not adversely affected. The paper first describes the LFS data collection and estimation procedures prior to the design change. We then describe the weighting adjustments to the estimation methodology under the new data collection procedures and show the results of simulations derived from using the new weighting on existing data.

1. Description of the Labour Force Survey

The Labour Force Survey (LFS) is the UK’s largest continuous social survey. It provides an up-to-date source of labour market statistics, which are widely used both within and outside of Government. In recent years, budget pressures within ONS have required the Office to introduce a number of savings, and given the large sample size of the LFS it was identified as one potential source of savings. In particular, as the principal focus of the survey is to produce labour market estimates which are normally based on either working age or economically active individuals, it was decided to explore the possibility of reducing the amount of interviewing of older people in the survey. However, it was important first to establish that any such change would not have an adverse effect on the precision of key estimates.

Since 1992, the LFS has operated a rotating panel survey design. Each calendar quarter a new cohort of addresses is added to the sample and interviewed over five successive quarters before being replaced by another new cohort. Consequently, in any given calendar quarter, the LFS has a sample that is constructed of five cohorts of respondents, each cohort is either on its first, second, third, fourth or fifth and final interview (see Figure 1).

Karl Ashworth, Matthew Greenaway Pruning the Labour Force Survey And Paul Smith

SMB 68 3/11

40

Figure 1. Illustration of LFS design and quarterly dataset structure

Time y1q1 y1q2 y1q3 y1q4 y2q1 y2q2 cohort 1 wave 5 cohort 2 wave 4 wave 5 cohort 3 wave 3 wave 4 wave 5 cohort 4 wave 2 wave 3 wave 4 wave 5 cohort 5 wave 1 wave 2 wave 3 wave 4 wave 5 cohort 6 wave 1 wave 2 wave 3 wave 4 wave 5 cohort 7 wave 1 wave 2 wave 3 wave 4 cohort 8 wave 1 wave 2 wave 3 cohort 9 wave 1 wave 2 cohort 10 wave 1

Each new quarterly cohort is selected from the small-user Postcode Address File (PAF) using an equal probability which gives a responding sample of around 55,000 addresses across the five cohorts, with all adults (aged 16 and above) at an address approached for an interview. Up until January 2011, all wave 1 interviews were carried out face-to-face15, but interviews at waves 2-5 were (and continue to be) mixed mode using telephone where possible and face-to-face otherwise. The principal reason for this mixed mode approach is to reduce costs through decreasing the number of more expensive face-to-face interviews.

2. The Revised Design: Excluding People Aged 75+

Since its inception in 1992, the five-wave structure has always applied to all responding households. The revised design retains individuals aged 75 or above (75+) for their first interview but then excludes them from interviews in waves 2-5. This construction of the survey still enables estimates to be made of the total and older populations, albeit with reduced precision, whilst making important fieldwork cost savings.

Preliminary analysis showed that individuals aged 75+ tend to be overwhelmingly economically inactive, with an employment rate of around 2%. Consequently, this subgroup tends to have little impact on labour market estimates for the whole population, and has negligible impact when estimates are only based on working-age (16-64 year old) people. It is this low prevalence of labour market participation which justifies the reduction in the amount of data collected on this particular subgroup of the population.

3. Weighting

The LFS produces both household level and person level estimates of labour market statistics. The estimation procedures for these two levels are very similar but weighting is carried out separately. Both person and household level estimates are 15 Telephone interviewing was introduced to the first wave respondents in January 2011.


SMB 68 3/11

41

based on a calibration approach but the initial weight to be calibrated is a post-hoc constructed design weight based on approximate selection probabilities from the responding sample.

The LFS uses a calibration weighting procedure to assign a weight qkw to each responding person k for each quarter q. If the wave of each person k is represented by c, and the total number of people interviewed at wave c in quarter q is qcn , then the total number of people interviewed in quarter q can be calculated as –

∑=

=5

1cqcq nn (1)

Each individual is first given a design weight qkd equal to the inverse of the

probability of selection qkπ . The design weight before the change to sampling over-75s was approximated by -

1qk

qd π= (2)

qq

q

nNπ = (3)

Where qN equals the population total at quarter q (that is, the total eligible population of the UK). This gives a design weight which is equal for all respondents, reflecting the equal probability for selection. The design weight typically was around 550.

The calibration weights qkw are then calculated to sum to a set of known calibration

totals t, minimising the distance between qkw and qkd . If the membership of the

calibration groups is represented by a set of auxiliary values kx , then the problem can be represented as -

Select kg such that ∑k

kk wddist ),(min and twx kk

k =∑ (4)

k k kw g d= (5)

The final calibration weight qkw is the product of the g weight and the design weight.

The calibration process ensures that the weights of respondents within any given calibration category are set to sum to the corresponding population total. The LFS calibration model is highly detailed and several sets of calibration categories are used, many of which, but not all, include a breakdown by age within their cross-tabulated totals.

Calibration has a number of desirable properties. One is that estimates of key variables are consistent with known population totals across different surveys. ONS uses calibration across its social surveys in order to:


SMB 68 3/11

42

• At least partially compensate for any differential non-response effect on outcomes that are linked to the calibration model; this can often result in bias reductions. This potential bias reduction is important for the LFS because attrition is not explicitly adjusted for outside of calibration.

• Ensure that users have estimates which are consistent with the population totals by age, sex and region published by ONS, and that such estimates are consistent across different surveys carried out at the same point in time.

• Finally, if the calibration model is a good predictor of the parameter being estimated, estimates can be more precise (have lower variance) when using the calibration model in the variance estimator.

There are two weights calculated on the LFS: an individual weight and a household weight. The household weight uses the same design weight as the person level weight as an input to calibration. However, all eligible individuals enumerated in the household (whether respondent or not) are included in the auxiliary vectors for constructing the calibration model. This procedure results in an integrative calibration solution (Lemaître & Dufour, 1987); each person in the household receives the same weight, which is also the household weight. It enables an unbiased estimate of the number of households. In the absence of non-response within respondent households, this would be the preferred weight also for the person level dataset. However, we use a person level weight calibrated on the respondents for the individual level data. This procedure corrects (to some extent) for partial non-response within households and forces consistency in estimated population totals from the respondent dataset.

4. New Weighting Strategy

A simple adjustment is to calculate separate design weights for the group of respondents age 75+ and those aged below 75 and increase the design weight by a factor of five (number of cohorts) for respondents aged 75+. However, as selection probabilities are approximated on the basis of the size of the respondent sample across each cohort, the respondent numbers are subject to attrition. Consequently, simply multiplying the design weight for 75+ Cohort 1 respondents by a factor of five would be equivalent to assuming no attrition for this age group over their next four interviews. The result of this would be to over-represent people aged 75+ in relation to their expected sample size in the quarter and would undermine our attempt to remain consistent with the previous estimation procedure.

Our solution was to make separate adjustments for each age group where qn is calculated as in (3) for group=1 (respondents aged under 75). Whereas, for group=2 (respondents aged 75+) qn is calculated as the product of the sample size for wave 1

over-75s ( 75n ) and an attrition adjusted ( nrw ) number of cohorts ( C ). We

estimated nrCw =4.26 using recent LFS person-level data collected prior to implementing the design change, and it will remain fixed in the future.


SMB 68 3/11

43

⎪⎩

⎪⎨

⎧

=

=∑==

2,

1,

75

5

1

groupnCw

groupnn

nrc

cqgroup (6)

We note that calibration could have been used without adjusting the design weight by a constant ( nrCw ) for respondents age 75+. This would still have achieved the correct calibration totals for the 75+ group because over-75s make up one of the calibration groups. However, the minimization function would have resulted in much larger calibration weights than is achieved using (6) as an input. Ignoring the design weight adjustment will increase the variance of the weights and consequently the size of the standard error of an estimated population parameter. Moreover, as ONS uses generalised regression (GREG) estimation techniques to calculate standard errors for the LFS, the design weight would have been incorrect, introducing error into this estimator.

Further complications arise in that mixed households, i.e. those with at least one person under 75 and at least one person aged 75+, will continue to be interviewed in full in Waves 2-5. For the person level dataset this is resolved through exclusion from the dataset of individuals aged 75+ in waves 2-5.

The integrative calibration procedure used in household weighting requires the additional constraint that all individuals enumerated in the household must be accounted for in the calibration. Consequently, we cannot exclude respondents aged 75+ in Waves 2-5 in mixed households, and these individuals will still receive a weight (albeit one that is not used in practice for person level estimation). However, households where all respondents were aged 75+ were excluded from Waves 2-5 and the household design weight was adjusted following the procedures outlined in (6), using the same constant as is used for adjusting the person level design weight for respondents aged 75+.

5. Results

There are two components of error explored here through comparing the original and revised designs. The first is the potential for bias, i.e. the extent to which an estimator produces a value which differs systematically from the truth. The second component is sampling error, which is a measure of precision.

Labour market statistics estimate various parameters concerned with measures of population size or amount, typically totals, means and ratios (e.g. number of employed males to number of working age males). In addition, each of these estimates of amount has an associated standard error, which is also estimated from the data and reflects the extent to which the sample estimate varies around its expected value. It is a measure of precision, with a smaller standard error reflecting increased precision.

Bias is measured in this paper through taking the difference between the estimate obtained using the full sample and its associated weighting scheme compared to the estimate produced by the revised scheme. Standard errors are calculated for each of these estimates to obtain estimates of precision. Precision is expected to be minimally affected for estimates based on the working age or economically active


SMB 68 3/11

44

populations, decreased slightly for the total population and decreased substantially for the older population.

Estimates of the impact of the design change were created by removing Wave 2-5 respondents aged 75+ from the January-March 2009 (JM09) and April-June 2010 (AJ10) LFS datasets and re-weighting the data using the new weighting scheme. Results from both sets of data were very similar so only a selection of results is presented. We note that there may also be an impact from under-75s moving into households previously occupied only by over-75s. While the results reported here does not take this into account, separate analysis suggests that there are very few such cases in any given quarter and the impact on estimates will be negligible.

We first compare the impact of the revised weighting scheme on the final calibration weight before comparing estimates and their corresponding standard errors on the individual person level respondent dataset. Finally, results are shown for a small selection of household level estimates. The estimates used throughout this report are not seasonally adjusted so will differ from ONS published figures.

5.1 Properties of the weights

The average weight under the original design was 552, which increased to 588 under the revised design (Table 1). The impact of the exclusion of the 75+ group is most notable through an increase in the maximum weight, from 1907 to 7527, and the corresponding impact on the variance of the weights.

Table 1. Change in individual weight properties, AJ10

Prior weights, overall

New weights, overall

Mean weight 552 588Maximum weight Minimum weight

1,907 166

7,527161

Variance of weights 19,360 86,062Coefficient of variation

25 50

One criterion for success was minimal impact on the under-75 respondent group, which was achieved under the new design: the mean weight was 555 under both designs, the maximum weight and variance increased only slightly under the revised design and the coefficient of variation remained constant (Table 2). The high degree of correspondence between the old and new weights is also shown diagrammatically in Figure 2.

Table 2. Change in individual weight properties, AJ10, under-75s only

Prior weights, overall New weights, overall

Mean weight 555 555Maximum weight 1,907 1,918Minimum weight 166 161Variance of weights 19,614 19,822Coefficient of variation

25

25


SMB 68 3/11

45

Figure 2. Plot of prior and new weights for respondents under 75, JM09

The impact of the re-weighting on the over 75s was to increase the mean weight from 512 to 236516. The minimum and maximum weights increased by around a factor of 5 and the variance was dramatically increased (Table 3). However, the coefficient of variation rose only slightly from 24 to 28. Table 3. Change in individual weight properties, AJ10, over-75s only

Prior weights, overall New weights, overall Mean weight 512 2365Maximum weight 1,782 7,527Minimum weight 166 875Variance of weights 14,675 435,381Coefficient of variation

24 28

In conclusion, when under-75s only are considered, properties of the weight remain stable, with only a small increase to the variance and range. This finding demonstrates that the change to the sampling and weighting of over-75s does not substantially affect the weights for the under-75s. These results consolidate our expectation that the precision of the estimates for the working age population will remain stable under the old and new designs with the decreased precision only apparent for estimates based on the 75+ group and the full population.

5.2 Change in labour market estimates – individual level

The impact of the new sampling and weighting scheme on economic activity estimates for people aged under 75 was negligible both overall and when distinguished by respondent gender (Table 4).

16 This increase is equivalent to multiplication through by the factor 4.62 ( nrCw in equation (6)).


SMB 68 3/11

46

Table 4. Change in UK individual-level labour market estimates, AJ10

Prior estimate

New estimate

Percentage change in estimate

total employed male 15,464,801 15,472,540 0.05 total employed female 13,468,382 13,473,094 0.03 total employed all 28,933,183 28,945,634 0.04 total unemployed male 1,460,708 1,459,250 -0.10 total unemployed female 972,478 972,940 0.05 total unemployed all 2,433,186 2,432,190 -0.04 total inactive male 7,367,915 7,361,634 -0.09 total inactive female 11,075,881 11,070,707 -0.05 total inactive all 18,443,796 18,432,341 -0.06 rate employed male 75.27% 75.29% 0.02 rate employed female 65.34% 65.35% 0.01 rate employed all 70.29% 70.29% 0.01 rate unemployed male 8.63% 8.62% -0.14 rate unemployed female 6.73% 6.73% 0.01 rate unemployed all 7.76% 7.75% -0.08 rate inactive male 17.46% 17.45% -0.04 rate inactive female 29.83% 29.83% -0.02 Rate

Inactive

all

23.67% 23.67% -0.03

Table 5. Change in UK standard errors for individual-level estimates, AJ10

Prior SE New SE Percentage change in SE

total employed male 46,618 47,320 1.51 total employed female 50,400 50,743 0.68 total employed all 73,703 74,557 1.16 total unemployed male 28,262 28,301 0.14 total unemployed female 21,927 22,050 0.56 total unemployed all 36,739 36,846 0.29 total inactive male 41,802 42,566 1.83 total inactive female 48,848 49,224 0.77 total inactive all 68,687 69,596 1.32 rate employed male 0.226% 0.226% 0.06 rate employed female 0.259% 0.259% 0.10 rate employed all 0.183% 0.183% 0.06 rate unemployed male 0.165% 0.165% 0.20 rate unemployed female 0.151% 0.152% 0.53 rate unemployed all 0.116% 0.117% 0.32 rate inactive male 0.201% 0.201% 0.06 rate inactive female 0.250% 0.250% 0.08 Rate

Inactive

All

0.168% 0.168% 0.04


SMB 68 3/11

47

There was no substantial impact on estimates of levels or on the economic activity rates. Following LFS convention, levels are presented as estimates for the population aged 16+, employment and inactivity rates are for the working age population, and unemployment rates are for the economically active adult population.

The impact of the design change on the corresponding standard errors of the estimates was also relatively small and the standard error increases were deemed insubstantial (Table 5). The maximum increase for standard errors of level estimates was 1.8%. All standard error increases for rates were less than one per cent.

Regional analysis is another key labour market output and each region is, necessarily, based on a smaller sample size than the total population. Even here, using Government Office Region (GOR) there is no noticeable design change impact on estimates of unemployment rates (Table 6).

Table 6. Change in regional unemployment rates, AJ10

Government office region

Prior unemployment rate

New unemployment rate

Percentage change

North East 9.4% 9.4% 0.11North West 7.8% 7.8% -0.19Yorkshire 9.2% 9.2% 0.02East Midlands 7.5% 7.5% -0.68West Midlands 8.4% 8.4% 0.15Eastern 6.8% 6.8% -0.57London 9.2% 9.2% -0.04South East 6.0% 6.0% -0.19South West 6.1% 6.1% 0.44Wales 8.6% 8.6% 0.11Scotland 8.5% 8.5% 0.01Northern Ireland

6.6% 6.6%

-0.16

The impact on standard errors was assessed on a finer regional breakdown than GOR, to provide a more stringent comparison on smaller groups. This comparison showed that a small number of areas showed decreases in precision, worst affected was Strathclyde with an increase in the standard error by six per cent. Conversely, some areas showed small improvements in precision. This is because standard errors are estimated from the data, and are also subject to sampling error; therefore it is possible for them to decrease slightly even with a smaller sample size.

Labour market statistics based on changes over time are also key indicators. Tables 8 and 9, respectively, show the impact of the design on standard error estimates both for the total employment level in the AJ10 quarter and the one-quarter change between AJ10 and the previous quarter. For both level and change there is minimal impact on standard errors by age group until reaching the age group 65+, when the standard error increases by around 30%. For people aged 75+, the standard error increases dramatically by 173% for a level and 354% for change. The reason for the larger impact on the estimate of change is because there is no longer an overlap in the 75+ sample across the quarters under the revised design. Consequently, the covariance over time is zero and the variance of change is around twice that of level.


SMB 68 3/11

48

Table 7. Change in regional standard error for employment levels, AJ10

Government office region

Prior employment total SE

New employment total SE

Percentage change

Tyne & wear 10,289 10,325 0.35 Rest of NE 10,894 10,896 0.01 G. Manchester 19,673 19,564 -0.55 Merseyside 17,066 16,545 -3.05 Rest of NW 15,582 16,136 3.55 S. Yorkshire 15,101 14,640 -3.05 W. Yorkshire 17,795 17,290 -2.84 Rest of Yorkshire 11,769 11,710 -0.50 E. Midlands 18,776 19,021 1.30 W. Midlands Met. 17,030 17,135 0.61 Rest of W. Mids. 14,539 14,636 0.67 E. England 20,659 20,859 0.97 Inner London 20,356 20,508 0.75 Outer London 22,551 22,522 -0.13 S. East 25,261 25,507 0.97 S. West 20,237 20,787 2.72 Wales 17,128 17,170 0.25 Strathclyde 13,837 14,651 5.89 Rest of Scotland 15,213 15,292 0.52 N. Ireland

10,984 11,132 1.35

Table 8. Standard errors for total employment by age-group, AJ10

Estimate

Prior

New Percentage change in SE

Total Employed 18-24 36,379 36,086 -0.806 Total Employed 25-34 31,626 31,052 -1.814 Total Employed 35-49 35,934 35,821 -0.316 Total Employed 50-64 36,805 36,961 0.425 Total Employed 65-74 18,819 19,128 1.641 Total Employed 65+ 19,850 25,834 30.143 Total Employed 75+

6,176

16,834 172.587

In conclusion, for respondents aged under 75, the redesign shows no adverse impact on point estimates. No key estimates of population amounts changes by more than 0.4%, and all changes in estimates of amounts are well within the sampling errors for the estimates. Impacts on standard errors are slightly higher but are still judged to be acceptable. The main impact is, as expected, confined to those estimates including the 75+, where precision is reduced.


SMB 68 3/11

49

Table 9. Standard errors for change in total employment, JM10-AJ10

Prior New Percentage change in SE of change

Total Employed 18-24 30,868 30,620 -0.81Total Employed 25-34 26,835 26,349 -1.81Total Employed 35-49 30,491 30,395 -0.32Total Employed 50-64 31,230 31,362 0.42Total Employed 65-74 15,969 16,230 1.64Total Employed 65+ 16,843 21,921 30.14Total Employed 75+

5,240 23,807 354.31

5.3 Change in labour market point estimates – household level

A corresponding investigation was carried out on the LFS household estimates. As with the person level investigation, 75+ households were removed from the April-June 2009 LFS household dataset and this dataset was re-weighted using the new weighting scheme.

Table 10. Change in household-level labour market estimates, AJ09

Prior estimate

New estimate

Percentage change

North East 25.6% 25.8% 1.02North West 20.8% 21.0% 0.62Yorkshire 21.5% 21.6% 0.75East Midlands 18.4% 18.5% 0.82West Midlands 21.0% 21.2% 0.90Eastern 15.5% 15.6% 0.71London 21.0% 21.1% 0.48South East 15.2% 15.3% 0.79South West 18.1% 18.2% 0.55Wales 23.4% 23.7% 0.90Scotland 21.6% 21.8% 0.88Northern Ireland 23.6% 23.8% 0.76

Rate

Workless households All 19.9% 20.0% 0.71

White 5,147,092 5,191,997 0.87Mixed 78,880 79,745 1.10Asian 341,970 342,867 0.26Black 225,903 228,641 1.21Chinese 44,031 44,854 1.87Other 151,438 153,035 1.05

Total

Working age people in workless households

All

5,995,960 6,047,888

0.87

The conclusion was that key estimates were relatively unaffected. The largest impact on the workless household rate was for the North-East, which increased by 1 per cent from 25.6% to 25.8% (Table 10). The estimate of the number of Chinese workless


SMB 68 3/11

50

people of working age rose by 1.8%, the largest increase by ethnic group. These increases were well within the standard error under the original design.

A similar story emerged when comparing standard errors of household estimates under the two designs. As with person level data, the impact on standard errors was larger than on location estimates; but again increases were relatively minor. The impact on standard errors for totals, which are based on all adults aged 16+ in the population, is greater than the impact on standard errors for rates, which are based only on the working-age population.

6. Conclusions

The simulations supported the requirements that the redesign resulted in no adverse impact on bias and minimal impact on precision of estimates for the working age or economically active population, although the precision of estimates where over-75s make up a large part of the base will be substantially reduced. Consequently, we concluded that a smaller sample of over-75s would result in a substantial saving with only a moderate (and acceptable) impact on the quality of key LFS estimates. Accordingly, the decision was taken to exclude households where all members are over the age of 75 from waves 2-5 of the survey, interviewing them only in wave 1 from July 2010. Over-75s in a household with individuals under the age of 75 will be retained in the sample and re-interviewed.

References

Fuller W A (2002) ‘Regression Estimation for Survey Samples’, Survey Methodology 28, pp 5-23. Lemaître G and Dufour J (1987) ‘An integrated method for weighting persons and families’, Survey Methodology 13, pp 199-207. Werner B (2006) ‘Reflections of fifteen years of change in the LFS: How the UK’s unemployment statistics were transformed by using the LFS’ Labour Market Trends August 2006, available at http://www.statistics.gov.uk/articles/labour_market_trends/Reflections.pdf

Forthcoming Conferences, Seminars and Courses

SMB 68 3/11

51


1. Young Statistician’s Meeting (YSM2011) The next Young Statistician’s Meeting will be held at the University of Southampton on 12-14 April 2011. The Young Statisticians' Meeting (YSM) is an annual event specifically designed for career young statisticians, organised by career young statisticians. Further information is available at: https://sites.google.com/site/ysm2011conference/presentations 2. UNECE Work Session on Statistical Data Editing This conference will be held in Ljubljana, Slovenia on 9-11 May 2011 3. Joint Statistical Meeting (JSM) The 2011 Joint Statistical Meetings will be held July 30 - August 4 2011, at the Miami Beach Convention Center, located at 1901 Convention Center Drive, Miami Beach, FL 33139-1820.

For information, contact [email protected] or phone toll-free (866) 421-7169. Further information is available at: www.amstat.org/meetings/jsm/2011/index.cfm 4. ASC 6th International Conference The 6th ASC International Conference will be held at Wills Hall, Bristol University, UK on 22-23 September 2011. Topics for the conference are currently being finalised with a full call for papers taking place in early September 2010. We anticipate that topics will include developments in cloud computing, census developments, web analytics, mobile computing, online panels, researching global markets, developments in language translation, Social Networks, EFMs to name but a few…. We strongly urge you to put the dates in your diary and complete the contact form at http://www.asc.org.uk/contact specifying BRISTOL 2011 in the subject field. We will then keep you up to date as planning progresses. For more information about this event please visit: www.asc.org.uk/activities/conferences/september-2011


SMB 68 3/11

52

5. IC Health Surveys Programme Network.

This forum is designed to inform and update all interested parties on the NHS IC’s current programme of health surveys.

The Survey Team commission and manage contracts with external bodies to undertake a programme of household and individual based surveys on behalf of the Health and Social Care Community.

Membership of the NHS IC's Health Surveys Programme Network is open to all interested parties, if you know of others who would like to receive regular updates on the NHS IC Survey Programme, please ask them to get in touch ([email protected]) and we will add them to our mailing list.

6. The Cathie Marsh Centre for Census and Survey Research

Courses Date

Social Network Analysis 1: Introduction to Social Network 11/04/2011 Analysis using UCINET and NETDRAW

Parental characteristics and intergeneration transmission 12/04/2011 of party preference in multiparty systems: evidence from Switzerland

Social Network Analysis 2: Advanced Methods for One 13/04/2011 Mode, Two Mode and Egonetworks

Social Network Analysis 3: Statistical Analysis of Social 13/04/2011 Networks

Multiple Linear Regression 26/04/2011

Logistic Regression 27/04/2011

Analysing and Evaluating Social Change in the Human 10/05/2011 Empowerment Framework

Event History Analysis 11/05/2011

Multilevel Modelling 13/05/2011

Demographic Concepts and Methods 16/05/2011

A multilevel simultaneous equations model for within-cluster 17/05/2011 dynamic effects, with an application to reciprocal relationships between maternal depression and child behaviour

Population Estimating and Forecasting 18/05/2011

Demographic Forecasting with POPGROUP 19/05/2011

Understanding US mid-term elections 24/05/2011

The Psychology of ‘Nudge’ and Social Cohesion 31/05/2011


SMB 68 3/11

53

Longitudinal Data Analysis 01/06/2011

Causal Analysis in STATA 06/06/2011

Fuzzy Set Analysis 07/06/2011

The antecedents, effects and multi-dimensionality of issue 07/06/2011 ownership

Family Resources Survey User Meeting 11/06/2011

Introduction to R 20/06/2011

7. MSc in Official Statistics

This MSc programme has been developed jointly by the University of Southampton with National Statistics to cover the core skills and knowledge needed by professional government statisticians working in the UK and in other countries:

Semester 2

Course Date

DEMO6022 – Demographic methods II 21-25 Mar 2011 Southampton STAT6039 – Analysis of longitudinal data (data 21 Mar-1 Apr 2011 analysis V) Southampton STAT6038 – Index numbers 4-8 Apr 2011 Southampton STAT6041 – Time series modelling 16-20 May 2011 Southampton 8. The Centre for Applied Social Surveys (CASS) The Centre for Applied Social Surveys (CASS) hosts a programme of short courses, which is managed by the Southampton Statistical Sciences Research Institute at the University of Southampton. www.s3ri.soton.ac.uk/cass/programme.php Course Date Structural Equation Modelling for Cross-Sectional 25-27 May 2011 and Panel Data (This course is now full. (Waiting list available) Longitudinal Data Analysis: Population Average and 29 Jun – 1 Jul 2011 Random Effects Models


SMB 68 3/11

54

Questionnaire Design 3-4 Oct 2011

Questionnaire Design 10-11 Nov 2011

Survey Data Analysis 1: Introducing Descriptive and 23-25 Nov 2011 Inferential Statistics

Applied Multilevel Modelling 25-27 Jan 2012

Survey Data Analysis 1: Introduction to Linear Regression 29 Feb-2 Mar 2012 Modelling

Handling Non-Response in Sample Surveys 5-6 Mar 2012

Regression Methods 18-20 Apr 2012

Essentials of Survey Design and Implementation tba

Structural Equation Modelling for Cross-Sectional and tba Panel Data

SMB 68 3/11

55

The National Statistics Methodology Series

Currently available:

1. Software to Weight and Gross Survey Data, Dave Elliot

2. Report of Task Force on Seasonal Adjustment

3. Report of the Task Force on Imputation

4. Report of the Task Force on Disclosure

5. Gross Domestic Product: Output methodological guide, Peter Sharp

6. Interpolating annual data to monthly or quarterly data, Michael Baxter

7. Sample design options for an integrated household survey, Dave Elliott and Jeremy Barton

8. Evaluating non-response on household surveys, Kate Foster

9. Reducing statistical burdens on business, Andrew Machin

10. Statistics on Trade in Goods, David Ruffles

11. The 1997 UK pilot of the Eurostat Time Use Survey, Patrick Sturgis and Peter Lynn

12. Monthly statistics on Public Sector Finances, Jeff Golland, David Savage, Tim Pike and Stephen Knight

13. A review of sample attrition and representativeness in three longitudinal surveys, Gad Nathan

14. Measuring and Improving Data Quality, Vera Ruddock

15. Gross Domestic Product: Output Approach, Peter Sharp

16. Report of the Task Force on Weighting and Estimation, Dave Elliott

17. Methodological Issues in the Production and Analysis of Longitudinal Data from the Labour Force Survey, P S Clarke and P F Tate (Winter 1999)

18. Comparisons of income data between the Family Expenditure Survey and the Family Resources Survey, Margaret Frosztega and the Households Below Average Income team (February 2000)

19. European Community Household Panel: Robustness Assessment Report for United Kingdom Income Data, Waves 1 and 2 (March 2000)

20. Producer price indices: principles and procedures, Ian Richardson (March 2000)

The National Statistics Methodology Series

SMB 68 3/11

56

21. Variance estimation for labour force survey estimates of level and change, D.J. Holmes and C.J. Skinner (May 2000)

22. Longitudinal data for policy analysis, Michael White, Joan Payne and Jane Lakey (May 2000)

23. Report on GSS survey activity in 1997, Office for National Statistics, Survey Control Unit (December 2000)

24. Obtaining information about drinking through surveys of the general population, Eileen Goddard (January 2001)

25. Methods for automatic record matching and linking in their use in National Statistics, Leicester Gill (July 2001)

26. Designing surveys using variances calculated from Census data, Sharon Bruce, Charles Lound, Dave Elliot (March 2001)

27. Report on GSS survey activity in 1998, ONS Survey Control Unit, (September 2001)

28. Evaluation and criteria for statistical editing and imputation, Ray Chambers, (September 2001)

29. Report on GSS survey activity in 1999, ONS Survey Control Unit (October 2001)

30. The value and feasibility of a national survey of drug use among adults in the UK, Eileen Goddard (April 2002)

31. Report on GSS survey activity in 2000, ONS Quality Centre, (April 2002)

32. Gross domestic product: output approach (Gross Value added) – Revised, Peter Sharp

33. Life expectancy at birth: methodological options for small populations, Barbara Torson, Allan Baker

34. Making a population estimate in England and Wales, Julie Jefferies and Ruth Fulton

35. Smoothing of Standardised Mortality Ratios: A preliminary investigation

36. Statistics on Trade in Goods – Revised

37. Estimating alcohol consumption from survey data: updated method of converting volumes to units

38. 100 Years of the Census of Production in the UK, Paul Smith (October 2009)

Reports No. 5 and 15 are £20 each; the remaining reports are each priced at £5.All reports are available to GSS members free of charge.

The above publications are available by calling +44(0)1329 813060.

Or visit www.statistics.gov.uk/methods_quality/publications.asp

tracking procedures era

Documents