united nations economic commission for europe statistical division applying the gsbpm to business...

United Nations Economic Commission for EuropeStatistical DivisionUnited Nations Economic Commission for EuropeStatistical Division

Applying the GSBPM to Business Register Management

Steven ValeUNECE

steven.vale@unece.org

Background - Changing roles for NSOs?

Data integration Quality assurance More focus on analysis and interpretation Partnerships for dissemination

Changing staff and cost profiles Changing organisational culture

Opportunities and threats for statistical business registers

Reduced role of surveys and sampling frames Greater use of external and mixed data sources BR becomes “gateway” for business data

More satellite registers? More sophisticated matching techniques needed More integration between statistical registers Register or business statistics database? Source of new statistics

Mapping business register processes to GSBPM

New Eurostat project:• Build up the capacity for using GSBPM and

GSIM to document the national statistical business register processes

• Describe national statistical business register processes by using GSBPM and GSIM

• Grants available

Does GSBPM apply to BRs?

Business register maintenance is a continuous activity, not a single process

But BRs have:• Inputs “collected” from different sources• A sequence of processing and analysis• Outputs – statistics and sampling frames

Therefore BR maintenance can be seen as similar to other statistical production

5.4 Edit & Impute

- Impute for missing units or variables - Edit “unusual” data according to pre-defined edit rules

Collect Process Analyse Disseminate

4.1 Create frame &Select

sample

4.2 Set up collection

- Prepare for incoming survey and admin data files

4.3 Run collection

- Receive data files

4.4 Finalise collection

5.1 Integrate data

- Load data - Perform automatic / clerical matching

5.2 Classify & code

- Perform automatic / manual coding of variables

5.3 Review & validate

- Perform validation checks

5.5 Derive new variables

& units - Profiling

5.7 Calculate aggregates

- Population totals

6.1 Prepare draft outputs - Create “dummy” sampling frames or data outputs

6.2 Validate outputs

- Check quality - Compare to expected values and other data sources or time periods

6.3 Interpret & explain

outputs - Explain any differences found in 6.2

6.4 Apply disclosure

control - To any outputs made available to those not authorised to see micro-data

6.5 Finalise outputs

- Prepare and authorize for release

7.1 Update output systems - Load data into output databases

7.2 Produce dissemination

products - Tables, publications, datasets etc.

7.3 Manage release of

dissemination products

7.5 Manage user support

- Answer queries from survey statisticians

7.4 Promote dissemination

products

5.6 Calculate weights

5.8 Finalise data files

- Maximize register quality prior to creating outputs

The dotted lines show the different routes depending on whether the output is a sampling frame or a statistical product

Is this just an academic exercise?

No – there are practical benefits:• Standardisation of terminology• Standard framework for benchmarking• Facilitates use of common tools / methods• Efficiency savings• Tool for managing process quality

Detailed application of GSBPM to statistical business registers

Phases 1-3

Relevant for business register re-engineering, but not for regular management and maintenance

Same principle as for regular surveys

DesignSpecify Needs Build

Covers the activities necessary to prepare to receive the incoming data• Survey data• Administrative sources• Other data sources

Includes configuring systems and processes

4.2

Set up collection

Refers to the task of obtaining data, e.g.• Receive a tax data file• Receive a survey file

It includes managing relationships with data providers

4.3

Run collection

Refers to the task of loading data to the business register, e.g.• Load tax data file• Load survey file

It can be an ad-hoc activity, e.g.• Manually input data from a company web site

4.4

Finalise collection

This activity covers automatic and clerical matching between units from different sources, using:• Common identification numbers• Name / address / other variables

5.1

Integrate data

The allocation of codes, based on textual descriptions and/or other variables, e.g.• Economic activity codes• Geographical codes• Legal status codes

Can be automatic or manual

5.2

Classify & code

Checking units, variables and aggregates to identify possible anomalies, errors or missing data

Can happen in different places in the statistical production process

5.3

Review & validate

The treatment of anomalies, errors and missing data found in 5.3, including:• Imputing missing values• Correcting errors

Note: the risk of introducing biases should be considered

5.4

Edit & impute

Includes the derivation of different types of statistical units (profiling)

Also includes the creation of derived variables such as turnover per employee

5.5

Derive new variables & units

Includes the calculation of population and sub-population totals to support the creation of sampling frames

Also includes the preparation of aggregate data on business demography and other types of statistics directly based on business registers

5.7

Calculate aggregates

Includes actions to maximise and verify register quality before creating outputs, such as:• Survey frames• Statistics and analyses

5.8

Finalise data files

This includes the preparation of “dummy” sampling frames or data outputs

It is usually done to check quality

6.1

Prepare draft outputs

Includes actions to check the quality of register outputs, such as:• Survey frames• Statistics and analyses

This can include comparisons with expected values or outputs for previous periods or from other sources

6.2

Validate outputs

Includes investigation and explanation of any issues found in 6.2

This may require checking source data and/or re-running some previous sub-processes

6.3

Interpret & explain outputs

Only applies for statistical outputs from business registers that will be published, e.g. business demography data

6.4

Apply disclosure control

This includes the final approval of register outputs for release, as well as providing any supporting information:• Metadata• Text explaining unusual values

6.5

Finalise outputs

Includes loading data into output databases

Only relevant for data outputs

7.1

Update output systems

Includes creating tables, web, pdf or paper publications, micro-data sets for researchers etc.

7.2

Produce dissemination products

Provision of data to users / subscribers• Survey statisticians• Eurostat

Managing access to confidential data

7.3

Manage release of dissemination

products

Answering queries from survey statisticians or external users of data products

7.5

Manage user support

List of functions

Co-operation with sources and data users

Identifying new sources

4.3

Run collection

2.3

Design collection

1.2

Consult & confirm needs

7.5

Manage user

support

1.5

Check data availability

8.2

Conduct evaluation

List of functions

Analyzing the quality of incoming data.

Development of data processing rules

5.3

Review & validate

6.2

Validate outputs

2.5

Design processing & analysis

List of functions

Updating the register

Processing the data requests (SQL queries from the register)

4.3

Run collection

Process

Process7.5

Manage user support

List of functions

Process of producing the frame Maintenance of the frame

6.1

Prepare draft outputs

6.2

Validate outputs

6.3

Interpret & explain outputs

6.5

Finalise outputs

List of functions

Register developments

Profiling activity

8.3

Agree an action plan

Specify Needs

Design Build

4.3

Run collection

5.1

Integrate data

5.5

Derive new variables & units

List of functions

Data exchange with Eurostat and other users

Dissemination of data

7.2

Produce dissemination

products

7.3

Manage release of dissemination

products

7.2

Produce dissemination

products

7.3

Manage release of dissemination

products

4.3

Run collection

List of functions

Quality checks of the register

… and all GSBPM sub-processes

Evaluate

Conclusion

GSBPM can be applied to statistical business register maintenance

There is clear potential benefits in terms of shared knowledge, methods and tools

Questions and Comments?

steven.vale@unece.org

www.unece.org/stats/gsbpm