1

SASQAFSouth African Statistical Quality Assessment

Framework

21 November 2013Ravi Naidoo

2

OVERVIEW

• Foreword• The NSS• What is SASQAF• Why SASQAF• Dimensions of SASQAF• Where SASQAF• The SASQAF Assessment Process• Purpose of official Statistics

3

“The environment speaks volumes to those who produce statistics”

Statistician General, Pali Lehohla, quoted this in his introductory note during the implementation of the 2011 – 2015 Strategic Plan Document.

He further mentions that the challenge that Stats SA faces, is that the statistical environment is new and requires not only cycle focused statistics but statistics that relate to structure, thus Stats SA must articulate how it position itself in response to the new environment.

• to expand the statistical information base by increasing its depth, breadth and geographic spread.

• to lead the development & coordination of statistical production within SA• to enhance public confidence and trust in statistics• to improve productivity and service delivery• to invest in the learning & growth of the organization• to promote international cooperation and participation in statistics

Foreword

4

NSS

There is a buzzword frequently heard in Stats SA, and now perhaps more so outside Stats SA too. This word is the “NSS”, which means National Statistics System.

There are various models of a National Statistics System. In the South African context, the definition of the NSS is:

“The National Statistics System is a grouping into a ‘partnership’ of users, producers and suppliers of official and other statistics for the ‘Efficient production of statistics’ and ‘Effective use’ of these statistics for planning and decision making”

These 3 concepts prompted Stats SA to promote the establishment of the NSS in South Africa.

5

WHAT IS SASQAF

• SASQAF is one of the statistical tools or framework at the centre of a National Statistical System.

• Gazetted in September 2009, this framework will focus on the ‘quality improvement’ of all official statistics that ultimately forms milestones in the building of the NSS.

• The main focus of SASQAF which serves to deliver data of high integrity, is mainly to address the question of “QUALITY”

• The framework of SASQAF comprises of 8 Quality dimensions which play a pivotal role in achieving this objective.

• SASQAF is also a framework that is in line with the United Nations Fundamental Principles of Official Statistics, which means there is coherence with “International best practice”

6

WHY SASQAF? Quality framework• Statistics must and be should be assessed in terms of their

quality and fitness for use.

• Drive to improve the quality of Stats SA products

• For survey areas and organs of state to produce official statistics “by design and not by default”

• the need to certify as official statistics, statistics produced by other organs of state or government and who are members of the NSS.

• to expand the statistical information base by increasing its depth, breadth and geographic spread.• to lead the development & coordination of statistical production within SA• to enhance public confidence and trust in statistics• to improve productivity and service delivery• to invest in the learning & growth of the organization• to promote international cooperation and participation in statistics

7

Provide a flexible structure for the assessment of Statistical products

SASQAF…

Statistical product

• Self assessment by producers of statistics

• Reviews by a Data Quality Assessment Team (DQAT) in the context of the NSS

• Assessment by users of Statistics based on the producers quality declaration

• Assessment by international agencies based on the quality declaration

Uses of SASQAF

8

QUALITY DIMENSIONS

9

QUALITY DIMENSIONS

1. Relevance• The degree to which the data meets the real needs of clients. It is

concerned with whether the available information sheds light on the issues most important to the users.

2. Accuracy• The degree to which the output correctly describes the phenomena it was

designed to measure

3. Timeliness• Refers to the delay between the reference point to which the information

pertains and the date on which the information becomes available.

4. Accessibility• Statistical information and metadata refers to the ease with which it can be

obtained from the producing agency. Also the ease with which the existence of information can be ascertained and suitability.

10

QUALITY DIMENSIONS

5. Interpretability• The ease with which users understand statistical information through

provision of metadata

6. Coherence• The degree to which it can be successfully brought together with other

statistical information within a broad analytic framework and over time

7. Methodological Soundness• The application of international standards, guidelines and agreed practices

to produce statistical outputs. Application of such standards fosters national and international comparability

8. Integrity• Integrity refers to the values and related practices that maintain user’s

confidence in the agency producing statistics or statistical product.

11

QUALITY DIMENSIONS review

12

The SASQAF Assessment Process

This assessment is undertaken by a Data Quality Assessment Team, (DQAT).

Data owners/producers who wishes to have their statistics certified should satisfy the following criteria:

• the producing must be an member of the NSS• the statistics need to meet user needs beyond those

specific and internal to the producing agency; and • the statistics produced should be part of sustainable

series, not once-off collection.

13

SASQAFSASQAF correctly applied enables DQAT to certify statistics on one of four levels:

• Level 4: Quality Statistics – these are statistics that meet all the quality requirements as set out in SASQAF.

• Level 3: Acceptable Statistics – These are statistics that meet most, but not all the quality requirements as set in SASQAF.

• Level 2: Questionable Statistics – These are statistics that meet few of the quality requirements as stipulated in SASQAF.

• Level 1: Poor Statistics – These are statistics that meet almost none of the quality requirements of SASQAF.

DQAT makes recommendations where improvements might be/will be needed.

14

PURPOSE OF OFFICIAL STATISTICS

The dimensions of quality are therefore in line with the “Statistics Act” No. 6 of 1999 which defines the purpose of official statistics as follows:

15

PURPOSE OF OFFICIAL STATISTICS

16

Thank You