1

Data Quality Standardsat the

U.S. Census Bureau

Pamela D. McGovern and John M. BusheryU.S. Census BureauQuality Program Staff

Washington, DC 20233

European Conference on Quality in Official StatisticsJuly 2008

2

Agenda

• Background• Development Process• Framework and Structure• What Worked Well• Next Steps• Challenges

3

Background

4

Office of Management and Budget (OMB)

• Provides direction on Government-wide budget and management issues

• Issued Information Quality Guidelines (2002)• Issued Standards and Guidelines for

Statistical Surveys (2006)

5

U.S. Census Bureau

Composed of separate program directorates:• Economic• Demographic• Decennial

6

Quality Program

• Initiated in 1998• Purpose

– Advance and maintain the Census Bureau interests in quality of its products and processes

• Key players– Methodology and Standards (M&S) Council– Quality Program Staff

Reference: Landman, et al., (Q2001)

7

Methodology and Standards (M&S) Council

• Steering Committee for the Quality Program• Chiefs of methodology & research divisions

and senior statisticians and methodologists• Ensure sound statistical and methodological

practices

8

Quality Program Staff (QPS)

• Support staff for the M&S Council• Facilitate development and implementation of

standards and guidelines• Evaluate implementation of standards

9

Development Process

10

Original Approach

• Informal process– Team formed based on need– No formal deadlines

• Detailed requirements• Development periods lengthy

– Took from 1 to 4 years

11

New Approach

Structured approach that uses:• Dedicated Staff• Contractor Assistance• Directorate Reviewers• Focus Groups

12

Dedicated Staff (QPS)

Two Mathematical Statisticians and one Survey Statistician

Advantages:• Development times decrease• Cohesion of content improve• Consistency of format improve

13

Contractor Assistance

Advantages:• Provides expertise in conducting focus

groups• Brings outside perspective

Disadvantage: • Lacks familiarity with Census Bureau issues

14

Directorate Reviewers

Focus group participants and program area managers

Advantages:• Allows QPS to obtain directorate input early in

process• Promotes buy-in

Disadvantage:• Risk exists that can lose buy-in

15

Focus GroupsSeven to ten mid-level managers; Two hours in

lengthAdvantages:• Forum for discussing important issues across

directorates• May increase buy-in• Participants more attentive during the review

Disadvantage:• Scheduling difficulties can slow down process

16

Process Steps• QPS sends draft standard to reviewers• QPS compiles comments and produces

discussion issues for focus group• Contractor conducts focus group• QPS revises standard and conducts walk-

throughs with the M&S Council• QPS prepares “final draft” standard and sends to

reviewers along with responses• QPS and quality assurance staff review the

requirements to determine if they are auditable• QPS submits set for final review and approval

17

Framework and Structureof the Standards

18

Framework

• Lists standards and their purpose– Based on the OMB standards – Based on our existing Census Bureau standards

(See our Information Quality website http://www.census.gov/quality/)

• Consists of two types of standards– Process Standards (18)– Supporting Standards (5)

19

Process Standards

Reflect the phases of the life cycle of a data product:

• Planning and Design• Obtaining Data• Processing Data• Analyzing Data• Disseminating Products

20

Supporting Standards

Apply to all phases of the life cycle:• Protecting Confidentiality• Managing Data and Documents• Ensuring Quality• Applying for a Waiver

21

Structure of the Standards

High-level requirements in list form:• State “what” is required; avoid the “how”• Include examples to provide guidance• Provide links to reference documents and to

related standards

22

What Worked Well

23

Accelerated Development

Reduced development times due to:• Early input• Straightforward requirements• Individual walkthroughs with the M&S Council

members

24

Program Area Input

• Interviews with program area experts to get assistance in drafting requirements

• Focus group approach– Structure the discussion based on comments– Overlap participants on related standards– Record the session

25

Focus Group Evaluation

Scale: 1= strongly disagree to 5= strongly agree

Characteristic Rating

The goal of the focus group was clear. 4.2

The participant’s point of view was heard. 4.5

The goal of the focus group was accomplished. 4.0

The length of the focus groups was sufficient. 4.2

The participant’s satisfaction with the outcome. 4.0

26

Next Steps

27

Next Steps

• Review the full set of standards for completeness, consistency, and cohesion

• Submit the standards to the directorates for final review and approval

• Expect completion by September 2009

28

Next Steps

• Implementation of the standards– QPS assists the program directorates– Training

• Evaluations of the implementation– Identify aspects of the standards that need

improvement

29

Challenges

30

Challenges

• Risk of excessive burden– How compliance will affect costs and schedules

• High-level standards– Avoid being too vague– Program areas interpret standards differently

• Verification of compliance– Audit program (Bushery et al., Q2006)

31

Thank you

Contact Information:Pamela D. McGovern4600 Silver Hill RoadU.S. Census Bureau

Washington, DC 20233pamela.d.mcgovern@census.gov