themes of my talk
Post on 23-Jan-2016
39 Views
Preview:
DESCRIPTION
TRANSCRIPT
The Committee on NationalStatistics (CNSTAT)
36 Years of Care and Feeding of the Federal Statistical System
Presentation to APDU, 9/24/2008
Constance F. CitroDirector, CNSTAT
Themes of My Talk• CNSTAT and the National Academies:
• What, Why, and How?• Value for/Perspective on the Statistical System
• Range of Studies/Workshops/Seminars:• Concepts—Cost of Living, Poverty, Food Insecurity• Reviews of Surveys/Reviews of Agencies/Programs• Reviews of Censuses Past and Present—and the ACS• Cross-Cutting Issues—Confidentiality and Data Access,
Survey Automation• “P&P”—Principles and Practices for a Federal
Statistical Agency• How APDU Can Build on Work of CNSTAT and Others
The National Academies Structure
National Research Council created as operational arm in 1916
National Academy of Sciences given congressional charter in 1863National Academy of Engineering, Institute of Medicine added in 1964, 1970
The National Academies Structure
NRC divided into major operational units for studies/workshops/seminarsCommittee on National Statistics is standing board under the social sciences
Committee on National Statistics (CNSTAT)
Created in 1972 on recommendation of President’s Commission on Federal Statistics: “to provide an outside review of federal statistical activities”
Unique among other Academies units, foreign science academies for its focus on the federal statistical system
Helps to bring together the agencies in a highly decentralized system
Key Features of Academies/ CNSTAT Consensus Panels
Volunteers: Members of study panels serve pro bono (reimbursed only for expenses)
Independence from Sponsor Control: Sponsors do not select panel members and have no access to report until final release
Rigorous External Review: Overseen by Academies’ Report Review Committee; reviewers chosen for expertise and balance
Wide Dissemination of Reports: Sponsors do not control release
Value for Agencies
Breadth and Depth of Expertise: CNSTAT panels engage leading academics and other experts on agencies’ scientific/technical challenges
Cross-cutting Issues: CNSTAT tackles system-wide challenges (e.g., data access and confidentiality in the Internet age)
Credibility: Reports often enable support from Congress/executive for innovation and improvements in agency programs
Downsides for Agencies• Slow:
• Projects, particularly panel studies, can take 1-2+ years• Events may overtake the study
• Pricey: • Volunteers are free, but staff, conference facility, travel, management, etc. are not
• Time-Intensive:• To get most value from study, agency staff must work hard to respond to panel (there is an offsetting benefit to agency from interacting with outside experts)
• Independent:• Two-edged sword
CNSTAT Studies
CNSTAT Studies—Concepts
Observations> Concepts need to keep up with a changing world• Consumer Price Index has had several revisions, in data sources, measurement, and the meaning of major components, such as housing• BEA is moving forward on satellite accounts to capture important components of economic activity that are not easy to include in the main accounts
> So, too, the poverty measure needs to be revised (NYC has led the way; bill being drafted)
NYC Poverty Rates by Borough
2006 ACS Official NYC*Bronx 26.6% 27.9%Brooklyn 21.5 27.0Manhattan 16.8 20.4Queens 11.7 19.6Staten Island 8.4 13.1
New York City 18.9 23.0
*Using NAS method and auxiliary data sources
CNSTAT Survey Reviews
Observations, Suggestions• Major ongoing surveys need regular review, updating, and, sometimes, complete redesign from the ground up• The ACS should be viewed as in a developmental stage at least for the next 10 years; users should be vocal in support and in suggestions for improvement• APDU should press producers for full documentation and funders for resources for “methods panel”-type R&D• APDU dissemination of Federal Register notices re new and revised surveys for comment is great; perhaps could also disseminate advisory committee meeting minutes, other information
CNSTAT Agency/ProgramReviews 1985: Immigration Statistics
1986: NCES
1997: BTS
2007: State and Local Government Statistics (Census)
2007, 2009: BJS
2008: Vital Statistics Workshop
Observations, Suggestions>Agencies sometimes need a major overhaul—
• All too often today, agencies lack the resources to carry out their entire portfolio well, but no one wants to give up long-standing data series• Sometimes agencies have lost—or never sufficiently acquired—a sense of what it means to be a statistical agency
>Change is possible—it takes leadership and inside/outside support; APDU can help
2000 Census Evaluation>Panel to Review the 2000 Census conducted a comprehensive assessment, issuing 9 reports from 1999-2004
>Major problems—wholly imputed households; duplicate enumerations; high long-form sample nonresponse rates; poor quality group quarters data; inadequate evaluations for planning the next census
Bottom Line:>Major successes—halted decline in mail response through advertising, outreach, design of mailing package; hired more than enough enumerators; data capture via OCR
2010 Census PlanningPanel on Research on Future Census Methods reviewed emerging 2010 census plan, including replacing long-form sample with ACS
As early as 2001 letter report—and with particular strength in 2004 final report—panel presaged Bureau’s current problems with handheld computers
Recommended “system architect,” attention to logical architecture
Warned of failure to focus on requirements
American Community Survey2007 Report—Comprehensive Review of the ACS and Differences/Similarities with the Long-Form Sample
Major challenges for users:
• Period estimates (1-, 3-, 5-year)
• Multiple estimates for larger areas
•Larger sampling error than long-form sample
•Inadequacies of population controls
•Collision of ACS/controls/ census in 2010
Current Census/ACS Work
Current panels charged to advise on:
2010 census coverage measurement
2010 CPEX program leading to testing program for 2020
Recent panel examined a custom addition to the American Community Survey—a question on field of study in college, used by NSF
May study group quarters for the ACS and census
Cross-Cutting Issues>Very important function of CNSTAT
>Often have to cobble together funding
>Confidentiality/data access issues always with us—APDU contributed helpful letter; need to be vigilant that the balance doesn’t tip too much against access
CNSTAT White Paper
First edition, 1992 Second edition, 2001 Third edition, 2005 Fourth edition, 2009
(forthcoming)
PRINCIPLES/GOALS>Relevance, Credibility with Users, Trustworthiness with Providers
PRACTICES (selected)>Strong Position of Independence>Openness about Sources and Limitations of Data>Wide Dissemination of Data>Commitment to Quality/Active Research Program>Professional Advancement of Staff
UNDER THREAT TODAY>Not only resources, but also independence, research capability, and adequacy of technical staff pipeline
2005 P&P
Cooperation/CoordinationStatistical Agencies
OMB Statistical and Science Policy OfficeAPDU – CNSTAT – COPAFS – COSSA et al.
Statistical AgenciesOMB Statistical and Science Policy Office
APDU – CNSTAT – COPAFS – COSSA et al.
• The agencies need more help than ever• Cooperation and coordination are essential to make the best use of limited agency resources and to leverage the work of outside groups• As a long-time APDU member, I’m thrilled to see it revitalized so that it can play its part in ensuring a vital federal statistical system for the future.
Questions?/Feedback
http://www.nationalacademies.org/cnstat
Reports: http://www.nap.edu
Connie Citro: ccitro@nas.edu; (202) 334-3009
top related