Performance Assessment

Assessment of Organizational Excellence

NSF Advisory Committee for

Business and OperationsMay 5-6, 2005

Performance Assessment

Develop and use performance assessment tools and measures to provide an environment of continuous improvement in NSF’s intellectual investments as well as its management effectiveness.

(one of four areas—ALL now need to be successful)

Expert Assessments Integrated Expert Assessments Integrated Throughout NSFThroughout NSF

NSF MissionNSF Mission

STRATEGIC

OPERATIONAL

INDIVIDUAL

OUTCOMES

OUTPUT

ACTIVITY

• Strategic or Long-Term Planning

• Scientific Advisory Committee Reviews

• NSF Performance Planning

• Advisory Committee for GPRA Performance Assessment (AC/GPA)

• Business & Operations Advisory Committee

• Committees of Visitors (COVs)

• Merit Review

• Project Reports

• Program Assessment Ratings Tool (PART)

• Individual Performance Assessments Directly Linked to Mission and Goals

Components

In red, added 2001 or later

Performance Assessment:FY 2005 Focus Areas• PART Activities

– New Efficiency Metric– Highest Ratings in All Programs

• AC/GPA Activities– Refined processes– Categorizing “Innovative High-Risk”

PART Activities I

Achievements

Reached consensus with OMB and implemented change.

Achievements

Reached consensus with OMB and implemented change.

Results

Results

Factors

New Efficiency Measure: Percent of award decisions made available to applicants within six months of proposal receipt or deadline date, while maintaining a credible and efficient competitive merit review system, as evaluated by external experts.

Factors

New Efficiency Measure: Percent of award decisions made available to applicants within six months of proposal receipt or deadline date, while maintaining a credible and efficient competitive merit review system, as evaluated by external experts.

PART Activities II

Achievements

100% of NSF PARTs (8 out of 8 in total) received the highest rating of “Effective” as compared with only 15% of 607 PARTs government-wide.

Achievements

100% of NSF PARTs (8 out of 8 in total) received the highest rating of “Effective” as compared with only 15% of 607 PARTs government-wide.

Results

Results

Factors

Completed PARTs in three NSF Investment Categories and one Priority Area:- Institutions (People)- Collaborations (People)- Polar Tools, Logistics, and Facilities (Tools)- Biocomplexity in the Environment (Priority Area)

Factors

Completed PARTs in three NSF Investment Categories and one Priority Area:- Institutions (People)- Collaborations (People)- Polar Tools, Logistics, and Facilities (Tools)- Biocomplexity in the Environment (Priority Area)

AC/GPA I

Achievements

- More substantive discussions at AC/GPA meeting

-Positive comments from AC/GPA

Achievements

- More substantive discussions at AC/GPA meeting

-Positive comments from AC/GPA

Results

Results

Factors

Work on individual indicators performed through subgroup chairs prior to annual meeting.

Factors

Work on individual indicators performed through subgroup chairs prior to annual meeting.

AC/GPA II

Achievements

- Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories

-Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses

Achievements

- Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories

-Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses

Results

Results

Factors

Identifying “high-risk” and “multidisciplinary” activities

Factors

Identifying “high-risk” and “multidisciplinary” activities

A STARTING POINT FOR THE AC/GPA DISCUSSION ON INNOVATIVE-HIGH RISK RESEARCH – FROM PROGRAM OFFICER COMMENTS IN NUGGETS

“Innovative High-Risk”

Forefront, Novel or Transformative but Untried or Untested

High Reward but Significant Technical Challenges and/or High Probability of Failure

Innovative and Contrary to Current Theory or Conventional Paradigms

“Other High Risk” (few in number)

Risk from Not Succeeding

Risk to Principal Investigators/Others

Accomplishments Indicators of SuccessWhy a Priority

Coming Attractions/Comments

PART Activities

Priority Initiatives

Establishes agency-wide evaluation framework.

4 PARTs completed: - Institutions, - Collaborations, - Polar Tools, and - Biocomplexity

All received the highest rating: “effective”

2 PARTs underway:- Fundamental Science and Engineering- FFRDCs

AC/GPA Recommendation

NAPA Recommendation

Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories

-Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses

Definitions and categories for “innovative high risk”

Performance Assessment: FY 2005

Refining AC/GPA Process

Identify Appropriate Efficiency Measure

Replacement in PART

Use Measure that Combines Quality and Timeliness

Reached Consensus with OMB Effective this year

Identifying “high-risk” and “multidisciplinary” activities

Workload issues for AC/GPA at annual meeting

New process well-received

-More substantive discussions at AC/GPA meeting

-Comments from AC/GPA

Creating of separate OE subgroup

Comments

2. “Innovative High Risk Research”

Future Initiatives

-Complete Fundamental Science and Engineering PART and Federally Funded Research and Development Center PART

-Initiate five remaining PARTs for next year

1. PART

-Work with AC/GPA to define categories for further use (e.g. by COVs, POs in writing accomplishments (“nuggets”), input to the NSB Task Force on Transformative Research)

Initiatives on the Horizon

Conclusion

NSF has demonstrated significant achievement in Performance Assessment.