1 ci administrator’s meeting october 26-27, 2005 ccny, new york city formal reviews (wed)...

1

CI Administrator’s MeetingOctober 26-27, 2005

CCNY, New York City

• Formal Reviews (Wed)– Transition to new review policy– NESDIS timelines and guidelines– NESDIS example of technical questions

• New Policies for the NOAA CIs (Thurs)

2

Transition to New Review Policy

• NOAA committed to uniform administration of all CIs• Fully competitive• 5 year agreements, with one 5 year extension• Re-competition after second 5 years• Science/admin review is part of new policy

– More uniform schedule and oversight– Review under the auspices of NOAA SAB, but NESDIS will

still administer the view

• CICS review Nov 2005 under old policy• CIOSS review will follow new review rules when

possible

3

.Time Before End of First Five-Year Agreement

Task

36 months prior LO coordinates with CI to schedule the review.

32 months prior LO coordinates with CI to obtain suggested science reviewers, checks with reviewers for availability.

30 months prior LO AA submits formal review request to the SAB.

29 months prior LO coordinates with the NOAA SAB to obtain approval of science reviewers.

29 months prior LO selects administrative reviewers.

24 months prior CI science and administrative review.

22 months prior Review committees submit draft reports to LO.

22 months prior LO submits draft reports to CI to check for accuracy .

21 months prior LO submits draft reports back to reviewers for final approval.

20 months prior LO submits final report to SAB and schedules presentation by the review chair. CI director and LO representatives should attend the presentation.

16 months prior SAB presentation by review chair. SAB transmits report to NOAA LO AA along with any recommendations.

15 months prior LO makes recommendation for renewal, conditional renewal or termination to the RC through the NOAA CI committee based upon SAB response. RC determines renewal conditions.

14 months prior LO communicates renewal recommendation to CI.

13 months prior LO sends response to the review to the SAB.

12 months prior LO begins work with CI and GMD to process the renewal if appropriate.

General Timeline for New NOAA Review Policy

4

Time Before the Review

Task

12 months prior LO identifies NOAA review coordinator and coordinates with the CI to schedule the review.

8 months prior LO coordinates with CI to obtain suggested reviewers, checks with reviewers for availability.

7 months prior LO identifies administrative reviewers, including representatives from the GMD.

6 months prior LO coordinates with the NOAA SAB for approval of science reviewers.

4 months prior SAB sends formal invitation letter sent to science and administrative reviewers.

3 months prior Roles and responsibilities sent to the reviewers. This should include the following: 1) A brief summary of the NOAA review process; 2) CVs of the review team; 2) The expected time commitment of the reviewers; 3) What to look for (Why are they there?). 4)

3 months prior LO coordinates with CI to identify review attendees, including the CI’s University/Institution’s Office of Sponsored Research, the CI Administrative Staff/Representatives, and others at the invitation of the Director of the CI.

3 months prior CI begins preparation of a briefing book organized around the review guidelines and science and administrative review questions. Three ring notebooks are suggested.

2 months prior LO and CI finalize review agenda.

2 months prior LO begins preparation of travel orders for LO staff and review teams

2 to 3 weeks prior

CI completes briefing book and sends copies to LO review coordinator. The LO will acknowledge receipt of the book and distribute copies to the review teams.

Science and Administrative reviews

Detailed Pre-Review Timeline

5

Detailed Post-Review Timeline

Time After the Review

Task

Science and Administrative reviews

2 months after Science and Technical review teams complete preliminary review reports, including three tier overall rating (Outstanding, Satisfactory, Unsatisfactory)

2 months after LO submits preliminary reports to CI to check for accuracy

3 months after LO submits preliminary reports with possible modifications due to CI input back to reviewers for final approval

4 months after LO submits final report to SAB and schedules presentation by the review chair at the next SAB meeting. CI director and LO representatives should attend the presentation.

8 months after SAB presentation by review chair.

9 months after LO makes recommendation for renewal, conditional renewal or termination to the RC through the NOAA CI committee based upon SAB response.

10 months after LO communicates renewal recommendation to CI

11 months after LO sends response to the review to the SAB

12 months after LO begins work with CI and GMD to process the renewal if appropriate

6

Technical Review Criteria Under the New CI Policy

(Details still being formulated by the NOAA SAB)

1) Quality, creativity, integrity and credibility2) Timeliness, scale and scope3) Science connected to the application and operational implementation of policy4) Capacity-building5) Education6) Efficiency7) Social science integration8) Diversity

7

Sample Technical Review Questions

1. Science PlanA. What is the scientific (not programmatic) vision for the Institute?B. How is it related to the NOAA Strategic Plan?C. What are the goals and objectives and how well do the CI goals and specific scientific efforts align with NOAA’s Five-Year Research Plan and the NOAA 20-Year Research Vision?D. What criteria are used internally to measure progress in accomplishing these goals and objectives?E. What are the major scientific themes?1. How were they identified?

2. How have they evolved over the past two decades?3. Which themes/sub themes are near completion?4. What are the emerging thematic areas? Why? 5. Are any changes in program emphasis being considered for the next cycle?6. What projects would you stop or start given:Level funding without inflation increases?10% program increase or decrease?

8


2. Scientific Partnerships A. What is your relationship to the NESDIS Laboratories and other NOAA entities?

B. What, if any, formal procedures do you have for joint planning?

3. Science Review

A. What are the Institute’s most recent scientific highlights and accomplishments? (Note that this is an opportunity for early-mid career scientists to become acquainted to/by upper NOAA management.).B. What are the most important new or emerging research areas of interest to NOAA? C. What is the extent of actual collaborations between the CI and NOAA, explicitly highlighting collaborations with NESDIS?

4. Technology TransferA. How are research results from the Institute shared with NOAA as well as the broader

scientific community? B. How are NOAA’s research interests conveyed to the Cooperative Institute?C. What has been the most important research or development result that has been transferred

to customers during the last four years?D. What is planned for the next 1-2 years?

9


5. Education/OutreachA. What types of educational activities/opportunities (K-12, undergraduate and graduate students) does the Institute offer on an ongoing basis?B. What are the current and planned outreach efforts?

6. Science Management PlanA. How does the Institute identify new intellectual opportunities?B. What are some recent examples of intellectual opportunities?C. What is the strategy for new starts (projects, techniques, campaigns, etc.)D. What review mechanisms are in place to make program adjustments?

E. How much of the Institute resources are reserved for new opportunities or bright ideas?

F. What is the internal review process for science proposals?G. Do you have any quantitative performance measures for tracking your research

program?

10

New Policies for the NOAA Cooperative Institutes

• 2003 NOAA SAB Research Summary– NOAA should establish common procedural structure for CI

administration

• All future CIs should be competitively established• 5 year agreement with one 5 year renewal• Comprehensive science review before the end of first

5 years• Open competition at the end of 10 years

– Existing CIs can compete

• Transition plan for all non-competitively selected CIs

11

Roles and Responsibilities

a. NOAA RC - Responsible for reviewing recommendations from LOs or the GTs for establishing CIs, designating the LO that is responsible for maintaining the CI, approving the review guidelines for the renewals, and overseeing the CI program.

b. NOAA CI Committee - Establishing and reviewing all procedures pertaining to NOAA CIs and the implementation of the CI policy. The CI Committee will provide aggregate financial and performance information on the NOAA CIs upon request of the RC and/or any NOAA office. Includes representatives from NESDIS, OAR, NMFS, NWS, NOS, NOAA Grants Office,

c. NOAA LO - Administering the CI award, including oversight of the initial CI competition, evaluating CI performance, funding to the CI throughout the award period, managing the renewal review process, arranging re-competition, and sunsetting (if necessary)

d. NOAA Goal Teams and Programs - Goal Team(s), and their relevant Program(s) propose the formation of CIs with the relevant LOs.

e. CI Director - The CI director is responsible for oversight of all NOAA-funded activities within the CI and the submission of any required reports associated with the CI award.

f. NOAA SAB – Is the official reviewing authority, approves science reviewers, and makes recommendation(s) after the review.

12

Four NOAA Documents Describe new CI Policy

• NAO – describes new policy (Completed)– All CIs will be selected competitively– 5 years with a renewal for 5 more years

• Transition Plan – Provides re-competition dates

• CIRA, CICS – June 30, 2009• CIMSS - June 30, 2010• CIOSS - June 30, 2013• Note: 10 others from OAR/NWS/NMFS before NESDIS CIs

• Transition Guidance Memo – Guidelines for renewal during transition period

• CI Handbook (Dec 05)– Provides details of establishment, management, review, renewal, and sunset of CIs– Being drafted by NOAA CI committee

13

New NOAA CI Policy Implementation Timeline

09/02/05 -09/12/05 LO/GT/GMD/GC Review of CI Transition Plan Completed

09/02/05 Submit CI Committee Charter to RC for Approval Completed

09/14/05Submit Transition Plan (and Continuation Schedule) to

Research Council and GMD for ApprovalCompleted

10/13/05Submit Transition Plan to NOAA Administrator for

ApprovalCompleted

10/14-25/05 LO/GT/GMD/GC Review of Draft CI Handbook In progress

10/17/05Send NAO and Transition Plan to SAB *per NOAA's

Research Review response

10/31/05 – 11/7/05 Research Council Reviews Draft CI Interim Handbook

11/21/05 – 1/20/06 Handbook Released for 60-day Public Comment Period

01/30/06Submit Final Draft of CI Interim Handbook to RC for

Approval

02/15/06 RC Approval of Handbook

14

New CI Procedures - 1

• CI committee will play role in developing and modifying CI policy

• Establishment of new CIs or re-competing old ones managed by LO, Goal Teams and RCs

• Uniform evaluation criteria for new CIs– Capability to enhance NOAA research– Scientific expertise and recognized graduate program– Research plan aligned with NOAA research goals– “Business Plan” for fiscal and human resource

management, strategic planning– Demonstrated record of performance on NOAA projects (if

applicable)– Cost sharing

15


• Performance measures– Mutually agreed upon by NOAA and CI

• Uniform funding terminology– Task 1 – admin, outreach, student support– Task 2 - Research with strong NOAA

collaboration– Task 3 – Research without strong NOAA

collaboration

• Uniform annual reporting– Annual guidance memorandum, similar to CIRA

procedure

• Annual meeting of all NOAA CIs (~March)

16


• Three-tier Review Rating System– a. Outstanding –The CI has demonstrated consistent achievement of all initially agreed goals

and outstanding performance, as well as evidence of a continuous resource commitment that enhances NOAA’s resources to support collaborative research. For outstanding performance, NOAA will renew a CI for up to an additional five years at a funding level commensurate with their level of performance, pending availability of funding.

– b. Satisfactory – The CI has reasonably achieved some or all of its agreed goals and has demonstrated acceptable performance. Its performance, however, is not considered outstanding and/or the CI’s resource commitment provides a limited enhancement of NOAA’s resources. For acceptable performance, NOAA may opt to renew a CI for a period less than 5 years that may be at a significantly reduced funding level, pending availability of funding.

– c. Unsatisfactory – The CI has demonstrated a failure to achieve some or all of its agreed goals and its performance is unacceptable and/or the CI has also provided minimal resources to enhance NOAA’s resources to conduct collaborative research. For unacceptable performance, NOAA will not renew the award or, for serious problems, will terminate the CI award according to the procedure identified in Chapter 7.

17


• All CIs sunsetted after 10 years performance measures

• Re-competition if NOAA need still exists after 10 years

• Timeline for new or re-competition requires 18 months– Re-competition process would start 2 years before end of 2nd

5 years

• Sunsetting will be a formality if existing university wins the re-competition

• Early termination (before 10 years) possible for unsatisfactory performance– Would usually occur following science review