What Happens to your NIH Grant After You Hit the Send Button

Arrival at the NIH

• All grants arrive in one central office• Each grant is assigned a grant number, to an

Institute, and to a Study Section for review• In several weeks, you will be able to access

this information on your Commons site

The NIH Institutes (Branches)

• Each has an intramural program (investigators who work at and receive direct support from the NIH) and extramural program (all of the grantees in the scientific community outside of the NIH)

• Each branch has areas of research interest which influence funding decisions, dictate “RFPs or requests for proposals” (announcements about specific funding opportunities for which money has been set aside)– Watch for RFPs to see if your work fits

• Each branch has its own “pay line”, a score below which a grant will be funded (low scores are good!!)– Think about targeting your grant to a specific Institute

Your Grant Number

• The type of grant – For example R01, K08, P01 and if it is new or has been around a while

• For example, 1R01 versus 2R01

• The institute to which it has been assigned– For example, CA for Cancer, GM for General Medicine– May have two listed, CA/HL, Cancer and Heart Lung

• The grant number • The year of the grant

– A new grant will be -01, a continuing grant will be -04, 05, 06 etc• If the grant is a resubmission (previously reviewed, but not

funded) it is designated an A1– If an “A1” is not funded, you cannot resubmit the grant

The Study Section

• Based on key words in the title and abstract, your grant will be assigned to an Institute and a Study Section – A group of your peers, mostly from academic institutions but others as well

(investigators from biotech, pharma, etc), who are charged with reviewing grant applications in a non-biased manner focusing solely on scientific merit

• The study section is chaired by a senior investigator with prior review experience

• Study section members meet 3 times per year. Each permanent member is expected to serve a four year term

• Each meeting will include ad hoc members (noted by an asterisk on the roster list)

• Some study sections now meet in silico

Your SRA (Scientific Review Administrator)

• An NIH employee (works for the Center for Scientific Review) who coordinates efforts of the Study Section. The SRA is responsible for making sure that all grants are reviewed fairly

• The name of your SRA and his/her contact information are listed on the grant assignment notification

• The SRA is neutral and will not tell you about details of the review

• You can contact this person for questions about your submission BEFORE the review (for example, is supplemental material allowed, if so, when, etc), but not about the review itself

Your Reviewers

• The study section chair and SRA will assign each grant to three reviewers: a primary, secondary, and reader

• Each reviewer is required to read the grant thoroughly and prepare a critique

• Both the primary and secondary reviewers prepare written critiques, which the applicant will eventually see. Occasionally the “reader” will also prepare a written evaluation, which the applicant will also see

• The review is based on the original grant as well as supplementary material (usually up to three pages), which can be submitted several weeks before the review

The Review Process

• Prior to the study section, reviewers will evaluate the grants, prepare reviews which are submitted electronically, and submit a preliminary score

• Grants are scored from 1 to 9, with one being the best• Rarely, a grant will be determined not appropriate for further consideration.

This decision is made only if there is a major problem making the reviewer feel that the grant is inappropriate

• Reviewers are also asked to determine if a particular grant would likely fall into the bottom half of the scores. Those grants may be “triaged” at the study section

• Once reviews are submitted, the other reviewers’ comments can be viewed by members of the Study Section

The Study Section Meeting (I)

• The study section meets every four months, usually in February, June, and October. Grants are reviewed that were submitted about five months previously

• The meeting begins with introductions and the SRA describing the rules about the discussion to be held, voting processes, etc

• The SRA will have assembled a list of grants for which one, two, or three of the reviewers recommended “triage”. A brief discussion is held about whether or not to triage these grants. If all reviewers agree, the remainder of the study section is asked if any of the proposed triaged grants should be discussed and scored. A triage list is then generated

• Triaged grants are not discussed. The applicants will receive unedited comments from the reviewers.

The Study Section Meeting (II)

• Your Program Officer– The non-triaged grants are then discussed in order based on score. Institute

representatives (program officers) are present at the study section as observers, and they listen to the discussion of the grants assigned to their institute. Every grant should have a program officer present for the discussion

– Unlike the SRA who is neutral, the Program Officer is on your side. They will listen to the discussion and take notes. They generally are silent at the Study Section but can give you feedback and advice once the discussion is over

– You should talk to and make friends with your Program Officer. They can be very helpful. Call after the Study Section has met

The Study Section Meeting (III)The Review Criteria

• All reviews are prepared based on five criteria. Each criterion impacts your grant’s score, although some have greater influence than others

• Each grant will be given an overall priority/impact score. A score between 1 and 9 will be also given for each of the following five criterion. The overall priority score is not an average but the reviewer’s gestalt

• The NIH’s description of these criteria are posted:http://grants.nih.gov/grants/peer/critiques/rpg.htm#rpg_01

The Study Section Meeting (IV)The Review Criteria

– Significance

• Is the project important, or “Who Cares?”

• If significance is high, it is a plus. If no one cares whether or not your project is done, it is a minus. For most grants, significance is considered high, but not amazing and has little impact on the final score.

The Study Section Meeting (V)The Review Criteria

– Investigator• Are you well trained, experienced, and able to do the work

described?• If you are a senior investigator, have you been productive?• If you are a new investigator, especially if you are staying

at the same institution where you trained, are you independent? Do you have the experience and training for the experiments that you have proposed?

– Independence requires an institutional commitment, talk to your chair!

The Study Section Meeting (VI)The Review Criteria

– Innovation• Are the approaches standard (perhaps outdated) or new

and clever?– You are not hurt by using standard, state-of-the-art

techniques if they are appropriate to the question, but are hurt if you are using old approaches that have been proven to be inferior to newer techniques

– Generally, this aspect does little to influence your score, unless you are proposing a clever new technology which will help your score or if your methods are clearly inappropriate for the studies which will hurt your score

The Study Section Meeting (VII)The Review Criteria

– Approach (What is the experimental plan?)• Even with the new scoring, this is most critical aspect of the review

– Is there a clear hypothesis? – Is the approach flawed or reasonable? – Are alternatives presented?– How will the results be interpreted?– Are the best available scientific approaches being employed?– Is the grant too ambitious (almost always a concern for junior

investigators)? More rarely, does the grant not go far enough?• All of these questions must be clearly answered in the proposal

The Study Section Meeting (VIII)The Review Criteria

– Environment• Does your institution have the resources for you to do the

work (core facilities, lab space, etc)?• Do you have access to these resources?• Are there others around who can help?• Generally, environment is not an issue for grants from

Penn

The Study Section Meeting (IX)Discussing the Grants

• The non-triaged grants are discussed in order• Each of the reviewers (primary, secondary, and reader) provide

a score• The primary reviewer then describes the grant and discusses

his/her assessment based on each of the review criteria– Other study section members may ask questions or give their

perspectives• The secondary reviewer then discusses how he/she agrees or

disagrees with the primary reviewer, generally a much shorter discussion

• The reader does the same as the secondary reviewer

The Study Section Meeting (X)Discussing the Grants (cont)

• After all of the assigned reviewers present their views, the grant is open for discussion by the entire Study Section

• When the discussion comes to a natural conclusion, each of the reviewers provides a new score based on what has been discussed– Scores may change or may remain the same. Often, the range of

scores is small (ie there is considerable agreement); however, the range may be quite high even after a very long discussion

• All members of the study section then record their votes confidentially

The Study Section Meeting (XI)Discussing the Grants

• After everyone votes, the reviewers are then asked about the budget and length of time for support– They may recommend to fund as requested or to decrease (never increase) the

budget – A justification must be provided to decrease the budget or the requested length of

time– The study section then votes by a show of hands on the “time and amount”

• The reviewers then indicate if there are concerns about animals or human subjects. If so, an administrative note is made

• Neither budget nor animal/human subjects discussion influences the score of the grant

The Study Section Meeting (XII)

• Once all of the grants have been discussed, everyone rushes out to National Airport to get the next plane home

• The process is exhausting and time consuming. Most reviewers are responsible for 8 to 12 grants. The meeting itself last 1-2 days, usually with working lunch

• Our experience is that everyone tries hard to be fair

Scoring the Grants

• After the meeting, all of the scores are tabulated. They are then compared to the previous two rounds of that study section and normalized (the assumption being that the scores should average similarly from meeting to meeting, but that reviewers may not be completely consistent)

• Scores are then converted to percentiles with the lowest being the best (what percentage of grants scored better than you). The percentile matters, not your absolute score

• Funding decisions are made several months later by the institutes (at the meeting of their councils), but you can get a good sense based on your percentile

• Funding varies from institute to institute

When are the Results Available?

• Your score will be available within a few days of the study section meeting– Can be accessed via the web (the Commons website)– You can call your Program Officer

• Regardless of the score, it is worth a call to your Program Officer to find out what they heard at the study section– If things went well, you become a voice, not just a grant– If things did not go well, you can get a sense of what to do

next

When are the Results Available?The Pink Sheets

• Four to six weeks after the review, the critique will be available on the Commons Website

• If the grant was discussed (not triaged), the SRA will have provided a summary of the discussion indicating the major points (positive and negative) brought up at the study section

• The written reviews (largely unedited) follow• The roster of the study section is also provided

• Read the pink sheets carefully!

Getting Funded

• The institute councils meet one to two months after the study sections. It is then that final funding decisions are made

• These are generally based strictly on percentiles, however there is some room for discretion– Program officers can request that an occasional grant be

funded out of order (“select pay”) based on programmatic issues or other unusual circumstances

– It is worth knowing your Program Officer and for them to know how important and topical your work is

What if You Do Not Get the Money?

• Try again– Get advice from the Program Officer– Carefully read the pink sheets– Discuss the grant and reviews with senior colleagues with expertise in

the subject area of your grant • Before resubmitting the grant, make sure you can address the

major critiques– The next reviewers will see the pink sheets from the previous

submission– The most important things to address will be in the summary paragraph

written by the SRA– If you cannot do what they want, have a good reason why you need not– In the resubmission, answer all of the major concerns (both in your

three page introduction and the body of the grant)