|Policy & Guidance|
|Compliance & Oversight|
|Research Involving Human Subjects|
|Office of Laboratory Animal Welfare (OLAW)|
|Animals in Research|
|Peer Review Policies & Practices|
|Intellectual Property Policy|
|Acknowledging NIH Funding|
|Invention Reporting (iEdison)|
|NIH Public Access|
Frequently Asked Questions
Peer Review Policies & Practices
Initial Posting: September 27, 2011
Last Revised: July 11, 2013
|On This Page:|
A. General Questions
NIH permits one resubmission of an unfunded application (see NOT-OD-09-016).
For all application due dates after April 16, 2014, following an unsuccessful resubmission (A1) application, applicants may submit the same idea as a new (A0) application for the next appropriate new application due date (see NOT-OD-14-074).
Resubmissions (A1) must be submitted within 37 months of the new (A0) application (see NOT-OD-10-140).
For more details on the Resubmission Policy, visit the Resubmissions webpage.
Reviewers are instructed to evaluate the resubmission application as presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project. For resubmitted renewals, the committee will also consider the progress made in the last funding period.
B. Review Criteria and Considerations
For research grants and cooperative agreements, the five scored review criteria are Significance, Investigator(s), Innovation, Approach, and Environment. For a full overview of the scored review criteria, and additional review criteria and considerations for many funding mechanisms, visit the Review Criteria at a Glance document.
If it is the opinion of the reviewers that the project is not likely to answer the questions it poses, then overall impact is likely to be low. The degree of uncertainty about feasibility will likely determine whether this is rated as a minor, moderate or major weakness.
Significance is a stand-alone assessment of the project’s goals in the context of the relevant field, and to a large extent assumes that the investigator(s), approach and environment are adequate to allow for successful completion of the aims of the project even if later discussion of each of these review criteria will identify problems. When reviewers assess the Overall Impact of an application they are expected to take into account the scored review criteria (e.g., significance, investigator(s), innovation, approach and environment) and the additional review criteria to judge the potential of the project to exert a sustained, powerful influence on the field. For more information, visit the Overall Impact versus Significance document.
The Overall Impact score is a synthesis that takes into consideration all of the scored review criteria (i.e., for research applications: significance, investigator(s), innovation, approach and environment) as well as all of the applicable additional review criteria.
Not necessarily. The Overall Impact score considers all scored review criteria as well as all applicable additional review criteria. In addition, an application does not need to be strong in all scored review criteria to be judged likely to have a major scientific impact. Therefore, it is possible for one or more review criteria to overshadow the other review criteria, thus driving the Overall Impact score up or down. Please remember that there is no formula to derive the overall impact score from the individual criterion scores. Reviewers are instructed to weigh the different criteria as appropriate for each application in deriving the Overall Impact score.
One can envision such scenarios. For example, a talented investigator in a very strong environment proposes a highly innovative and very sound approach to address a generally important problem (e.g., breast cancer). However, the proposed project will be relevant to only a narrow area within the larger field of breast cancer research, thus reducing its Significance. Nevertheless, the Overall Impact score could still be strong since the strengths of the project in the other core review criteria give this work the potential to have a sustained, powerful influence on that part of this important field.
Yes. The Overall Impact score synthesizes all scored review criteria as well as all applicable additional review criteria. Thus, while the significance of the project is very strong, the investigator might lack key credentials, the innovation might be minimal, the approach might be problematic, and the environment might not offer adequate support for the project.
Yes. No single review criterion (e.g., Approach) alone determines the Overall Impact score. A project may have numerous minor weaknesses that affect the score for Approach, yet still have a very strong Overall Impact score if the application is exceptionally strong in the other review criteria and the quality of the team and environment lend confidence that the project will have a major overall impact on the field. “Minor weaknesses” are defined as “addressable weaknesses that do not substantially lessen overall impact.”
Not necessarily. The Significance score reflects whether a project addresses an important problem or critical barrier to progress within the field. For example, while a project may generally address a devastating disease with high prevalence, the specific problem addressed in the project may be only tangentially related to the disease, the problem may not be very important for patients with the disease, the proposed work may duplicate already published reports, or the expected results may be unlikely to substantially change knowledge, concepts and/or practice in the field.
No. Work on rare diseases, highly prevalent diseases with modest burden, and highly focused research questions can still be extremely important. Reviewers should judge whether the proposed goals and aims address an important problem or critical barrier to progress in the field, whether the proposed work will improve scientific knowledge, technical capability, and/or clinical practice in the field, or if the project will change the concepts, methods, technologies, treatments, services, or preventative interventions that drive the field.
The mission of the NIH is to support research in pursuit of knowledge about the biology and behavior of living systems and to apply that knowledge to extend healthy life and reduce the burdens of illness and disability. To accomplish this mission, the NIH supports biomedical and behavioral research representing a wide array of research fields as well as tool development, clinical trials and other projects in support of the biomedical research enterprise. In an effort to fairly evaluate scientific and technical merit through the peer review system of a broad range of applications (those that seek cures, not only for diabetes, heart disease, and autism, but also for the lesser recognized orphan diseases and those that ask basic biomedical questions), it is important that Significance and Overall Impact be evaluated within the context of the research field involved. NIH program staff and Institute leadership will evaluate each project’s relevance to their Institute mission in making funding decisions.
FOAs for IC-specific RFAs and targeted trans-NIH initiatives (e.g., Roadmap or Common Fund), including infrastructure and capacity building programs, may include other FOA-specific review criteria in addition to the scored review criteria and the standard NIH additional review criteria (human subjects, animal welfare, renewal, resubmission, biohazards). The Overall Impact score for applications submitted for these initiatives should reflect the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the scored review criteria as well as all additional review criteria (as applicable for the project proposed) and the likelihood that the project will advance the stated goals and objectives of the program as articulated in the FOA.
For fellowship applications (Fs), the overall impact score should reflect the reviewers’ assessment of the likelihood that the fellowship will enhance the candidate’s potential for, and commitment to, a productive independent scientific career in a health-related field, in consideration of the scored criteria (i.e., Fellowship Applicant, Sponsors/ Collaborators/Consultants, Research Training Plan, Training Potential and Institutional Environment & Commitment to Training) as well as all applicable additional review criteria.
For career development award applications (Ks) the overall impact score reflects the reviewers’ assessment of the likelihood for the candidate to maintain a strong research program, in consideration of the scored review criteria (i.e., Candidate, Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring, Research Plan, Mentor(s)/Consultant(s)/Collaborator(s), Environment and Institutional Commitment to the Candidate) as well as all applicable additional review criteria.
For shared instrumentation applications (S10s), the overall impact/benefit score reflects the reviewers’ assessment of the potential benefit of the instrument requested for the overall research community and on NIH-funded research in consideration of the scored review criteria (i.e., Justification of Need, Technical Expertise, Research Projects, Administration, Institutional Commitment) as well as all applicable additional review criteria.
The Overall Impact paragraph provides the reviewer with the opportunity of explaining how the Overall Impact score was derived (i.e., those factors that contributed to the score). If a project has a strong/weak overall impact score then the reviewer should highlight those scored criteria that contributed to the favorable/poor score. For example, if the potential significance of a study was so great as to overshadow a number of methodological weaknesses then this should be clearly stated. Likewise, if the design of the study is so flawed as to negate any potential significance and/or innovation of the study then this should be clearly stated. Importantly, the Overall Impact paragraph should provide a clear view into the reviewer’s thought process that led to his/her Overall Impact score. It is not intended to simply summarize and/or restate the strengths and weakness detailed in the critique.
The current scoring scale for individual reviewers’ scores uses a 9-point scale, with a score of 1 indicating an exceptionally strong application with essentially no weaknesses. A score of 9 indicates an application with serious and substantive weaknesses with very few strengths; 5 is considered an average score. Ratings are in whole numbers only (no decimal ratings). This scale is used by all eligible (without conflict of interest) Scientific Review Group members to provide an overall impact score and for assigned reviewers to score review criteria (e.g., Significance, Investigator(s), Innovation, Approach, and Environment for research applications). For more information, visit the Scoring System and Procedure document.
In some cases, an SRO may wish to bring in a reviewer who is a content expert on a particular part of an application. For example, a statistician might be used as a reviewer for an application that relies heavily on the use of statistics to analyze a biomedical issue. In this case, it would be inappropriate for the statistician to score parts of the application for which he/she does not have expertise.For this reason, reviewers are free to leave sections of the critique template empty of comments if they do not apply for his/her evaluation. In addition, entering criterion scores into eRA systems is not mandatory. However, if a criterion score is entered, a critique must be uploaded.
No. To enter any score, a critique must be entered, with the exception of some mail reviewers and unassigned reviewers who do not need to enter a critique.
D. Reviewer Practices
No, reviewers download a Word file that contains the critique template for the mechanism they are reviewing. Many standard review critique templates are available on the Guidelines and Fill-able Templates for Reviewers page. Once reviewers fill out the template in Word, they upload the entire file into IAR.
The hyperlinks that are associated with each review criterion only work with Microsoft Word 2007 or later versions. These hyperlinks all point to the same Web site, but bring the user to different “anchor” points on the page that correspond to a particular review criterion or consideration. For users with Microsoft 2003, the hyperlinks associated with each review criterion do not work, but an accessible hyperlink is provided at the top of the template that links to the same Web site, so that the same information is available.
No, but discussants can use the critique template for the mechanism they are reviewing and only fill out the “Overall Impact” section of the template.
IAR accepts scores for only 5 criteria. Any additional scored criteria have to be added into the text of the critique and added manually on the summary statement. Please contact your SRO (Scientific Review Officer) for details.
E. Post Submission Materials Policy
The post-submission materials policy states “the only post-submission grant application materials that the NIH will accept are those resulting from unforeseen administrative issues”, and lists the following acceptable post-submission materials:
Yes, the NIH does not view fixing a copying problem to be submitting post-submission material, as long as the master copy that was submitted by the due date was intact and complete.
The only post-submission grant application materials that the NIH will accept are those resulting from unforeseen administrative issues. Post-submission grant application materials are those submitted after submission of the grant application but prior to the initial peer review. This option is to be used when an unexpected event such as the departure of a Senior/Key Person, natural disaster, etc. has occurred, not to correct oversights/errors discovered after submission of the application. For conference or meeting grant applications, acceptable post-submission materials include: