Peer Review Process

Photo of Obama

"To maintain our edge . . . we've got to protect our rigorous peer review system and ensure that we only fund proposals that promise the biggest bang for taxpayer dollars . . . that's what's going to maintain our standards of scientific excellence for years to come."

Remarks by President Barack Obama on the 150th Anniversary of the National Academy of Sciences, April 29, 2013
On This Page:

 


Overview

The core values of peer review drive the NIH to seek the highest level of ethical standards, and form the foundation for the laws, regulations, and policies that govern the NIH peer review process. The NIH dual peer review system is mandated by statute in accordance with section 492 of the Public Health Service Act and federal regulations governing "Scientific Peer Review of Research Grant Applications and Research and Development Contract Projects". NIH policy is intended to promote a process whereby grant applications submitted to the NIH are evaluated on the basis of a process that strives to be fair, equitable, timely, and free of bias.

The first level of review is carried out by a Scientific Review Group (SRG) composed primarily of non-federal scientists who have expertise in relevant scientific disciplines and current research areas.

The second level of review is performed by Institute and Center (IC) National Advisory Councils or Boards. Councils are composed of both scientific and public representatives chosen for their expertise, interest, or activity in matters related to health and disease.

Only applications that are recommended for approval by both the SRG and the Advisory Council may be recommended for funding.  Final funding decisions are made by the IC Directors. 

 
First Level of Review

Initial peer review meetings are administered by either the Center for Scientific Review (CSR) or one of the NIH ICs with funding authority as specified in the funding opportunity announcement (FOA).  

Each FOA specifies all of the review criteria and considerations that will be used in the evaluation of applications submitted for that FOA. Requests for Applications (RFAs) and certain Program Announcements may include additional review criteria and considerations. Other types of funding opportunities (e.g., for construction or fellowship applications) may use different review criteria and considerations (See the Review Criteria at a Glance). Unless the FOA specifies otherwise, standard NIH review procedures will be followed, including the NIH scoring system described in NOT-OD-09-024.

Peer review meetings are announced in the Federal Register. The meetings are closed to the public, although some meetings may have an open session; the Federal Register provides the details of each meeting. 

A.  Peer Review Roles and Meeting Overview

Scientific Review Officer:

Each SRG is led by a Scientific Review Officer (SRO). The SRO is an NIH extramural staff scientist and the designated federal official responsible for ensuring that each application receives an objective and fair initial peer review, and that all applicable laws, regulations, and policies are followed. 

SROs:

  • Analyze the content of each application, and check for completeness.
  • Document and manage conflicts of interest.
  • Recruit qualified reviewers based on scientific and technical qualifications and other considerations, including:
    1. Authority in their scientific field
    2. Dedication to high quality, fair, and objective reviews
    3. Ability to work collegially in a group setting
    4. Experience in research grant review
    5. Balanced representation
  • Assign applications to reviewers for critique preparation and assignment of individual criterion scores.
  • Attend and oversee administrative and regulatory aspects of peer review meetings.
  • Prepare summary statements for all applications reviewed. 

SRG Members

Chair:

  • Serves as moderator of the discussion of scientific and technical merit of the applications under review.
  • Also serves as a peer reviewer for the meeting. 

Reviewers:

Other NIH Staff

  • Federal officials who have need-to-know or pertinent related responsibilities are permitted to attend closed review meetings.
  • NIH Institute/Center staff or other federal staff members wishing to attend an SRG meeting must have advance approval from the responsible SRO. These individuals may provide programmatic or grants management input at the SRO's discretion. 
  Applicants
  • Must maintain the integrity of the peer review process by not contacting reviewers to influence the outcome of the review; not sending information directly to a reviewer; and not accessing information related to the review. There are consequences to any of these actions (See Integrity and Confidentiality in NIH Peer Review).

B. Peer Review Criteria and Considerations

Review Criteria for Research Grants and Cooperative Agreements (for criteria for other types of grants, like training grants, please see Review Criteria at a Glance)

The mission of the NIH is to support science in pursuit of knowledge about the biology and behavior of living systems and to apply that knowledge to extend healthy life and reduce illness and disability. As part of this mission, applications submitted to the NIH for grants or cooperative agreements to support biomedical and behavioral research are evaluated for scientific and technical merit through the NIH peer review system.

Overall Impact: Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following review criteria, and additional review criteria (as applicable for the project proposed).

Scored Review Criteria

  • Significance
  • Investigator(s)
  • Innovation
  • Approach
  • Environment
Additional Review Criteria. As applicable for the project proposed, reviewers will evaluate the following additional items while determining scientific and technical merit and in providing an overall impact score, but will not give separate scores for these items.
  • Protections for Human Subjects
  • Inclusion of Women, Minorities, and Children
  • Vertebrate Animals
  • Biohazards
  • Resubmission
  • Renewal
  • Revision
Additional Review Considerations. As applicable for the project proposed, reviewers will consider each of the following items, but will not give scores for these items and should not consider them in providing an overall impact score.
  • Applications from Foreign Organizations
  • Select Agent
  • Resource Sharing Plans
  • Authentication of Key Biological and/or Chemical Resources
  • Budget and Period Support 

C.  Scoring

The NIH utilizes a 9-point rating scale (1 = exceptional; 9 = poor) for all applications; the same scale is used for overall impact scores and for criterion scores (Scoring Guidance).
Before the SRG meeting, each reviewer assigned to an application gives a separate score for each of (at least) five review criteria (i.e., Significance, Investigator(s), Innovation, Approach, and Environment for research grants and cooperative agreements; see Review Criteria at a Glance). For all applications the individual scores of the assigned reviewers and discussant(s) for these criteria are reported to the applicant.

In addition, each reviewer assigned to an application gives a preliminary overall impact score for that application. In many review meetings, the preliminary scores are used to determine which applications will be discussed in full at the meeting. For each application that is discussed at the meeting, a final impact score is given by each eligible committee member (without conflicts of interest) including the assigned reviewers. Each member's score reflects his/her evaluation of the overall impact that the project is likely to have on the research field(s) involved.

The final overall impact score for each discussed application is determined by calculating the mean score from all the eligible members' final impact scores, and multiplying the average by 10; the final overall impact score is reported on the summary statement. Thus, the final overall impact scores range from 10 (high impact) through 90 (low impact). Numerical impact scores are not reported for applications that are not discussed (ND), which may be reported as ++ on the face page of the summary statement and typically rank in the bottom half of the applications.

Applicants just receiving their scores or summary statements should consult our Next Steps page for detailed guidance. Applicants seeking advice beyond that available online may want to contact the NIH Program Official listed at the top of the summary statement.

An application may be designated Not Recommended for Further Consideration (NRFC) by the SRG if it lacks significant and substantial merit; presents serious ethical problems in the protection of human subjects from research risks; or presents serious ethical problems in the use of vertebrate animals, biohazards, and/or select agents. Applications designated as NRFC do not proceed to the second level of peer review (National Advisory Council/Board) because they cannot be funded. 

D.  Summary Statement

Applications that are not discussed at the meeting will be given the designation "ND" (which may be reported as ++ on the face page of the summary statement) as an overall impact score, but the applicant, as well as NIH staff, will see the written comments and scores from the assigned reviewers and discussants for each of the scored review criteria as feedback on their summary statement. 

Understanding the Percentile

  • A percentile is the approximate percentage of applications that received a better overall impact score from the study section during the past year (see blog on Paylines, Percentiles and Success Rates).
  • For applications reviewed in ad hoc study sections, a different base may be used to calculate percentiles.
  • All percentiles are reported as whole numbers.
  • Only a subset of all applications receive percentiles. The types of applications that are percentiled vary across different NIH Institutes and Centers.
  • The summary statement will identify the base that was used to determine the percentile. 

E.  Appeals

NIH established a peer review appeal system (see NOT-OD-11-064) to provide investigators and applicant organizations the opportunity to seek reconsideration of the initial review results if, after consideration of the summary statement, they believe the review process was flawed for reasons of either bias of a  reviewer, conflict of interest, absence of appropriate expertise, or factual errors by one or more reviewers that could have substantially altered the review outcome.  This policy does not apply to appeals of the technical evaluation of R&D contract projects through the NIH peer review process, appeals of NIH funding decisions, or appeals of decisions concerning extensions of MERIT award.  

 
Second Level of Review - Advisory Council or Board

Who Reviews the Application?

The Advisory Council/Board of the potential awarding Institute/Center performs the second level of review (See Advisory Councils or Boards page). Advisory Councils/Boards are composed of scientists from the extramural research community and public representatives (NIH Federal Advisory Committee Information). Members are chosen by the respective IC and are approved by the Department of Health and Human Services. For certain committees, members are appointed by the President of the United States. 

Recommendation Process

  • NIH program staff members examine applications and consider the overall impact scores given during the peer review process, percentile rankings (if applicable) and the summary statements in light of the Institute/Center's priorities.
  • Program staff provide a grant-funding plan to the Advisory Board/Council.  Council members have access to applications and summary statements pending funding for that IC in that council round.
  • Council members conduct a Special Council Review of grant applications from investigators who currently receive $1 million or more in direct costs of NIH funding to support Research Project Grants (see NOT-OD-12-140). This additional review is to determine if additional funds should be provided to already well-supported investigators and does not represent a cap on NIH funding.
  • The Advisory Council/Board also considers the Institute/Center’s goals and needs and advises the Institute/Center director concerning funding decisions.
  • The Institute/Center director makes final funding decisions based on staff and Advisory Council/Board advice. 
 
Post-Review

Not Funded - Next Steps?

The NIH receives thousands of applications for each application receipt round and competition for funding can be fierce. If the original application is not funded, applicants may resubmit the application, making changes that address reviewer concerns, or they may submit a new application. Once an applicant receives a summary statement, they are directed to information on Next Steps, and they may contact the NIH program official assigned to their application for guidance.   

Fundable Score - Next Steps?

Some of the ICs publish paylines as part of their funding strategies to guide applicants on their likelihood of receiving funding.  Application scores can only be compared against the payline for the fiscal year when the application will be considered for funding, which is not necessarily the year when it was submitted.  There may be a delay of several months to determine paylines at the beginning of fiscal years.   If the application is assigned to an IC that does not announce a payline, the program official listed at the top of the summary statement may be able to provide guidance on the likelihood of funding. After the Advisory Council meeting, if an application results in an award, the applicant will be working closely with the program official of the funding Institute or Center on scientific and programmatic matters and a Grants Management Officer on budgetary or administrative issues. The Grants Management Specialist will contact the applicant to collect information needed to prepare the award. 
   
How to Volunteer to Be a Reviewer
For those interested in volunteering on NIH review panels, please see:

Those interested in becoming a reviewer for a specific NIH IC should browse the individual IC websites for information on contacting SROs at the ICs.

 
More Details

For more details about Peer Review, visit Peer Review Policies & Practices