Enhancing Peer Review: The NIH Announces Enhanced Review Criteria for Evaluation of Research Applications Received for Potential FY2010 Funding

Notice Number: NOT-OD-09-025

Update: The following update relating to this announcement has been issued:

  • March 12, 2010 - See Notice NOT-NS-10-013 NINDS Announces Availability of Funds for Competitive Revision Applications for Grand Challenge on Optimal Cortical Control of DARPA Revolutionizing Upper-Limb Prosthetics (R01 and R37).
  • September 16, 2009 - See NOT-OD-149 for information on Restructured Application Forms and Instructions for Submissions for FY2011 Funding .
  • March 4, 2009 - See Notice (NOT-OD-09-054) Recovery Act of 2009: NIH Review Criteria, Scoring System, and Suspension of Appeals Process.

Key Dates
Release Date: December 2, 2008

Issued by
National Institutes of Health (NIH), (http://www.nih.gov)

Background

In June 2007, the NIH initiated a formal, agency-wide effort to review the NIH peer review system (http://enhancing-peer-review.nih.gov/). After careful deliberation and consideration of the recommendations resulting from this year-long effort, a number of key actions will be implemented in the NIH peer review system. These actions include the implementation of enhanced review criteria for evaluating the scientific and technical merit of applications submitted to the NIH for grants or cooperative agreements to support biomedical or behavioral research.

Additional information is available in Guide Notices NOT-OD-09-023 Enhancing Peer Review: The NIH Announces Updated Implementation Timeline and NOT-OD-09-024 Enhancing Peer Review: The NIH Announces New Scoring Procedures for Evaluation of Research Applications Received for Potential FY2010 Funding .

The enhanced criteria will replace the review criteria adopted October 12, 2004 (see http://grants.nih.gov/grants/guide/notice-files/NOT-OD-05-002.html) and modified May 11, 2006 (see http://grants.nih.gov/grants/guide/notice-files/NOT-OD-06-069.html). A side-by-side comparison of the enhanced review criteria described below, and the criteria that will be replaced, is available on the OER website (http://grants.nih.gov/grants/peer/guidelines_general/comparison_of_review_criteria.pdf).

Implementation

The enhanced review criteria (below) will be effective for all applications for research grants and cooperative agreements that are submitted for funding consideration for fiscal year 2010 (FY2010) and thereafter. The first standing due date for FY2010 is January 25, 2009; the enhanced criteria will be used for applications submitted in response to Parent Announcements and Program Announcements, including PARs and PASs published before or after this Guide Notice. An important aspect of the implementation of the enhanced criteria is to use them in a consistent manner for applications considered in a given fiscal year. Therefore, some RFAs and PARs for funding consideration in FY2010 have due dates before January 25, 2009 and responses to these will be evaluated using the enhanced criteria. Likewise some RFAs and PARs for FY2009 have due dates after January 25, 2009 and responses to those will be evaluated using the present criteria. RFAs and some PARs may include additional review criteria and considerations that are related to specific requirements of the RFA or PAR.

These enhanced criteria may not be applicable for some other types of applications (e.g., construction grants, fellowship applications). Criteria for these other programs will be described in the Funding Opportunity Announcements (FOAs).

Enhanced Review Criteria

The mission of the NIH is to support science in pursuit of knowledge about the biology and behavior of living systems and to apply that knowledge to extend healthy life and reduce the burdens of illness and disability. As part of this mission, applications submitted to the NIH for grants or cooperative agreements to support biomedical and behavioral research are evaluated for scientific and technical merit through the NIH peer review system.

Overall Impact.Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following five core review criteria, and additional review criteria (as applicable for the project proposed).

Core Review Criteria. Reviewers will consider each of the five review criteria below in the determination of scientific and technical merit, and give a separate score for each. An application does not need to be strong in all categories to be judged likely to have major scientific impact. For example, a project that by its nature is not innovative may be essential to advance a field.

Significance. Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

Investigator(s). Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?

Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?
If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?

Environment. Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?

Additional Review Criteria. As applicable for the project proposed, reviewers will consider the following additional items in the determination of scientific and technical merit, but will not give separate scores for these items.

Protections for Human Subjects. For research that involves human subjects but does not involve one of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate the justification for involvement of human subjects and the proposed protections from research risk relating to their participation according to the following five review criteria: 1) risk to subjects, 2) adequacy of protection against risks, 3) potential benefits to the subjects and others, 4) importance of the knowledge to be gained, and 5) data and safety monitoring for clinical trials.

For research that involves human subjects and meets the criteria for one or more of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate: 1) the justification for the exemption, 2) human subjects involvement and characteristics, and 3) sources of materials.

Inclusion of Women, Minorities, and Children. When the proposed project involves clinical research, the committee will evaluate the proposed plans for inclusion of minorities and members of both genders, as well as the inclusion of children.

Vertebrate Animals. The committee will evaluate the involvement of live vertebrate animals as part of the scientific assessment according to the following five points: 1) proposed use of the animals, and species, strains, ages, sex, and numbers to be used; 2) justifications for the use of animals and for the appropriateness of the species and numbers proposed; 3) adequacy of veterinary care; 4) procedures for limiting discomfort, distress, pain and injury to that which is unavoidable in the conduct of scientifically sound research including the use of analgesic, anesthetic, and tranquilizing drugs and/or comfortable restraining devices; and 5) methods of euthanasia and reason for selection if not consistent with the AVMA Guidelines on Euthanasia.

Resubmission Applications. When reviewing a Resubmission application (formerly called an amended application), the committee will evaluate the application as now presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project.

Renewal Applications. When reviewing a Renewal application (formerly called a competing continuation application), the committee will consider the progress made in the last funding period.

Revision Applications. When reviewing a Revision application (formerly called a competing supplement application), the committee will consider the appropriateness of the proposed expansion of the scope of the project. If the Revision application relates to a specific line of investigation presented in the original application that was not recommended for approval by the committee, then the committee will consider whether the responses to comments from the previous scientific review group are adequate and whether substantial changes are clearly evident.

Biohazards. Reviewers will assess whether materials or procedures proposed are potentially hazardous to research personnel and/or the environment, and if needed, determine whether adequate protection is proposed.

Additional Review Considerations. As applicable for the project proposed, reviewers will address each of the following items, but will not give scores for these items and should not consider them in providing an overall impact score.

Budget and Period Support. Reviewers will consider whether the budget and the requested period of support are fully justified and reasonable in relation to the proposed research.

Select Agent Research. Reviewers will assess the information provided in this section of the application, including 1) the Select Agent(s) to be used in the proposed research, 2) the registration status of all entities where Select Agent(s) will be used, 3) the procedures that will be used to monitor possession use and transfer of Select Agent(s), and 4) plans for appropriate biosafety, biocontainment, and security of the Select Agent(s).

Applications from Foreign Organizations. Reviewers will assess whether the project presents special opportunities for furthering research programs through the use of unusual talent, resources, populations, or environmental conditions that exist in other countries and either are not readily available in the United States or augment existing U.S. resources.

Resource Sharing Plans. Reviewers will comment on whether the following Resource Sharing Plans, or the rationale for not sharing the following types of resources, are reasonable: 1) Data Sharing Plan (http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm); 2) Sharing Model Organisms (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-04-042.html); and 3) Genome Wide Association Studies (GWAS) (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-07-088.html).

Inquiries

Questions should be directed to:

Sally A. Amero, Ph.D.
NIH Review Policy Officer
OD/OER/ODP
National Institutes of Health
6705 Rockledge Drive, Room 3520
Bethesda, MD 20892
Telephone: (301) 435-1418
Email: ameros@od.nih.gov

Or send an email to ReviewPolicyOfficer@mail.nih.gov.

For more information on NIH’s Enhancing Peer Review effort visit http://enhancing-peer-review.nih.gov/.