December 8, 2022
NOT-OD-24-085 - Simplified Review Framework for NIH Research Project Grant Applications - Update and Implementation Plans
NOT-OD-24-084 - Overview of Grant Application and Review Changes for Due Dates on or after January 25, 2025.
NOT-OD-24-012 - Online Briefing on NIHs Simplified Peer Review Framework for NIH Research Project Grant (RPG) Applications: for Applicants and Reviewers
NOT-OD-24-010 - Simplified Review Framework for NIH Research Project Grant Applications
NATIONAL INSTITUTES OF HEALTH (NIH)
The purpose of this Request for Information (RFI) is to solicit public input on a proposed revised framework for evaluating and scoring peer review criteria for National Institutes of Health (NIH) research project grant (RPG) applications. NIH is proposing a revised simplified framework that will reorganize five major regulatory criteria under three scored categories and reduce the number of non-score driving review considerations that reviewers evaluate in judging the scientific merit of RPG applications. The proposed changes pertain to those RPGs with standard review criteria. All the factors required by regulation will continue to be evaluated. NIH is not proposing to revise the regulatory criteria. Rather, NIH is proposing to revise its policy of how peer reviewers score the criteria, and how NIH organizes the criteria for review purposes. NIH believes that these changes will allow peer reviewers to refocus on the critical task of judging scientific merit and will improve those judgements by reducing bias.
Current Process
The first stage of NIH peer review serves to provide expert advice to NIH on the scientific and technical merit of grant applications. The NIH peer review regulations at 42 C.F.R. Part 52h.8 state that for research project grant applications,
the scientific peer review group shall assess the overall impact that the project could have on the research field involved, taking into account, among other pertinent factors:
- The significance of the goals of the proposed research, from a scientific or technical standpoint;
- Approach: The adequacy of the approach and methodology proposed to carry out the research;
- Innovation: The innovativeness and originality of the proposed research;
- Investigator(s): The qualifications and experience of the principal investigator and proposed staff;
- Environment: The scientific environment and reasonable availability of resources necessary to the research;
- The adequacy of plans to include both genders, minorities, children and special populations as appropriate for the scientific goals of the research;
- The reasonableness of the proposed budget and duration in relation to the proposed research; and
- The adequacy of the proposed protection for humans, animals, and the environment, to the extent they may be adversely affected by the project proposed in the application.
By NIH policy, peer reviewers are currently also required to evaluate Biohazards, Resubmissions, Foreign Organizations, Select Agents, Resource Sharing Plans, and Authentication of Key Biological and/or Chemical Resources. NIH currently gives the first five of the regulatory factors the following categorical labels: Significance, Approach, Innovation, Investigator(s), and Environment.
The NIH peer review regulation does not address scoring. Scoring of all regulatory factors is determined by NIH policy. Currently, peer reviewers provide an Overall Impact Score (scored 1-9) that reflects the overall scientific and technical merit of the application and individual criterion scores for Significance, Investigators, Innovation, Approach, and Environment. The remaining factors, Protections for Human Subjects, Inclusion, Vertebrate Animals, Biohazards, Resubmission, Renewal, and Revision are evaluated and factored into the Overall Impact Score; however, they are not given individual scores. When reviewers judge any of these to be unacceptable, they are asked to provide justification for that assessment. Beyond these factors, reviewers are asked to assess the following additional review considerations, but these considerations are not considered when reviewers determine an Overall Impact Score: Applications from Foreign Organizations, Select Agents, Resource Sharing Plans, Authentication of Key Biological and/or Chemical Resources, Budget & Period of Support.
Proposal Development
NIH gathered input from many sources in forming this proposal. Unsolicited comments over a period of years, reflecting sustained concerns from reviewers and applicants regarding complexity of review criteria, administrative load, and potential biases led the Center for Scientific Review (CSR) to form a working group to the CSR Advisory Council. To inform that group, CSR published a Review Matters blog which was cross-posted on the Office of Extramural Research blog, Open Mike. The blog received more than 9,000 views by unique individuals and over 400 comments. The working group presented interim recommendations to the CSR Advisory Council, which adopted the recommendations, at public CSR Advisory Council meetings (March 2020 video, slides; March 2021 video, slides). Final recommendations from the CSR Advisory Council (report) were considered by the CSR Director, as well as major internal NIH extramural-focused committees that included leadership from across NIH institutes and centers. This process produced many modifications and the final proposal presented below. Additional background information can be found here.
Proposed Revised Simplified Review Framework
An Overall Impact Score (scored 1-9) will reflect the overall scientific and technical merit of the application. Reviewers will take into account their assessments of the three factors below and the following additional criteria in determining an Overall Impact Score. Of the three factors, only Factor 1: Importance of the Research and Factor 2: Feasibility and Rigor, will receive individual scores. In the revised framework, Factor 3: Expertise and Resources will not receive an individual score. The additional review criteria below will not receive individual scores but will be considered in arriving at the Overall Impact Score. Two review considerations will be evaluated but have no effect on the Overall Impact Score. Detailed descriptions of the three factors can be found here.
Factor 1: Importance of the Research (scored 1-9)
Factor 1 is based on the criteria Significance and Innovation.
Factor 2: Feasibility and Rigor (scored 1-9)
Factor 2 is based on the criterion Approach.
Factor 3: Expertise and Resources (rated as fully capable , appropriate or additional capability/expertise needed or additional resources needed )
Factor 3 is based on the criteria Investigator and Environment. If additional expertise/capability needed or additional resources needed is selected, justification must be provided.
Additional Criteria (not scored, but affecting Overall Impact):
- Human Subject Protections
- Inclusion of Women, Minorities, and Individuals Across the Lifespan
- Vertebrate Animals
- Biohazards
- Resubmission/Renewal/Revisions
Each of the Additional Criteria except the last will be rated as Appropriate , with no comments required, or as Concerns , which must be briefly justified. Resubmission/Renewal/Revisions will be given brief written evaluations.
Additional Review Considerations (not scored and having no effect on Overall Impact):
- Authentication of Key Biological and/or Chemical Resources
- Rated as Appropriate with no comments required, or as Concerns , which must be briefly described.
- Budget and Period of Support
- Rated as Appropriate , Excessive , or Inadequate ; the latter two ratings requiring a brief account of concerns.
The additional review considerations, including Foreign Organizations, Select Agents, and Resource Sharing Plans, will no longer be evaluated by peer reviewers.
Restructuring the categorization and scoring of criteria in this way reduces the number of scores reviewers need to provide, and policy considerations reviewers need to take into account when evaluating scientific merit. It focuses reviewers on the two most important judgements about a proposed research project; how important the research is, and how rigorous and feasible the approach is. Evaluation of the investigators and research environment is framed in terms of whether the expertise and resources needed to accomplish the project are available, thus diminishing halo effects-- diffuse judgements of investigator or institutional reputation that bias judgements of research importance, rigor, and feasibility.
90-day Comment Period
Comments must be received by 11:59:59 PM (ET) on March 10, 2023, to ensure consideration.
Information Requested
NIH is specifically requesting public comment on the Proposed Revised Simplified Review Framework, a proposed revised framework for evaluating and scoring peer review criteria for NIH research project grant applications, described above. Response to this RFI is voluntary.
Submitting a Response
Comments should be submitted electronically to the following webpage at: https://rfi.grants.nih.gov/?s=638509b5409baa49f803e572.
This RFI is for planning purposes only and should not be construed as a policy, solicitation for applications, or as an obligation on the part of the Government to provide support for any ideas identified in response to it. Please note that the Government will not pay for the preparation of any information submitted or for its use of that information.
Please do not include any proprietary, classified, confidential, or sensitive information in your response. Responses will be compiled and a content analysis will be shared publicly after the close of the comment period. The NIH may use information gathered by this Notice to inform future policy development.
and Human Services (HHS)
