April 24, 2023
NATIONAL INSTITUTES OF HEALTH (NIH)
The purpose of this Request for Information (RFI) is to solicit public input on proposed changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications that would restructure the review criteria and modify some sections of the NRSA application form (PHS Fellowship Supplemental Form). The goal of this effort is to facilitate the mission of NRSA fellowship peer review – to identify the most promising trainees and the excellent, individualized training programs that will help them become the outstanding scientists of the next generation. The proposed changes will 1) allow peer reviewers to better evaluate the applicant’s potential and the quality of the scientific training plan without undue influence of the sponsor’s or institution’s reputation; and 2) ensure that the information provided in the application is aligned with the restructured criteria and targeted to the fellowship candidate’s specific training needs.
The overall goal of the NIH Ruth L. Kirschstein National Research Service Award (NRSA) program is to help ensure that a diverse pool of highly trained scientists is available in appropriate scientific disciplines to address the Nation's biomedical, behavioral, and clinical research needs. NRSA fellowships support the training of pre-and postdoctoral scientists, dual-degree investigators, and senior researchers. The first stage of NIH peer review serves to provide expert advice to NIH by assessing the likelihood that the fellowship will enhance the candidate’s potential for, and commitment to, a productive independent scientific research career in a health-related field. The criteria for the review of NRSA fellowship applications derive from the NRSA regulation at 42 CFR 66.106:
(a) Within the limits of funds available, the Secretary shall make Awards to those applicants:
(1) Who have satisfied the requirements of §66.105; and (2) Whose proposed research or training would, in the judgment of the Secretary, best promote the purposes of section 487(a)(1)(A) of the Act, taking into consideration among other pertinent factors:
(i) The scientific, technical, or educational merit of the particular proposal;
(ii) The availability of resources and facilities to carry it out;
(iii) The qualifications and experience of the applicant; and
(iv) The need for personnel in the subject area of the proposed research or training.
The NIH peer review regulation does not address scoring. Scoring of all regulatory factors is determined by NIH policy. Currently, peer reviewers provide an Overall Impact Score (scored 1-9) that reflects their assessment of the likelihood that the fellowship will enhance the candidate’s potential for, and commitment to, a productive independent scientific research career in a health-related field. Peer reviewers provide individual criterion scores for five criteria: 1) Applicant, 2) Sponsors and Collaborators, 3) Research Training Plan, 4) Training Potential, and 5) Institutional Environment and Commitment. Additional review criteria are evaluated and factored into the Overall Impact Score but are not given individual scores: Protections for Human Subjects; Inclusion of Women, Minorities, and Individuals Across the Lifespan; Vertebrate Animals; Biohazards; and Resubmission. Beyond these criteria, reviewers are asked to assess the following additional review considerations; these considerations are not considered in the Overall Impact Score: Training in the Responsible Conduct of Research, Applications from Foreign Organizations, Select Agents, Resource Sharing Plans, Budget and Period of Support, and Authentication of Key Biological and/or Chemical Resources.
NIH gathered input from many sources in forming this proposal. Unsolicited comments over a period of years conveyed persistent concerns that the NRSA fellowship review process disadvantages some highly-qualified, promising applicants. In response, the Center for Scientific Review (CSR) formed a working group to the CSR Advisory Council. To inform that group, CSR published a Review Matters blog, inviting comments, which was cross-posted on the Office of Extramural Research blog, Open Mike. The working group presented an interim report to the CSR Advisory Council, which adopted the recommendations, at public CSR Advisory Council meetings (March 2022 video, slides; September 2022 video, slides). Final recommendations from the CSR Advisory Council (report) were considered by the CSR Director, as well as major internal NIH extramural-focused committees that included leadership from across NIH institutes and centers. Additional background information can be found here.
I. Revise the Criteria Used to Evaluate NRSA Fellowship Applications
As is currently the case, the Overall Impact Score (scored 1-9) will reflect the scientific and educational merit of the proposal and an assessment of the likelihood that the fellowship will enhance the applicant’s potential for, and commitment to an independent, productive research career in a health-related field. However, the current 5 scored criteria that inform the Overall Impact Score will be restructured into the following 3 scored criteria. Additional detail on proposed reviewer guidance can be found here.
Criterion 1: Scientific Potential, Fellowship Goals, and Preparedness of the Applicant (scored 1-9)
Criterion 2: Science and Scientific Resources (scored 1-9)
Criterion 3: Training Plan and Training Resources (scored 1-9)
The Additional Review Criteria (e.g., Protections for Human Subjects; Inclusion of Women, Minorities and Individuals Across the Lifespan; etc.) would not change.
The Additional Review Considerations (e.g., Training in the Responsible Conduct of Research, Resource Sharing Plans, Budget, etc.) would not change.
Revising the criteria simplifies the task of reviewers by focusing their attention on just three key assessments; the scientific potential of the applicant, the science and scientific resources, and the training plan and training resources. The criteria are defined to give applicants from heterogeneous backgrounds a fair chance; reviewers are asked to evaluate applicant accomplishments and trajectory in the context of the opportunities they have had. In addition to evaluating applicant accomplishments, reviewers are asked to evaluate personal characteristics that contribute to success in science, factors such as determination, persistence, and creativity. The revised criteria are also expected to reduce bias in review by reducing any consideration of sponsor and institutional reputation and instead focusing review on their specific, realistic, and current contributions to the scientific needs, goals, and training of the specific trainee. NIH believes these changes will better enable peer review to identify those applications with the highest potential for producing productive research scientists, regardless of where the applicant started or the applicant institution.
II. Revise the Fellowship Supplemental Section of PHS SF424
The NIH proposes to revise the following sections of the PHS 424 Fellowship Supplement: 1) Fellowship Applicant and 2) Sponsor(s), Collaborator(s), and Consultant(s), 3) Letters of Reference. There are no proposed changes to the Research Training Plan section. Additionally, the revision would allow an optional Statement of Special Circumstances.
The changes are intended to restructure the application so that the application content is better aligned with the review criteria, is less duplicative, and is easier for reviewers to assess. The changes emphasize substantive statements that pertain to the individual applicant trainee, require detailed accounts from sponsors explaining their preparation and approach to training, and their availability to the student. The changes would shorten the application by up to 2 ½ pages. The proposed changes for each section are described below.
1. Revised Applicant Section of the Fellowship Supplement
Applicants would be asked to submit five statements:
Additionally, grades would no longer be required or allowed, however, applicants would be requested to include the titles of relevant courses completed.
2. Revised Sponsor and Co-sponsor Section of the Fellowship Supplement
Sponsors and Co-sponsors would be asked to submit three statements:
3. Revised Instructions for Reference Letters
NIH proposes to update the instructions for reference letters with more structure so that the resulting letters better assist reviewers in understanding the applicant’s strengths, weaknesses, and potential to pursue a productive career in biomedical science. Writers would be instructed to respond to four questions addressing:
4. Allow an Optional Statement of Special Circumstance
NIH recommends allowing fellowship applicants to submit an optional Statement of Special Circumstance to address situations that may have hindered the trainee’s progress, such as harassment, the COVID-19 pandemic, or other personal or professional circumstances.
Additional detail on proposed changes to the Fellowship Supplement can be found here.
NIH is requesting public comment on the Recommendations for Improving NRSA Fellowship Review, specifically, the proposed restructuring of criteria used to evaluate NRSA fellowship applications and proposed revisions to the Fellowship Supplemental Section of PHS SF424, as described above. Response to this RFI is voluntary. Responses may be submitted here: https://rfi.grants.nih.gov/?s=642ed5def0356688b20e6be3.