Recommendations for Improving NRSA Fellowship Review

Recommendations for Improving NRSA Fellowship Review

Recommendation 1: Revise the Criteria Used to Evaluate NRSA Fellowship Applications

An Overall Impact Score (scored 1-9) will reflect the scientific and educational merit of the proposal and an assessment of the likelihood that the fellowship will enhance the applicant’s potential for, and commitment to, an independent, productive research career in a health-related field. Reviewers will take into account their assessments of the three criteria in determining an Overall Impact Score. Each of the three criteria will receive an individual score. The additional review criteria below will not receive individual scores but will be considered in arriving at the Overall Impact Score. Two review considerations will be evaluated but have no effect on the Overall Impact Score.
 

View full text of the proposed review criteria along with reviewer guidance and proposed application changes.

---

1. Scientific Potential, Fellowship Goals, and Preparedness of the Applicant
   1. Criterion emphasizes their accomplishments in the context of their stage of training and the scientific opportunities they have had, as well as other factors that bear on their potential to succeed, such as determination, persistence, and creativity. (scored 1-9)
2. Science and Scientific Resources
   1. Criterion emphasizes the extent to which needed technical, scientific, and clinical resources are specified and are realistically available to the applicant, and whether the scientific expertise of the mentorship team is appropriate for the proposed science and that the role of each mentor is clearly defined. Peer reviewers would not assess financial support for the proposed research; that would be done by NIH staff. (scored 1-9)
3. Training Plan and Training Resources
   1. Criterion emphasizes whether the necessary training resources are well-specified and available, specifically the practical availability of resources, and to include an evaluation of the training philosophy of the sponsor, their approach to training, time commitments, and their accessibility. (scored 1-9)

---

Additional Criteria (not scored, but affecting Overall Impact; no changes proposed, see current language):

• Protections for Human Subjects
• Inclusion of Women, Minorities, and Individuals Across the Lifespan
• Vertebrate Animals
• Biohazards
• Resubmission
  
  Each of the Additional Criteria except the last will be rated as “Appropriate”, with no comments required, or as “Concerns”, which must be briefly justified. When applicable, Resubmission will be given brief written evaluations.

---

Additional Review Considerations (not scored and having no effect on Overall Impact) no changes proposed, see current language):

• Training in the Responsible Conduct of Research
  • Five subsections (format, subject matter, faculty participation, duration, frequency), each rated as “Appropriate” with no comments required, or as “Concerns”, which must be briefly described.
• Authentication of Key Biological and/or Chemical Resources 
  • Rated as “Appropriate” with no comments required, or as “Concerns”, which must be briefly described.
• Budget and Period of Support 
  • Rated as “Appropriate”, “Excessive”, or “Inadequate”; the latter two ratings requiring a brief account of concerns.
• Applications from Foreign Organizations
  • Rated as “Acceptable”, “Unacceptable”, or “Not Applicable”.
• Select Agents
  • Rated as “Acceptable”, “Unacceptable”, or “Not Applicable (No Select Agents)”.
• Resource Sharing Plans
  • Rated as “Acceptable”, “Unacceptable”, or “Not Applicable (No Relevant Resources)”.

 

---

Recommendation 2: Revise the PHS Fellowship Supplemental Form

Changes to the application instructions are needed to better align the information applicants provide with the revised review criteria. The PHS Fellowship Supplemental Form currently includes the following four sections: Fellowship Applicant; Research Training Plan; Sponsor(s), Collaborator(s), and Consultant(s); Institutional Environment and Commitment to Training. The NIH proposes to revise the Fellowship Applicant section and the Sponsor(s), Collaborator(s) and Consultant(s) section. NIH also proposes to change the instructions for reference letters. An additional proposed change would allow an optional Statement of Special Circumstances from the fellowship applicant. Changes to each section are summarized below. The full text of the proposed changes to the application can be found here.

Summary of Revisions to Applicant Section

Grades would no longer be required or allowed; however, the titles of courses completed would be required.

Applicants would be asked to submit five statements: (1) a statement of professional and fellowship goals, (2) fellowship qualifications, (3) a self-assessment, (4) a statement of scientific perspective, and (5) a detailed account of activities planned under this award. Additionally, grades would no longer be required or allowed; however, the titles of courses completed would be required.

Summary of Revisions to Applicant Sponsor and Co-sponsor Section

Sponsors and Co-sponsors would be asked to submit three statements: (1) an account of the training plan, environment, and research facilities; (2) the number of fellows/trainees to be supervised during the fellowship; and (3) the applicant’s qualifications and potential for a research career.

Summary of Revisions to Applicant Instructions for Reference Letters

NIH proposes to update the instructions for reference letters with more structure so that the resulting letters better assist reviewers in understanding the applicant’s strengths, weaknesses, and potential to pursue a productive career in biomedical science. Writers would be instructed to respond to four prompts: (1) identify the 2-4 most important characteristics of this applicant that will contribute to this applicant becoming a successful scientist and explain why; (2) identify 2-4 areas for growth that the applicant should address to improve their prospects of becoming a productive biomedical scientist; (3) what scientific and intellectual contributions has the applicant made during their training/research experience with you?; (4) give an overall assessment of the applicant’s readiness for this fellowship taking into account their background, potential, current skills or abilities, and career goals.

Allow an Optional Statement of Special Circumstance

NIH recommends allowing fellowship applicants to submit an optional Statement of Special Circumstance to address situations that may have hindered the trainee’s progress, such as harassment, the COVID-19 pandemic, or other personal or professional circumstances.

Seeking External Input

NIH gathered input from many sources in forming this proposal. Unsolicited comments over a period of years, reflecting persistent concerns that the NRSA fellowship review process disadvantages some highly-qualified, promising applicants led the Center for Scientific Review (CSR) to form a working group to the CSR Advisory Council. To inform that group, CSR published a Review Matters blog which was cross-posted on the Office of Extramural Research blog, Open Mike. The blog received more than 1,500 views by unique individuals and numerous comments. The working group presented interim report to the CSR Advisory Council, which adopted the recommendations, at public CSR Advisory Council meetings (March 2022 video, slides; September 2022 video, slides). Final recommendations from the CSR Advisory Council (report) were considered by the CSR Director, as well as major NIH extramural program, review, and policy committees that included leadership from across NIH. Recommendations were presented to the NIH Advisory Committee to the Director in December 2022 (video, slides).

Through the request for information (RFI), NIH sought additional public input on the proposed revised review framework. The RFI closed June 23, 2023.

Summary of Feedback from the RFI