NOT-OD-16-011: Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications

Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications

Notice Number: NOT-OD-16-011

Key Dates
Release Date: October 9, 2015

Issued by
National Institutes of Health (NIH)
Agency for Healthcare Research and Quality (AHRQ)

Purpose

This notice informs the biomedical research community of updates to application instructions and review language intended to enhance the reproducibility of research findings through increased scientific rigor and transparency. These updates will take effect for most* research grant applications (including small business and complex research grant applications) submitted for due dates on or after January 25, 2016. For research contracts, this policy will be effective for proposals received on/after January 25, 2016 and expected to result in contract awards in Fiscal Year 2017 and beyond.

Updates include:

Revisions to application guide instructions for preparing your research strategy attachment
Use of a new "Authentication of Key Biological and/or Chemical Resources" attachment
Additional rigor and transparency questions reviewers will be asked to consider when reviewing applications

These updates focus on four areas deemed important for enhancing rigor and transparency:

1) the scientific premise forming the basis of the proposed research,
2) rigorous experimental design for robust and unbiased results,
3) consideration of relevant biological variables, and
4) authentication of key biological and/or chemical resources.

The basic principles of rigor and transparency and the four areas of focus apply to the full spectrum of research, from basic to clinical. Investigators will need to consider how all four areas apply to their proposed research. Likewise, reviewers will assess whether these areas have been appropriately addressed by the applicant through revised language defining the peer review criteria.

*Notes & Exceptions:

Research grant activity codes excluded from this policy include C06, G08, G11, G12, G13, G20, R13, S06, S10, S21, SB1, U13, U55, UB1, UC6, UC7, UG4, UH4, X02, and 333.
Research Resource and Related grants or components (P30, P40, P41, P2C, R24, R28, U24, U41, U42, and U2C) may have slightly revised review language; please refer to the Funding Opportunity Announcement.
Refer to NOT-OD-16-012 for updates to Career Development Award application instructions and review language.
Fellowship and Training grant applications submitted for the May 25, 2016 due date and beyond will include new instructions and review criteria to address this policy. Details on these changes will be available by December 2015.

Implementation for Grant Applications

Updates to Research Strategy Guidance

By November 25, 2015 application guide instructions will be updated to include the following additional guidance for the Significance and Approach sections of the Research Strategy, in addition to the existing instructions.

Significance
Describe the scientific premise for the proposed project, including consideration of the strengths and weaknesses of published research or preliminary data crucial to the support of your application.

Approach
Describe the experimental design and methods proposed and how they will achieve robust and unbiased results.

Explain how relevant biological variables, such as sex, are factored into research designs and analyses for studies in vertebrate animals and humans. For example, strong justification from the scientific literature, preliminary data, or other relevant considerations, must be provided for applications proposing to study only one sex. Refer to NOT-OD-15-102 for further consideration of NIH expectations about sex as a biological variable.

New Authentication of Key Biological and/or Chemical Resources Attachment

Grant applications for the activity codes covered by the policy must include a new PDF attachment related to the authentication of key biological and/or chemical resources.

Authentication of Key Biological and/or Chemical Resources
Briefly describe methods to ensure the identity and validity of key biological and/or chemical resources used in the proposed studies.

Key biological and/or chemical resources may or may not be generated with NIH funds and:
1) may differ from laboratory to laboratory or over time;
2) may have qualities and/or qualifications that could influence the research data; and
3) are integral to the proposed research.

These include, but are not limited to, cell lines, specialty chemicals, antibodies, and other biologics.

Standard laboratory reagents that are not expected to vary do not need to be included in the plan. Examples are buffers and other common biologicals or chemicals.

Reviewers will assess the information provided in this Section. Any reviewer questions associated with key biological and/or chemical resource authentication will need to be addressed prior to award.

Information in this section must focus only on authentication and/or validation of key resources to be used in the study; all other methods and preliminary data must be included within the page limits of the research strategy. Applications identified as non-compliant with this limitation will be withdrawn from the review process (see NOT-OD-15-095).

Applications submitted for due dates between January 25, 2016 and May 24, 2016 will use the FORMS-C forms and application guide. The general application guide will be updated by November 25, 2015 with instructions for this new attachment and guidance to upload your PDF document (titled "Authentication of Key Resources Plan") in the "Other Attachments" section of the "Other Project Information" form.

Applications submitted for due dates on or after May 25, 2016, will use updated FORMS-D forms. The PHS 398 Research Plan form will include a new "Authentication of Key Biological and/or Chemical Resources" attachment field. FORMS-D application forms and instructions will be available for all active Funding Opportunity Announcements at least 60 days prior to due dates that fall on or after May 25, 2016.

Application Review Information

Unless stated otherwise in the Funding Opportunity Announcement, reviewers will be asked to consider additional review questions in order to assess rigor and transparency in research grant applications. By November 25, 2015, all active Funding Opportunity Announcements will be updated to reference these additional review questions.

Scored Review Criteria

Significance
Is there a strong scientific premise for the project

Approach
Have the investigators presented strategies to ensure a robust and unbiased approach, as appropriate for the work proposed?

Have the investigators presented adequate plans to address relevant biological variables, such as sex, for studies in vertebrate animals or human subjects?

Additional Review Considerations

Authentication of Key Biological and/or Chemical Resources
For projects involving key biological and/or chemical resources, reviewers will comment on the brief plans proposed for identifying and ensuring the validity of those resources.

Research Performance Progress Reports

Research Performance Progress Reports (RPPR) submitted January 25, 2016 or later will be expected to emphasize rigorous approaches taken to ensure robust and unbiased results. Rigor should be addressed in the RPPR for any grant that funds research or training in research; grants that support other activities do not need to address rigor. This includes non-competing continuation reports (Type 5) for grants reviewed and awarded before implementation of the policy. The RPPR instructions will be updated by January 25, 2016. Reporting on rigor in RPPR will help NIH implement and evaluate the policy for both current and new awards, as well as prepare non-competing renewals for the next competitive renewal.

Resources

Website describing reproducibility efforts for NIH applicants and grantees
Frequently Asked Questions

Background

NIH’s mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability. Key to the successful application of that knowledge toward health outcomes is scientific rigor in conducting biomedical research. One of NIH’s four stated goals is to exemplify and promote the highest level of scientific integrity, public accountability, and social responsibility in the conduct of science (see http://www.nih.gov/about/mission.htm).

These rigor and transparency updates:

clarify long-standing expectations to ensure that NIH is funding the best and most rigorous science,
highlight the need for applicants to describe details that may have been previously overlooked,
highlight the need for reviewers to consider such details in their reviews through updated review language, and
minimize additional burden.

These are not new expectations, but NIH is formalizing these expectations in grant applications and reviews. Some investigators already address some of the four areas of rigor in their applications, while other investigators are doing so in their research but not providing details in applications and/or publications. All biomedical science will benefit from increased attention to rigor and transparency in research grant applications and reviews.

Inquiries

Please direct all inquiries to:

[email protected]