NOT-OD-17-050: Reporting Preprints and Other Interim Research Products

Reporting Preprints and Other Interim Research Products

Notice Number: NOT-OD-17-050

Key Dates
Release Date: March 24, 2017
Effective date for application: Applications submitted for the May 25, 2017 due date and thereafter
Effective date Research Performance Progress Report (RPPR): RPPRs submitted on or after May 25, 2017

Related Announcements
NOT-OD-17-006- Request for Information (RFI): Including Preprints and Interim Research Products in NIH Applications and Reports

Issued by
National Institutes of Health (NIH)

Purpose

The NIH encourages investigators to use interim research products, such as preprints, to speed the dissemination and enhance the rigor of their work. This notice clarifies reporting instructions to allow investigators to cite their interim research products and claim them as products of NIH funding.

Definitions

Interim Research Products are complete, public research products that are not final.

A common form is the preprint, which is a complete and public draft of a scientific document. Preprints are typically unreviewed manuscripts written in the style of a peer-reviewed journal article. Scientists issue preprints to speed dissemination, establish priority, obtain feedback, and offset publication bias.

Another common type of interim product is a preregistered protocol, where a scientist publicly declares key elements of their research protocol in advance. Preregistration can help scientists enhance the rigor of their work.

Notes:

Awardees are not required to create interim research products through their NIH award.
Applicants are not required to cite interim research products as part of their grant applications.
Since preprints are not published in peer-reviewed journals, they do not fall under the NIH public access policy.
This guide notice does not apply to clinical trial registration. See ClinicalTrials.gov about registration of clinical trial protocols.

Citing interim research products in applications, proposals and reports

Interim research products can be cited anywhere other research products are cited. These sections include the following:

R&R Other Project Information Form, Bibliography & References Cited
R&R Senior/Key Person Profile (Expanded) Form, Biographical Sketch
PHS 398 Research Plan, Progress Report Publication List
PHS 398 Career Development Award Supplemental Form, Progress Report Publication List
PHS Fellowship Supplemental Form, Progress Report Publication List
RPPR, section C - Products

To cite the product, applicants and awardees must include the Digital Object Identifier and the Object type (e.g. preprint, protocol) in the citation. Also list any information about the document version (e.g. most recent date modified), and if relevant, the date the product was cited.

Example: Bar DZ, Atkatsh K, Tavarez U, Erdos MR, Gruenbaum Y, Collins FS. Biotinylation by antibody recognition- A novel method for proximity labeling. BioRxiv 069187 [Preprint]. August 11, 2016 [cited 2017 Jan 12]. Available from: https://doi.org/10.1101/069187.

These requirements help reviewers understand that the product is public, interim, and identify the specific version is being referenced.

Note: Applicants are responsible for providing the information necessary to review a section of an application within the page limits of that section.

Claiming interim research products as products of NIH awards

NIH intends to maximize impact of interim research products that are developed with NIH funds. Therefore, NIH expects awardees to ensure a high level of public access to NIH supported interim products. To facilitate text mining and other analysis of these products as data, the NIH expects standardized terms of use. NIH also expects awardees will adhere to other norms of responsible scientific communication.

Specifically, to claim an interim research product as a product of an NIH award, the NIH expects that the awardee will:

Make the product publicly available. To maximize the impact of an interim research product, the NIH strongly encourages awardees to select a Creative Commons Attribution (CC-BY) license or dedicate their work to the public domain.
In the text of the document:

Acknowledge NIH funding in accordance with NIH Grants Policy Statement Chapter 8.2.1
Clearly state that the work is not peer-reviewed
Declare any competing interests, as an author would do for any journal article

For applications submitted for the May 25, 2017 due date and thereafter, awardees can claim these products on their progress report publication list. They can also report them on their RPPR as of May 25, 2017, and link them to their award in their My Bibliography account.

Guidance for selecting interim research product repositories

Interim research products rely on repositories to make them public. The repository market is growing rapidly, and in many scientific disciplines, norms for interim research products are still evolving.

The NIH would like to ensure that practices for interim products facilitate the impact, measurement and the integrity of the scientific record. Specifically, the NIH strongly encourages interim research products arising from NIH funds to be deposited in repositories that ensure:

Content is findable, accessible, interoperable and reusable.
Interim product metadata, including usage statistics, are open, and easy to access by machines and people (e.g. via application program interfaces).
Content is easy to use by machines and people. This access is both a function of permission (e.g. use of Creative Commons licenses) and technology (e.g. application program interfaces).
Policies about plagiarism, competing interests, misconduct and other hallmarks of reputable scholarly publishing are rigorous and transparent.
Records of changes to the product are maintained, and users have clear ways to cite different versions of the product.
Links to the published version, if available.
A robust archiving strategy that ensures long-term preservation and access.

Background and public comments

In the Request for Information (RFI) NOT-OD-17-006, the NIH sought input on the use of interim research products in NIH applications and reports, and the standards for reporting them. The NIH wanted to know if interim research products can increase the rigor and impact of NIH funded research. It also wanted to learn how interim research products arising from NIH funds can be created and used with integrity.

RFI respondents
The NIH received 351 responses, the majority (79%) submitted by scientists/authors. Twenty-two professional societies representing groups of scientists also submitted responses. Note, requests for information are not surveys. Some responses are from organizations representing thousands of people; other responses are from individuals. The NIH is grateful to receive rich responses and thoughtful advice through this information request, and used these findings to shape this policy.

Defining interim products
Some respondents were confused about what interim research products are and where they were used in the NIH grant process. The RFI could have more clearly stated that the NIH has never restricted the materials that can be cited in the reference section of a research plan. Further, several respondents referred to data and software as if NIH had not allowed them to be cited or claimed as a product of NIH funds, which is not correct. This guide notice, and related updates to application and RPPR instructions, intend to clarify these points.

Scientific impacts
Almost all respondents supported increasing the use of interim research products in NIH award processes. Many described specific scientific benefits. These include speeding dissemination by eliminating the months-long interval between submitting a manuscript to a publisher to the first public release of the manuscript. Several respondents reported that they would submit a manuscript to a journal and a preprint repository at the same time. Respondents also noted that public comments on their interim product can improve the rigor of their work, and are an opportunity to form new scientific collaborations. Finally, since interim research products are issued at the discretion of the author, they avoid publication bias.

Many respondents noted that interim research products can be especially helpful for new investigators. New investigators best work is sometimes so recent it has not had the time to be published. Preprints can help investigators share the complete drafts of their work sooner. Further, other interim research products, such as protocols, help new investigators to document their progress and engage other scientists in discussion about their work. Hosting repositories can amplify these benefits if they track utilization, comments, and other impact measures.

A few respondents claimed that interim products could help rivals finalize their research faster, thereby scooping the authors of interim products. However, several other respondents noted that since interim products are public, they establish priority for any inventions. Further, since authors have full control over the timing of an interim product release, they can always choose to not issue an interim product until a patent application is filed.

Information quality
Almost all respondents, even those that were strongly supportive of interim research products, felt that interim research products offered lower quality information than work that was formally peer-reviewed.

A few respondents noted special risks for the general public, clinicians, patients, and the media in accessing research products that have not been peer-reviewed. These risks are especially great for clinical research, and there are examples when even peer-reviewed findings have been hyped and misinterpreted by the media. The NIH expects the research community to be judicious in its use of interim research products, and for some disciplines (and their leading journals) to explore the use of interim products more slowly than other disciplines.

Interim research products and NIH processes
Several responses, including some from prominent scientific societies, noted that NIH processes can already be burdensome and involve many peer-reviewed products. They felt that including more non-peer-reviewed information into these processes will generate more burden than benefit.

Even more respondents argued that, on balance, interim products will be helpful. Interim products are similar in quality to the preliminary data section of a research plan that reviewers are already comfortable with. These respondents suggested that reviewers should be skeptical of all claims and citations, whether peer-reviewed or not.

The NIH agrees that interim research products offer lower quality information than peer-reviewed products. This policy is not intended to replace peer-review, nor peer-reviewed journals. Instead, the NIH sees interim research products complementing final research products.

The RFI collected the fewest responses to questions about how interim research products should be cited. Many respondents wanted to make sure that reviewers were aware that the interim product citation is not peer-reviewed, and suggested adding the label not peer- reviewed to the citation. This suggestion is difficult to enact because so many other standard citation formats that are not peer-reviewed do not say so in their citations (e.g. most book chapters and journal commentaries do not mention peer-review in the citation or text).

Instead, the NIH is instructing applicants and awardees that choose to cite interim research products to list the interim research product type (e.g. preprint) in the citation. Further, the NIH is instructing awardees to explicitly state in preprints text that the work is not peer-reviewed. These two practices should help reviewers easily identify interim products. The NIH will offer explicit guidance to reviewers reminding them that interim research products are not peer-reviewed. Further, since interim products are new to so many biomedical disciplines, the NIH hopes that these conventions will become the norm for all interim products, and will help the media and the public understand that interim products have undergone less review than peer-reviewed articles.

Finally, to ensure the integrity and impact of interim products, the NIH is borrowing from practices established for final products. This is why NIH is asking authors and repositories to ensure the integrity of these products by declaring competing interests, track versioning, etc. To ensure impact, the NIH is asking authors and repositories to ensure interim products are findable through DOIs, open metadata, etc. To ensure these products are usable as individual documents and in aggregate through computation analyses, the NIH is encouraging authors to adopt a number of license and technical processes (e.g. Creative Common Attribution licenses (CC-BY), application programming interfaces, archival plans, etc.), and to use repositories that support these practices.

Overall, the RFI responses described interim research products as a relatively small change that can provide benefits to NIH processes, and science as a whole. The NIH will be monitoring the use of interim products, their infrastructure, and their impacts. It may revise these standards as practice evolves.

Inquiries

Please direct all inquiries to:

Office of Policy for Extramural Research Administration (OPERA)
Email: [email protected]