April 19, 2023
AGENCY FOR HEALTHCARE RESEARCH AND QUALITY (AHRQ)
AHRQ seeks public input on the AHRQ plan for updating the agency policy for public access to scientific publications and scientific data resulting from AHRQ funding
(AHRQ Public Access Plan) (https://www.ahrq.gov/funding/policies/publicaccess/index.html).
On February 22, 2013, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled "Increasing Access to the Results of Federally Funded Scientific Research." This memorandum requires federal agencies to make the results of federally funded scientific research available to and useful for the public, industry, and the scientific community. In response, AHRQ published a plan for establishing a policy for public access to scientific publications and scientific data resulting from AHRQ funding (https://www.ahrq.gov/funding/policies/publicaccess/index.html) in February 2015, a Policy for Public Access to AHRQ funded Scientific Publications (NOT-HS-16-008) in February 2016,) and in May 2020, a Data Management Plan Policy (NOT-HS-20-011).
On August 25, 2022, the White House OSTP released a memorandum on Ensuring Free, Immediate, and Equitable Access to Federally Funded Research (2022 Memo). The 2022 Memo establishes new guidance for improving public access to scholarly publications and data resulting from federally supported research. Accordingly, AHRQ updated its Public Access Plan that outlines the proposed approach AHRQ will take to implement the new guidance, consistent with its longstanding commitment to public access.
Request for Information
AHRQ seeks public information regarding the updated AHRQ Public Access Plan. Comments are welcome on all elements of the AHRQ Public Access Plan.
How to Submit a Response:
Comments can be submitted to firstname.lastname@example.org with subject line as Response to AHRQ Updated Public Access Plan. Responses will be accepted through May 31, 2023.
Responses to this RFI are voluntary and may be submitted anonymously. You may also voluntarily include your name and contact information with your response. Other than your name and contact information, please do not include in the response any personally identifiable information or any information that you do not wish to make public. Proprietary, classified, confidential, or sensitive information should not be included in your response.
AHRQ Updated Public Access Plan:
This document is the Agency for Healthcare Research and Quality (AHRQ) plan for establishing an updated policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding.
1. Background & Purpose
6. Legal Authorities
7. Roles and Responsibilities
9. Metrics, Compliance, and Evaluation
12. Public Consultation
13. Interagency Coordination
14. Public Notice
15. Update and Re-Evaluation of the Policy
16. Additional Information
17. Material Superseded
1. Background & Purpose
On February 22, 2013, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled "Increasing Access to the Results of Federally Funded Scientific Research." This memorandum requires federal agencies to make the results of federally funded scientific research available to and useful for the public, industry, and the scientific community. In response, in February 2015, AHRQ published a plan for establishing a policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding, in February 2016 published a Policy for Public Access to AHRQ funded Scientific Publications (NOT-HS-16-008), and in May 2020, published a Data Management Plan Policy (NOT-HS-20-011).
On August 25, 2022, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled Ensuring Free, Immediate, and Equitable Access to Federally Funded Research (2022 Memo). The 2022 Memo provides policy guidance to federal agencies with research and development expenditures on updating their public access policies. In accordance with this memorandum, OSTP recommends that federal agencies, to the extent consistent with applicable law:
1. Update their public access policies as soon as possible, and no later than December 31, 2025, to make publications and their supporting data resulting from federally funded research publicly accessible without an embargo on their free and public release;
2. Establish transparent procedures that ensure scientific and research integrity is maintained in public access policies; and,
3. Coordinate with OSTP to ensure equitable delivery of federally funded research results and data.
This document is the AHRQ updated plan for updating policies for public access to scientific publications and scientific data in digital format resulting from AHRQ funding pursuant to recommendations from the 2022 Memo. AHRQ intends, to the fullest extent possible and contingent on the availability of funds, to make available to the public all scientific publications and data arising from unclassified research and programs funded wholly or in part by AHRQ.
The AHRQ mission is to produce evidence that makes health care safer, higher quality, more accessible, equitable and affordable, and to work within the U.S. Department of Health and Human Services (HHS) and with other partners to make sure the evidence is understood and used.
The AHRQ Public Access Policy promotes the following objectives:
The updated AHRQ Public Access Policy will be fully effective by December 31, 2025, with the policy implementing Scientific and Research Integrity in Agency Public Access Policies by December 31, 2026. The Public Access Policy will apply to all scientific publications and data in digital format. Implementation will be prospective and will not apply to any publication or digital data set arising from an AHRQ-sponsored grant, cooperative agreement, contract, or intramural research project funded prior to publication of the final updated AHRQ Public Access Policy.
The updated AHRQ Public Access Policy will apply to all research funded by AHRQ, except where such research is administered or performed by a partner Agency with a comparable Public Access Policy, in which case AHRQ will defer to the partner Agency's policies on the management of scholarly publications and digital data sets.
Digital scientific data that are covered by this policy include:
Per OMB Circular A-110, the following are not research data and therefore not subject to this policy: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens. However, investigators are encouraged to include relevant content from laboratory or field notebooks in their published data sets if that information provides documentation that would help future users successfully re-use the data. (That is, lab and field notebooks are not themselves scientific data, but may contain relevant metadata to be included in the published data set.).
For scholarly publications, the AHRQ Public Access Policy has required and will continue requiring authors to submit the final peer-reviewed accepted journal manuscripts to PubMed Central (PMC). In lieu of the final peer-reviewed manuscript, AHRQ has and will continue accepting the final published article, provided the authors can ensure AHRQ has the rights to make the published version public. AHRQ's Public Access Policy is subject to law; Agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed in the OSTP directive.
Scientific data underlying peer-reviewed scholarly publications resulting from federally funded research should be made freely available and publicly accessible by default at the time of publication, unless subject to limitations as described in Section 3(c)(i) of the August 25, 2022 OSTP memo and should be subject to federal agency guidelines for researcher responsibilities regarding data management and sharing plans, consistent with Section 3(c) of the memorandum.
Scientific Data include the recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings. Such scientific data do not include laboratory notebooks, preliminary analyses, case report forms, drafts of scientific papers, plans for future research, peer-reviews, communications with colleagues, or physical objects and materials, such as laboratory specimens, artifacts, or field notes. The definition of scientific data is similar to but broader than the term research data defined by 2 CFR 200.315 (e) and 45 CFR 75.322 (e).
AHRQ funded researchers must follow federal laws and OMB policies that govern federal agencies information management practices and protect certain types of data, to the extent that the scientific data created by, collected by, under the control or direction of, or maintained by the federal researchers is subject to those laws and policies.
To the extent feasible and consistent with applicable law and policy; agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed below, digitally formatted scientific data resulting from unclassified research supported wholly or in part by federal funding should be stored and publicly accessible to search, retrieve, and analyze. For sharing of data in digital format, all AHRQ-funded researchers have been and will continue to be required to include a data management plan for sharing final research data in digital format, or state why data sharing is not possible. Data management plans will be evaluated by AHRQ staff and peer review committees for merit while considering the values of long-term preservation, access, and the associated cost, and administrative burden. AHRQ expects the timely release and sharing of data to be no later than the acceptance for publication of the main findings from the final dataset. The specific time will be influenced by the nature of the data collected. Researchers will be directed to work with AHRQ staff and commercial repositories to deposit data upon publication. It may not be feasible to share all data; the costs and benefits of sharing data should be considered in data management planning. AHRQ intramural research is conducted using public use data. These data are also available to the public. AHRQ data that include personally identifiable data are available to the public in the AHRQ data center. AHRQ will attempt to make all intramural research data available to the public by creating de-identified public use data files.
If an AHRQ-funded grant, cooperative agreement, contract or intramural research project transforms or links datasets, rather than producing a new set of data, the researchers will be required to submit a data management plan. If there are limitations associated with the data sharing agreements for the original data that preclude subsequent sharing, the research applicants must explain these limitations.
Data from small studies can be analyzed and submitted for publication relatively quickly. If data from large epidemiologic or longitudinal studies are collected over several discrete time periods or waves, it is reasonable to expect that the data would be released in waves as data become available or main findings from waves of the data are published. AHRQ-funded extramural and intramural researchers will be expected to include updates on public access to publications and data in digital format resulting from their research in quarterly reports, where applicable, annual progress reports, final reports, and via NLM's Bibliography tool, My NCBI. All new grant and cooperative agreement applications, and contract proposals will include a section detailing the applicants' progress to-date with sharing of publication and data in digital format arising from previous AHRQ funding.
Scientific and Research Integrity in Agency Public Access Policies
AHRQ will take actions to ensure that scientific and research integrity policies are in place in order to strengthen public trust in federally funded science. By December 31, 2024, AHRQ plans to submit to OSTP and OMB a second update to their public access plans specifying approaches taken to implement the provisions in this Section 4 of the 2022 memo. AHRQ plans to complete and publish full policy development for plans implementing these provisions by December 31, 2026, with an effective date no later than one year after the publication of the agency plan.
In the plan to be submitted to OSTP and OMB by December 31, 2024, AHRQ will discuss and explore how AHRQ plans to act to achieve the goals specified and approaches recommended in the OSTP memo.
White House Office of Management and Budget Open Data Policy Memorandum (M-13-13)
AHRQ will meet the requirements of M-13-13 as specified in the accompanying memorandum, titled HHS Guiding Principles and Common Approach for Enhancing Public Access. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory for the data sets related to their major Information Technology investments. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory, cataloguing the various sets of data acquired or managed. The inventory will catalog both data sets that are made publicly available and data sets that are private to the Agency.
Scientific Data: scientific data include the recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings. Such scientific data do not include laboratory notebooks, preliminary analyses, case report forms, drafts of scientific papers, plans for future research, peer-reviews, communications with colleagues, or physical objects and materials, such as laboratory specimens, artifacts, or field notes. The definition of scientific data is similar to but broader than the term research data defined by 2 CFR 200.315 (e) and 45 CFR 75.322 (e).
Research Data: Research data is defined in OMB Circular A-110 as the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues. This 'recorded' material excludes physical objects (e.g., laboratory samples).
Research data do not include:
Digital Scientific Data: For the purpose of this plan and consistent with OMB Circular A-110, digital scientific data is defined as "the digital recorded factual material commonly accepted in the scientific community as necessary to validate research findings including data sets used to support scholarly publications, but does not include laboratory notebooks, preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens."
For the purpose of this plan, the definition of digital scientific data does not include software. However, AHRQ recognizes that in some cases, software and other tools such as interview protocols, measures, coding guides, or manuals may be necessary to interpret data. In such cases, the data management plan will be expected to include a description of these tools.
A published data set consists of at least one formal metadata document, the digital scientific data described by that metadata, and supplemental information provided to assist the data user. The metadata for scientific data will include, at a minimum, the common core metadata schema in use by the Federal Government, found at https://project-open-data.cio.gov/.
A published data set can be cited in the scientific literature and, following the practice used by most publishers of scholarly journals, has a persistent and unique identifier associated with it. A published data set is expected to persist over time, just like the scholarly articles that are based on the digital scientific data.
Data Sharing Plan: A data sharing plan outlines whether and how data will be made available to others. The plan must also explicitly describe how the data that underlies scientific publications will be available for discovery, retrieval, and analysis. It may include the expected timeline for when the data will be available, the format of the final dataset, the documentation and any analytic tools that will be provided, and the mode of data sharing (e.g., under their own auspices by mailing a disk or posting data on their institutional or personal website, through a data archive or enclave). A plan might also specify whether or not a data sharing agreement will be required and, if so, a brief description of such an agreement (including the criteria for deciding who can receive the data and whether or not any conditions will be placed on their use). If data sharing is not possible, the plan would provide an explanation.
Data Management Plan: Data management plans are more comprehensive than data sharing plans in that they include additional elements such as descriptions of the data to be produced in the proposed study, any standards to be used for collected data and metadata, mechanisms for providing access to and sharing of the data (including provisions for protection of privacy, confidentiality, security, intellectual property, or other rights), provisions for reuse and redistribution, and plans for archiving and long-term preservation of the data, or explaining why long-term preservation and access cannot be justified.
Final peer-reviewed manuscript: A final peer-reviewed manuscript is defined as an author's final manuscript of a peer-reviewed paper accepted for journal publication, including all modifications from the peer-review process.
Final published article: A final published article is defined as a publisher's authoritative copy of the paper, including all modifications from the publishing peer-review process, copy editing, stylistic edits, and formatting changes.
The AHRQ Public Access Policy will apply to the following groups:
6. Legal Authorities
7. Roles and Responsibilities
The Director of AHRQ:
AHRQ Director of the Office of Extramural Research, Education and Priority Populations (OEREP):
Grant, Cooperative Agreement, and Contract Awardees and their institutions:
Work with the publisher before any publication rights are transferred to ensure that all conditions of the AHRQ Public Access Policy can be met.
This Public Access Plan includes the following implementation steps.
AHRQ intends to establish an internal working groups/task forces with expertise in data access, data management and publications to assist with development of the update public access policy. The working groups/task forces will evaluate and determine the feasibility of implementing each of the recommendations as well as strategies of implementations.
AHRQ will continue to collaborate and cooperate with HHS and Federal Government-wide agencies to use a transparent process for soliciting views from stakeholders, including federally funded researchers, universities, libraries, publishers, users of federally funded researcher results, and civil society groups, and take such views into account.
AHRQ has contracted with National Library of Medicine (NLM) to use PubMed Central (PMC) since 2016 to make peer-reviewed scholarly publications publicly accessible. The current AHRQ Policy for Public Access to AHRQ funded Scientific Publications requires that AHRQ-funded authors submit an electronic version of the authors final peer-reviewed accepted manuscript to the NLM's PubMed Central (PMC) upon acceptance by the journal, and to be made publicly available within 12 months of the publishers date of publication.
PMC allows AHRQ to engage in public-private partnerships, since PMC is a public-private partnership. AHRQ's Public Access Policy is beneficial to businesses or will otherwise enhance innovation and competitiveness to satisfy a requirement in section 3 of the 2013 OSTP memo.
AHRQ plans to update its Policy for Public Access to AHRQ funded Scientific Publications (NOT-HS-16-008: https://grants.nih.gov/grants/guide/notice-files/not-hs-16-008.html ) to ensure, as appropriate and consistent with applicable law, that all peer-reviewed scholarly publications authored or co-authored by individuals or institutions resulting from federally funded research are made freely available and publicly accessible by default in agency-designated repositories without any embargo or delay after publication. In doing that, AHRQ will explore in close collaboration with NLM/PMC how to maximize equitable reach of public access to peer-reviewed scholarly publications, including by providing free online access to peer-reviewed scholarly publications in formats that allow for machine-readability and enabling broad accessibility through assistive devices, as appropriate and consistent with applicable law; and, the circumstances or prerequisites needed to make the publications freely and publicly available by default, including any use and re-use rights, and which restrictions, including attribution, may apply.
PMC is a public-private partnership to preserve and make public full-text journal articles in a nonproprietary and widely-distributed archival format. PMC makes the abstract and citation information available to NLM's abstract and metadata service, PubMed, where it can be accessed without charge. PubMed makes metadata available and combines them with any indexing terms or key works after the embargo period. Metadata of text are also stored in nonproprietary or widely distributed archival formats.
AHRQ will continue to maintain attribution to authors, journals, and original publishers through the use of PMC and PubMed. PubMed abstracts include a publisher-supplied link to the paper on a publisher site and a link to the PMC version if available. PubMed metadata, like the full text of PMC, are permanently archived. Full text displays of papers on PMC always include publisher-supplied links to the publisher site, authors, journal name and the other information required for a complete academic citation.
AHRQ will establish a mechanism for compliance with the public access policy.
AHRQ will continue to ensure the public can search, read, and download the final peer-reviewed manuscripts or published articles without charge and without embargo once established.
AHRQ will post the final updated Public Access Policy on the AHRQ website. AHRQ will also publish the final updated Public Access Policy in the NIH Guide for Grants and Contracts. In order to ensure full integration of the Public Access Policy with the AHRQ research and award management systems,
AHRQ will continue its partnership with the NIH to use PMC as the repository of publications resulting from AHRQ-funded grants, contracts, cooperative agreements, and intramural research.
AHRQ will explore and develop approaches and timelines for sharing other federally funded scientific data that are not associated with peer-reviewed scholarly publications wherever feasible.
AHRQ will explore the feasible approaches to accept and manage data submitted by extramural, intramural, and contract researchers, which may include contracting with a commercial repository.
AHRQ staff will coordinate with the researcher upon notification of funding, prior to the start of the research project, and throughout its life cycle to ensure the current usability, long-run preservation and access to the data. Data will be made available in a digital format free of charge to the public via the selected repository. AHRQ will publicize the location of the data to the public on its website and provide a link to the data. Information on how to access all AHRQ's funded data will be provided on AHRQ's website with links to the data.
AHRQ will continue working with OSTP and other federal agencies in developing guidance to researchers that ensures the digital repositories used align, to the extent practicable, with the National Science and Technology Council document entitled Desirable Characteristics of Data Repositories for Federally Funded Research.
The AHRQ Public Access Policy update and development process will include the following:
Continue and refine the iterative process of policy design, planning, implementation, evaluation/impact assessment, and policy redesign.
Work in full and open consultation with all stakeholders, including other Federal organizing bodies, to develop, maintain, and improve the Public Access Policy.
Explore new approaches and partnerships with publishers and other stakeholders to obtain final peer-reviewed manuscripts or published articles.
Maintain attribution to authors, journals, and original publishers through the use of PMC and PubMed. PubMed abstracts include a publisher-supplied link to the paper on a publisher site and a link to the PMC version if available. PubMed metadata, like the full text of PMC, are permanently archived. Full text displays of papers on PMC always include publisher-supplied links to the publisher site, authors, journal name and the other information required for a complete academic citation.
Establish a mechanism for compliance with the policy.
Ensure the public can search, read, and download the final peer-reviewed manuscripts or published articles without charge or embargo period.
AHRQ will post the final updated Public Access Policy on the AHRQ website. AHRQ will also publish the final updated Public Access Policy in the NIH Guide for Grants and Contracts.
AHRQ has included requirements to submit publications in accordance with the Public Access Policy in contract solicitations and grant Notices of Funding Opportunity and will update such requirements according to the updated policies. AHRQ will continue using PMC as an approach for optimizing search, archival, and dissemination features that encourage innovation in accessibility and interoperability, while ensuring long-term stewardship of the results of federally funded research. It will also continue encouraging public-private collaboration to maximize the impact of the federal research investment through the use of PMC. The significant usage of PMC underscores the value of public access to the peer-reviewed scientific literature.
The NLM databases, with their XML data accessible through a variety of Web services, can also support additional economic activity. NLM has found that more than 300 private sector firms obtain data from Medline for a variety of applications: to supplement literature database products and search tools; to build literature analysis tools for end users of research; to conduct research in a variety of biomedical areas; and to gain intelligence on experts and trends. Many others that we do not have the means to track use NLM's other resources.
Scientific and Research Integrity in Agency Public Access Policies
Public access policies that deliver transparent, open, secure, and free communication of federally funded research and activities in an expeditious manner are an important tool to uphold scientific and research integrity.
AHRQ will take actions to ensure that these elements of scientific and research integrity are in place in order to strengthen public trust in federally funded science. By December 31, 2024, AHRQ plans to submit to OSTP and OMB a second update to their public access plans specifying approaches taken to implement the provisions in Section 4 of the OSTP memo. AHRQ plans to complete and publish full policy development for plans implementing these provisions by December 31, 2026, with an effective date no later than one year after the publication of the agency plan.
In the plan to be submitted to OSTP and OMB by December 31, 2024, AHRQ will discuss how AHRQ plans to act to achieve the goals specified and approaches recommended in OSTP 2022 Memo.
Submission of Manuscripts/Publications
AHRQ has utilized PMC to enable the submission of final peer-reviewed manuscripts or final published articles that includes the following functionalities:
Access and Discoverability
The AHRQ has established and will continue an interagency agreement with NLM to use PMC for its primary repository for AHRQ publications. The use of PMC has ensured easy search, analysis, and download of peer-reviewed scholarly publications arising from research funded by AHRQ without charge no later than 12 months following publication. The public's ability to locate and access digital data resulting from AHRQ's federally funded scientific research as well as for publications will be improved through the use of PMC. The full text of PMC papers can be searched on PMC. In addition, the metadata (abstract, citation, and any index terms) can be searched in PubMed.
AHRQ will continue to ensure that the system is accessible to people with disabilities and compliant with Section 508a of the Rehabilitation Act (29 USC 794d). AHRQ will provide an accommodation for final publisher PDF versions that are not 508 compliant but cannot be remediated due to copyright constraints.
AHRQ's current maximum embargo period is 12 months. AHRQ will explore approaches to ensure all peer-reviewed scholarly publications authored or co-authored by individuals or institutions resulting from federally funded research are made freely available and publicly accessible by default in agency-designated repositories without any embargo or delay after publication, wherever feasible considering all applicable legal and regulatory constraints, in close collaborations with NIH PubMed.
Preservation is one of the Public Access Policy's primary objectives. As discussed in NIH's Public Access Policy, NLM maintains multiple backup copies of the PMC database, both onsite and at a remote, secure location.
Further, content on PMC is actively curated in that all article views are generated dynamically from the XML record of an article. In this way, every use of an article validates that the archival record is still usable and viable.
AHRQ will continue exploring feasible approaches to support a data repository to allow public access to data in digital format, which may include allowing data repositories identified and/or utilized by AHRQ funded researchers or recommending data repositories widely used by federally funded researchers. AHRQ will require all primary data collected by grant, contract, or intramural research to be submitted to a data repository, unless confidentiality restrictions prevent the data from being made public. For research using secondary data that is publicly available, such as survey or claims data, AHRQ is considering allowing researchers to submit command files, such as SAS or STATA files, to the AHRQ data repository. Researchers using public use data, such as the Medical Expenditure Panel Survey (MEPS) - a set of large-scale surveys that collect data on the cost and use of health care and health insurance coverage of families and individuals, their medical providers, and employers across the United States—or the Healthcare Cost and Utilization Project (HCUP), the largest collection of nationwide and State-specific longitudinal hospital care data in the United States, will be exempt from submitting these data to the data repository. The rationale for this exemption is that if the data are already publicly available, there is no value in making additional copies of the data available; what the public and other researchers need is the SAS or STATA code used to analyze the data. AHRQ-funded research that involves data use agreements or licensing agreements that prohibit data sharing will be exempt from submitting these data to the data repository.
AHRQ will maximize access, by the general public and without charge, to digitally formatted scientific data created with Federal funds by designing, implementing and maintaining a sustainable network of data management capabilities to enable discovery, appropriate use, and long-term management of digitally formatted scientific data.
This includes but is not limited to:
AHRQ will provide guidance to awardees and training, education, and workforce development for its employees about the new requirements related to scientific data management, analysis, storage, preservation, and stewardship in advance of the implementation date. This guidance and training will describe any required changes to the conditions of awards and a publicly available timeline for the implementation of the policy. AHRQ will also use public websites and internal mechanisms to disseminate information and solicit feedback before and after the implementation date. The capability for AHRQ and the science communities it supports to help one another through ongoing dialogue is expected to be a key part of a successful policy implementation.
AHRQ will modify existing policies or create new policies as necessary to incorporate the recommendations from the OSTP 2022 memo. AHRQ will include updated requirements to comply with the AHRQ Public Access Policy in contract solicitations and grant Notices of Funding Opportunity.
AHRQ will allow the inclusion of appropriate costs for data management and access in applications and proposals for AHRQ funding. The appropriateness of requested funding levels for data management plans will be assessed on a case-by-case basis through technical evaluation, and by program and grants management staff. Comparable assessments will need to be explored for intramural researchers. AHRQ will confer with interagency partners to estimate the costs of current data management activities in order to determine how to support future data management.
In order to ensure public access to data in digital format resulting from AHRQ-funded grants, contracts, and cooperative agreements AHRQ has required since 2020 all research applicants to include a data management plan as a component of their grant applications or contract proposals. Intramural researchers are also required to submit a data management plan, where applicable. The AHRQ Data Management Plan Policy will be updated to reflect the changes in response to the OSTP 2022 memo.
AHRQ will require periodic reporting on compliance with the approved data management plans as part of the terms and conditions of award and will develop mechanisms to monitor such compliance. The data management plan currently undergoes objective review and approval by AHRQ's grant and technical peer review groups to assess its merits. The data management plan has become part of the terms and conditions of award for all appropriate grants and contracts. AHRQ will continue to require all data management plans to be consistent with applicable laws and regulations governing the privacy and confidentiality of individual human data. AHRQ's Public Access Policy recognizes proprietary interests, business confidential information, and intellectual property rights and avoids significant negative impact on intellectual property rights, innovation, and U.S. competitiveness of research grants and cooperative agreements and appropriate contracts.
For requirements under AHRQ Data management plans, refer to AHRQ Data Management Plan Policy at https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-011.html.
AHRQ encourages grantees and contractors to register all clinical trials with ClinicalTrials.gov (whether or not they are subject to Title VIII of Food and Drug Administration Amendments Act (FDAAA)) and is in the process of developing a policy for intramural researchers to extend the registration requirement to all clinical trials. AHRQ will give further consideration to taking steps to expand submission of results to encompass all AHRQ-funded clinical trials, not only those subject to FDAAA.
In order to ensure long-term preservation and full access to the public, AHRQ will continue exploring feasible approaches to support a data repository to allow public access to data in digital format, which may include allowing use of data repositories identified and utilized by AHRQ funded researchers or providing guidance on recommended data repositories widely used by federally funded researchers. AHRQ staff will coordinate with the researcher upon notification of funding, prior to the start of the research project, and throughout its life cycle to ensure the current usability, long-run preservation and access to the data. Data will be made available in a digital format free of charge to the public.
Long-term preservation and sustainability are part of the current requirements of data management plans. Development of criteria for and conduct of periodic reviews to identify gaps in preservation coverage and respond to changing needs arising from new data types may become responsibilities of the Director of OEREP. In order to determine how best to develop and sustain repositories for digital scientific data, AHRQ intends to continue to collaborate with HHS operating divisions and other agencies that support research in related areas.
9. Metrics, Compliance, and Evaluation
AHRQ will continue to include the public access requirement in all grant and contract solicitations. Standard requirements are included in all research contracts and grant awards. All research proposals are required to include a Data Management Plan (DMP) as part of the overall research proposal. Approved DMPs are included in terms and conditions of grant notices of award and contract awards. Accordingly, failure to comply with the terms and conditions of the funding agreement could lead to enforcement actions, including the withholding of funding, consistent with 45 CFR 74.62. Researchers will be expected to follow the approved DMP, to the best of their ability. Failure to do so will negatively influence future funding opportunities. AHRQ will monitor all non-competing continuation grant awards using the electronic Research Performance Progress Reports (RPPR) to verify compliance with the public access policy. Current oversight of grants allows for withholding or adjustments of funds at the end of each performance period. Grant final reports will be used to confirm compliance with the Public Access Policy. Contract progress reports will also be monitored for compliance.
Prior to funding new grant and contract awards AHRQ will confirm awardees are in compliance with the AHRQ Public Access Policy. AHRQ will not make grant or contract awards in cases of non-compliance. AHRQ may delay processing of non-competing continuation awards that are not in compliance with the Public Access Policy.
Intramural research proposals are also required to include a DMP. AHRQ leadership will ensure intramural research compliance with the Public Access Policy.
AHRQ staff will periodically manually check applications, proposals, or reports for compliance with the Public Access Policy. They will confirm that a citation includes the appropriate reference number that indicates compliance, such as the PubMed Central Identifier (PMCID).
AHRQ will also use the NLM's Bibliography tool My NCBI. My NCBI tracks the public access compliance status of every paper associated with an award, generates a citation with the current identifier and clearly indicates its compliance status. It also allows awardees to collaborate with their colleagues to associate publications with AHRQ awards.
To simplify reporting and tracking, awardees will use My NCBI to report public access compliance on these awards. This process will be fully electronic for our Research Performance Progress Reports (RPPR).
AHRQ will report to OSTP, when requested, on the status of the public access plans and policy implementation, including relevant statistics collected by the agency, such as number of publications resulting from agency-supported research grants, contracts, and cooperative agreements.
Papers will be compliant when they are posted to PMC. This will form the numerator of an Agency-wide compliance measure. AHRQ's denominator for overall compliance will be determined by the number of papers AHRQ estimates fall under the policy. AHRQ will generate this number from papers linked to specific AHRQ awards by:
AHRQ's expectations for data sharing are and will continue to be communicated to intramural and extramural researchers through guidance documents and outreach activities, including through the notice of grant award or the contract award. AHRQ staff will be responsible for monitoring and taking steps to ensure compliance with the terms and conditions of the award. Data management plans that are approved and made a term and condition in the notice of grant award or the contract award can be enforced. Accordingly, failure to comply with the terms and conditions of the funding agreement could lead to enforcement actions, including the withholding of funding, consistent with 45 CFR 74.62.
Program staff overseeing extramural grants and contracts will also review progress reports carefully for adherence with data management plans. Under the intramural program, the AHRQ's Center Directors will be responsible for ensuring intramural researchers are in compliance with data management plans. AHRQ will conduct an annual audit of its intramural researchers to monitor overall compliance with data management.
Since 2016, AHRQ has required all intramural projects and new grant and contract research awards to submit manuscripts to PMC. Since 2020, AHRQ has required all intramural projects and new grant and contract research applications to submit data management plans. AHRQ will update its public access polices according to the deadlines recommended in sections 3 and 4 the OSTP 2022 memo.
AHRQ has used and will continue to use annual appropriation funds for the implementation of PMC and for the implementation of the data management plan policy.
12. Public Consultation
AHRQ will publish the revised draft public access plan for public input on the agency website and in the NIH Guide for Grants and Contracts. AHRQ will participate in any public consultations organized by OSTP and HHS, and will be responsive to feedbacks from such sessions and other channels.
13. Interagency Coordination
AHRQ will continue interagency coordination by the following:
14. Public Notice
The current AHRQ public access policies are posted on the AHRQ website and in the NIH Guide for Grants and Contracts (https://www.ahrq.gov/funding/policies/publicaccess/policy.html).
AHRQ will post the final updated Public Access Policy on the AHRQ website and in the NIH Guide for Grants and Contracts.
15. Update and Re-Evaluation of the Policy
AHRQ will conduct periodic updates on the final published Public Access Policy, which will be informed by feedback from all stakeholders. Feedback will be sought periodically through Requests for Information published in the NIH Guide for Grants and Contracts.
16. Additional Information
Additional public access policies and guidance include the following:
17. Material Superseded
Please direct all inquiries to:
Francis D. Chesley, Jr., M.D.
Director, Office of Extramural Research, Education, and Priority Populations