AHRQ Announces Interest in Innovative Methods Research to Increase the Utility of Systematic Reviews

Notice Number: NOT-HS-17-019

Key Dates
Release Date: August 10, 2017

Related Announcements


Issued by
Agency for Healthcare Research and Quality (http://www.ahrq.gov/)


The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable, and to work within the U.S. Department of Health and Human Services and with other partners to make sure that the evidence is understood and used. AHRQ achieves this mission by supporting evidence synthesis, translation, and implementation of evidence into practice.

This Special Emphasis Notice (SEN) supersedes NOT-HS-15-011 and NOT-HS-16-016. It informs the research community that AHRQ intends to support research designed to develop innovative systematic review methods that address the opportunities and challenges of the current era of data abundance, and make systematic reviews both more efficient and salient to improve the healthcare of Americans. Systematic reviews organize and combine individual studies into a more coherent body of evidence that can be assessed for quality and subsequently used to underpin clinical and healthcare delivery decisions, implementation of programs, health policies, quality indicators, patient decision aids, and other products that inform patient care. Innovative evidence synthesis methods may increase the speed and power of systematic reviews or improve utilization of systematic reviews by healthcare leaders to make evidence-based policy, organization, and programmatic decisions, increasing their value to healthcare systems and the patients that they serve.

Areas of Interest

This SEN is intended to generate new methods for systematic reviews that address the opportunities and challenges of the current era of data abundance and that facilitate the integration of systematic reviews into healthcare decisions. In the past, much of the work in systematic review methodology focused on developing basic evidence synthesis tools, especially reliable and standardized approaches to scoping, searching, assessing, synthesizing, and grading the evidence. There are now widely shared standards for how to conduct a high quality systematic review. The next challenge is developing novel or improved methods that optimize the efficiency, comprehensiveness, and predictive value of systematic reviews. Some potential areas of interest are described below:

Efficiency. Traditional systematic review approaches may not be sustainable given the ever increasing volume of studies and increasing demand for timely synthesized evidence. Faster techniques, without sacrificing validity and reliability, are increasingly valuable. Possible approaches include automation, incorporation of existing systematic reviews, predictors of marginal return, continuous updates and data reduction techniques.

New forms of data. Large data sets are currently being developed by various healthcare systems and organizations –individual patient-level data from randomized controlled trials and observational studies, patient registries, and systems-wide data on healthcare delivery. Tapping directly into this data could greatly enhance the power of systematic reviews and reduce the lag time between knowledge generation and implementation. Approaches include but are not limited to Bayesian stepwise analysis, informative priors, data linkage, subgroup analysis, etc. How best to incorporate qualitative data into systematic reviews would be of interest as well.

Predictive value. Systematic review methodology is highly detailed for processes, but much less developed regarding outcomes. Now that a substantial number of systematic reviews have been completed and indexed, it may be possible to empirically examine how accurately different reviews predict future research findings and to use this information to develop more robust methods. Managing the tradeoff between Type I and Type II error is of particular concern.

Complex interventions. Healthcare systems must decide how to organize and deliver care to improve quality and efficiency of healthcare for patients.  Understanding the most effective approach to delivering care is complicated by many factors within the health system and potentially affects strategies for their proposed implementation. AHRQ is interested in novel techniques or application of methods for synthesizing evidence for complex questions in systematic reviews so that they are useful to healthcare system leaders making decisions on how best to deliver healthcare, implement programs, or make policy decisions to improve patient care.  Examples include but are not limited to qualitative comparative analysis, finite mixture models, and frameworks for assessing programs or implementation studies.

Learning Health Systems.  The National Academy of Medicine (NAM), formerly the Institute of Medicine, defines a Learning Health System or Learning Healthcare System as "a system in which, science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience". AHRQ is interested in innovative methods that will help Learning Health Systems use evidence from systematic reviews to improve their care delivery and other processes. This may include improved methods for performing systematic reviews to answer questions of interest by Learning Health Systems, methods to integrate local data from Learning Health Systems and published studies, or methods to improve the relevance of evidence included in systematic reviews to Learning Health Systems.  For more information on Learning Health Systems, please read the Learning Health System Series by NAM at https://nam.edu/programs/value-science-driven-health-care/learning-health-system-series/.

Incorporating stakeholder perspectives. AHRQ is interested in determining the most effective way to incorporate the perspectives of various stakeholders during the systematic review development process.  Although AHRQ has particular interest in incorporating the perspectives of patients and patient advocates, stakeholders also include learning health systems, policymakers, guideline developers, etc. Studies which will identify types of activities and methods (e.g., primary and secondary data collection techniques), resources required, and the impact of including these perspectives into the systematic review are encouraged.

These are just a few areas of methodological innovation of interest to AHRQ. We welcome innovative applications addressing systematic review methods and issues not mentioned above. Applicants should describe the significance of their proposed research to enhancing the speed, power, predictive value, or otherwise advancing the utility of systematic reviews. To provide context in planned proposal submissions, applicants are highly encouraged to review information regarding the AHRQ Effective Health Care and Evidence-based Practice Center (EPC) programs using the following URLs:




Further Guidance

Priority Populations. Systematic review methods may be applied to primary research involving all populations regardless of age, gender, ethnicity, or socio-ethnic backgrounds. However, some populations are often underrepresented in clinical trials and therefore methods that address the issues discussed above in the context of priority populations are areas of interest to AHRQ. For purposes of this SEN, priority populations include low-income patients, the under- and uninsured, children, women, elderly, racial and ethnic minorities, and individuals with special health care needs.

Use of Funding Mechanism. AHRQ will use the standing R03 grant mechanism to support applications submitted in response to this notice. The standing R03 FOA, PA-15-147, can be found at https://grants.nih.gov/grants/guide/pa-files/PA-15-147.html.

Application Submission. Applicants should consider this SEN active for six award cycles. These cycles have due dates on October 16, 2017, February 16, 2018, June 16, 2018, October 16, 2018, February 16, 2019, and June 16, 2019. AHRQ will provide updates of new developments and research priorities as research budget information becomes available. Applications will be reviewed by AHRQ's standing study sections. We strongly encourage applicants to indicate in their applications that they are responding to this notice by including the title and number of this SEN (NOT-HS-17-013) in their Specific Aims page.


Please direct all inquiries to:

Lionel L. Bañez, M.D.
Agency for Healthcare Research and Quality (AHRQ)
Center for Evidence and Practice Improvement
Telephone: 301-427-1514
Email: lionel.banez@ahrq.hhs.gov