Required Application Instructions

It is critical that applicants follow the instructions in the PHS 398 Application Guide except where instructed to do otherwise (in this FOA or in a Notice from the NIH Guide for Grants and Contracts). Conformance to all requirements (both in the Application Guide and the FOA) is required and strictly enforced. While some links are provided, applicants must read and follow all application instructions in the Application Guide as well as any program-specific instructions noted in Section IV. When the program-specific instructions deviate from those in the Application Guide, follow the program-specific instructions. Applications that do not comply with these instructions may be delayed or not accepted for review.

Looking ahead: AHRQ/NIH are committed to transitioning all grant programs to electronic submission using the SF424 Research and Related (R&R) format and they are currently investigating solutions that will accommodate multi-project programs. See NOT-OD-13-075 and NIH’s Applying Electronically website for more information.

Part 1. Overview Information
Part 2. Full Text of Announcement
Section I. Funding Opportunity Description
Section II. Award Information
Section III. Eligibility Information
Section IV. Application and Submission Information
Section V. Application Review Information
Section VI. Award Administration Information
Section VII. Agency Contacts
Section VIII. Other Information

This FOA broadly invites applications to become a Center of Excellence on Comparative Health System Performance as part of AHRQ's Patient-Centered Outcomes Research (PCOR) dissemination work under Section 937(a) of the Affordable Care Act. PCOR is comparative clinical effectiveness research of the impact on health outcomes of two or more preventive, diagnostic, treatment, or healthcare delivery approaches. This definition is based on the definition used in section 6301(a) of the Patient Protection and Affordable Care Act (ACA) of 2010 (http://housedocs.house.gov/energycommerce/ppacacon.pdf). PCOR produces not only clinical findings (e.g., diuretics are superior to ACE inhibitors and calcium channel blockers in treating high blood pressure and preventing cardiovascular events), but also evidence about the effectiveness of different systems for delivering care.

Physicians and other providers increasingly work in delivery systems with new types of complex and virtual arrangements, incentives, and ownership structures that we do not yet fully understand. It is not clear how to effectively target PCOR and other comparative effectiveness research dissemination efforts to physicians or to delivery systems. AHRQ seeks to award cooperative agreements to up to three Centers of Excellence to address this problem. Each Center will carry out two related sets of activities:

(1) Each Center will create a Data Core that identifies and classifies delivery systems and includes data on their characteristics and their performance.

(2) Each Center will conduct up to six major inter-related projects, to be conducted starting in year two and to be completed over the five year project period, using data from the Data Core, that compare the performance of different types of health care systems. These projects will seek to identify, characterize, classify, and track the proliferating forms of health care systems, will assess the extent to which they use PCOR evidence, will assess the association between use of PCOR evidence and quality and cost of care, will seek to understand the characteristics of high performing systems, and will disseminate this information to system leadership, patients, policymakers, and payors. These projects will greatly improve the nation's understanding of best methods for disseminating PCOR to health care systems. Centers should plan and schedule their work so that they can expect to report some findings in the early years of the grant.

By directly addressing the impact of these large, growing, and evolving system components on PCOR dissemination and system performance, these Centers of Excellence will complement the work of another AHRQ initiative (Accelerating the Dissemination and Implementation of PCOR Findings into Primary Care Practice, RFA-HS-14-009) which focuses on improving PCOR dissemination within individual or small physician practices.

AHRQ seeks to improve the efficiency and value of U.S. health care by understanding how high performing health care systems quickly diffuse evidence-based medicine into practice to improve outcomes and reduce costs. AHRQ has had a long tradition of developing evidence-based medicine and helping to disseminate PCOR. However, many important PCOR findings still are put into practice very slowly, resulting in both underutilization and overutilization of health care. Examining 439 quality indicators, McGlynn (2003) found that Americans received only 55 percent of the recommended processes of care when they were treated. At the same time there is substantial overutilization: Cutler, Skinner, Stern, and Wennberg (2013) estimate that 36 percent of end-of-life spending and 17 percent of U.S. health care spending are associated with a segment of physicians that provide treatment beyond that which is supported by clinical evidence. Timbie et al (2012) document many recent cases of major PCOR findings still not being put into practice.

PCOR findings are only valuable if they are used by organizations and by the physicians and staff within these organizations (Masica and Ballard, 2009). AHRQ seeks to understand this phenomenon and to improve the uptake of PCOR findings among organizations and their clinicians. As a start, in 2010, AHRQ and NIH joined forces to examine the effectiveness of behavioral economic incentives and other ideas from nudge theory for encouraging clinicians to use PCOR findings. This research was conducted through randomized clinical trials (RCTs) under RFA-OD-10-001 and RFA-OD-10-002 (Behavioral Economics for Nudging the Implementation of Comparative Effectiveness Research). Also, in 2010, AHRQ funded the Comparative Effectiveness Delivery System Evaluation and Demonstration Grants under RFA-HS-10-012 and RFA-HS-10-013. However, these grants focused primarily on evaluating within-organizational processes, rather than the inter-relationships among organizations within the system (Casalino, 2014). AHRQ realizes that the real world is much more complex than RCTs, and that most clinicians work not just within organizations, but within complex delivery systems where there are complex interactions between organizations. Today many new forms of these delivery systems are quickly emerging, such as Accountable Care Organizations (ACOs) (Fisher, Staiger, Bynum, and Gottlieb, 2006), diverse Integrated Care Models, interconnected health IT networks, medical homes, etc.

Public policy issues dealing with systems of healthcare delivery have long been a focus of AHRQ (Eisenberg, 1998). For many decades, health care delivery remained a cottage industry, with 7,000 independent hospitals and more than a half-million doctors, many self-employed or practicing in small medical groups. However, by the mid-1990s health care delivery became significantly more consolidated, and many integrated delivery systems emerged. AHRQ research focused on these critical market-driven transformations of the delivery system in the 1990’s under RFA HS-95-005 (Market Forces in a Changing Health Care System). However, eventually consumer backlash and financial risks caused many of these early integrated systems, pure capitation incentives, and physician consolidations (such as Physician Practice Management companies and Physician Service Organizations) to unravel. This resulted in vertical dis-integration a shift from ownership to contractual relationships between providers and plans (Robinson, 1999). AHRQ research examined this restructuring in the delivery system under RFA HS-00-001 (AHRQ Centers of Excellence on Health Care Markets and Managed Care). In 2014, we see a major resurgence in the reforming, integration, and consolidation of health care delivery (Burns, Goldsmith and Sen, 2013). What is new is the sheer complexity of the current transformation, and the challenges this presents to successful dissemination and uptake of PCOR.

1. Many delivery systems are now an intertwining network of several different types of systems. A typical delivery system today is a clinical system tied to one or more systems of health IT networks with varying levels of inter-connectedness, while integrating not only a system of physician and hospital networks, but combining many sub-systems of patient-centered medical homes of varying complexity in care management. These new systems and their physicians are difficult to identify and track.

2. Physicians are often now tied to many of these systems in new types of complex and virtual arrangements, incentives, and ownership structures that we do not yet fully understand. Moreover, incentives often are combined, such as integrated care models involving bundled payments with varying levels of shared-savings, combined with a pay-for-performance incentive. Financial risks are now being shifted down to the provider level instead of the health plan and insurer levels.

3. The insurance market is transforming. New systems such as ACOs, which have traditionally lacked the actuarial capacity to manage financial risks, are now purchasing insurers to gain this capability. At the other end, insurers are purchasing managed care companies in order to build integrated delivery systems (Emanuel, 2014).

4. Delivery system reform is being catalyzed by the actions of both private and public payers actions that are changing the way that providers are organized and the incentives that providers are offered. The far-reaching federal reforms of the HITECH and the Affordable Care Act appear to have accelerated many changes in delivery systems. Examples include the Medicare Shared Savings Program, the Pioneer Accountable Care Organization Model, the Bundled Payments for Care Improvement initiative, the Hospital Readmissions Reduction Program, the Hospital Value-Based Purchasing Program, the Comprehensive Primary Care Initiative, and the State Innovation Models Initiative. States also have begun to exert an influence, for example, through legislation on delivery systems in the Medicaid program, and through public reports on quality and cost (For example, see http://www.commonwealthfund.org/Topics/State-Health-Policy-and-Medicaid.aspx).

This FOA seeks applications to improve the nation's understanding of how these new systems are organized, to assess dynamics in the uptake of PCOR findings among clinicians in systems, and to compare the performance of these systems to each other and to the performance of more traditional forms of delivery system.

The goal is to provide a better understanding of health care delivery systems in order to better target PCOR dissemination so that systems can more quickly adopt high-performance practices and improve patient outcomes. Thus, the primary objective of this program is to identify, characterize, and classify types of delivery systems and to compare the performance (quality and resource use) of health care delivery systems that are responsible for a broad spectrum of the patients care. To accomplish this objective, each Center of Excellence on Comparative Health System Performance will first create a Data Core to collect and develop data on systems over a five year period. Next, each Center will include up to six inter-related projects which use the data of the Data Center to compare systems. AHRQ expects to fund up to three Centers.

Systems that are Eligible to be Included in Project

Eligible systems will include physicians and hospitals, formally or informally connected to each other, and may also include formal or informal connections with other types of providers such as post-acute care facilities and home health agencies.

Formal connections between providers can occur via ownership or via contract. For example, a system based on ownership might include:

1. one or more hospitals and the physicians whom they employ
2. a health insurance company and the physicians whom it employs
3. a medical group and the hospital or hospitals it owns
4. one or more hospitals, the physicians they employ, and the rehabilitation facilities owned by the hospitals.

A system based on contracts might include, for example:

1. one or more hospitals and one or more physician-hospital organizations (PHOs) with which the hospitals contract
   
   
2. a medical group or independent practice association (IPA) and the hospital or hospitals with which the medical group or IPA contracts to operate as an accountable care organization (ACO)

Informal connections among providers may also create virtual systems of care. For example, informal systems might include:

1. a hospital and its voluntary medical staff (physicians who admit patients to the hospital but are not employed by the hospital)
   
   
2. a medical group or IPA that does not formally contract with a hospital, but that cooperates with the hospital through informal agreements on ways to improve patients care

The examples given above are meant to be illustrative, not fully inclusive; responses to this FOA may describe other system forms that are proposed for study.

System Comparisons that are Eligible under this Project

Both the structure of health care delivery systems and the processes that they use to improve the quality and control the costs of the care they provide are likely to affect systems performance. Furthermore, health care delivery systems do not operate in isolation. Applications will be viewed as responsive to this FOA to the extent that they include comparative analyses of system performance that include not only types of systems, but also the incentives systems receive, the integration of PCOR evidence into systems processes of care, the relationship between use of PCOR evidence and the quality and costs of care the systems provide, and the environment in which systems are located. Applications that focus on only one of these areas for example, applications that compare different pay-for-performance programs without regard to delivery systems, or applications that compare different processes for reducing readmissions, without regard to the delivery systems within which these processes are embedded, will not be considered responsive to this FOA and will not undergo peer review.

Applicants should explain how they will assess system performance across a broad spectrum of care. Some examples of performance measures are: the total cost of care for a population of patients; ambulatory care sensitive admissions; readmissions; performance on a broad range of quality measures (such as those used in the Medicare ACO programs); patient experience; and a broad range of patient-reported outcomes. This list is intended to provide examples; applications may suggest other measures appropriate to comparing performance across the full spectrum of care. Applications that focus only on a narrow area of care such as outcomes after joint replacement surgery will not be considered responsive to this FOA; however, comparative performance on narrow areas of care can be eligible under this FOA if the comparisons are included in a broader assessment of system performance. For example, some systems might have lower overall costs of care for their patients because they provide better orthopedic care, or better cardiac care.

Applications that propose to compare the performance of single types of organization that are not systems and cannot be held responsible for performance across a broad spectrum of care (for example, retail clinics, or nursing homes) will not be considered responsive to this FOA and will not undergo peer review. However, applications that compare systems performance across a broad spectrum of care and seek, as a project within the application, to show that a major component of better performance is due, for example, to the nursing homes within the system, would be responsive under this FOA.

Applications may compare performance both within classes of delivery systems and between classes of delivery systems. Centers may propose projects that would do one or the other or both of these types of comparison. Comparing the performance of systems may include, for example, some or all of the following types of projects:

1. comparing the performance of individual delivery systems to other delivery systems:

a) within the same class of delivery system

b) across different classes of delivery system

2. comparing the performance of one class of delivery system to one or more other classes of delivery system

3. comparing the change in performance over time of an individual delivery system or class of delivery systems to the status quo that is, the change over time in performance for a population of patients not attributed to the delivery system or class of delivery systems, but arguably comparable to the population of patients attributed to the delivery system or class of delivery systems.

Data Core

Each Center will have a Data Core, staffed by personnel with the appropriate background and experience in health delivery system data, measurement, and research methodologies, to develop data for the Center's projects, allowing projects to link datasets and share variables. AHRQ anticipates that a substantial portion of grant resources will be devoted to the Data Core. Greater amounts might be proposed, for example, for projects that envision substantial amounts of new data collection, or very high costs for creating analytical files. Smaller amounts might be proposed, for example, for projects in which a substantial amount of the data has already been processed and analyzed.

Under the cooperative agreement, grantees will be expected to share analytic results with a Coordinating Center contractor for purposes of creating a compendium of information that can be used by the nation to inform PCOR dissemination strategies, so that systems can more quickly adopt high-performance practices and improve patient outcomes. The systems compendium will be put together jointly by the grantees and the Coordinating Center. For purposes of the collaborative research, the systems compendium will list systems by name and include identifiers e.g., tax identification numbers (TINs), hospital identifier (AHA id, Medicare provider number, etc.), and physicians national provider identifiers (NPIs) as appropriate - with outcome measures and performance of the system listed. Using the systems compendium, the grantees will collaborate with the Coordinating Center to develop short Policy Briefs and Data Briefs on comparing systems, and to develop other PCOR dissemination initiatives.

Applications should describe how Data Centers will be skilled in acquiring and cleaning data, and in using this data to conduct analyses that convincingly compare health delivery system performance.

1. The Data Core will define, identify, characterize, and classify the various types of delivery systems nationally, regionally, within a state, and/or within one or more communities. Development of a taxonomy of systems would be highly desirable.

2. Information on systems performance may be obtained, for example, from Medicare, Medicaid, or private sector claims data, from the delivery systems themselves, and/or it may be collected as part of the Center’s inter-related projects as a primary data collection effort. Applicants should explicitly state why they will or will not risk-adjust performance data for patient demographics (possibly including socioeconomic status), patient clinical status, and/or other factors, and how they will risk adjust.

3. As the inter-related projects progress, more information on the characteristics of the systems should be obtained (possibly through primary data collection), possibly including topics such as use of PCOR evidence, ownership structures, culture, workforce, methods of compensation of physicians; sub-systems within the system (e.g., medical homes, health IT systems); type and volume of quality improvement activities, etc.

4. The Data Core should also include methodological work on how to statistically compare systems in terms of their characteristics and performance. Endogeneity issues, such as when certain providers self-select into systems, should be addressed, as well as risk-adjustment methods for outcomes, so that findings used to inform PCOR dissemination are not overly biased.

Inter-related Center Projects

The goal of the inter-related projects supported under this FOA is generation of knowledge that can be used to better understand how clinicians function within systems and outside of systems so that the uptake of evidence-based medicine by clinicians and the diffusion of PCOR findings within systems can be advanced at a greater rate. What are the characteristics of systems, how can one best measure and compare systems in terms of their use of PCOR findings, and what is the impact of systems characteristics and of the extent of their use of PCOR findings on quality and cost?

Projects should address the following questions: What kinds of systems, using what types of processes, with what types of incentives, and in what kind of environments or markets, speed up the diffusion of PCOR findings into practice to produce the best patient-centered outcomes?

To analyze these issues, each Center of Excellence will conduct up to six (6) inter-related individual projects. Applications should explain how these projects will have a major impact on health care in the United States, individually and as a group. Projects that compare the performance of a large number of systems across a large geographic area are within the scope of this FOA. For example, some projects might propose to place all hospitals and physicians in the country into a virtual system (while recognizing that some providers are not part of any system), and analyze all or most of those systems. For another example, some projects may propose to analyze the hundreds of Accountable Care Organizations that have been created. However, there may also be much to be learned from analyses focused on carefully selected smaller geographic areas. Analyses that develop more detailed knowledge about a smaller number of systems using surveys, interviews, and/or other methods may also be useful.

For example, how do these systems succeed in incorporating PCOR-based evidence into the provision of care? How compared to other systems do they achieve superior performance? If a system or class of systems has a lower overall cost of care, does this result from lower costs across the board, or from something special that the systems do -- for example, special efforts in the area of cardiac care or orthopedic care.

Projects may, for example, address questions similar to the following:

(1) Systems. What are the key characteristics of systems? How should systems be classified? What is the prevalence of different types of system? What are the different ways that systems include physicians? To what extent do systems include multiple components of the delivery system e.g., hospitals, physician groups, home health agencies, and post-acute facilities? How and to what extent do systems that include only some of these components coordinate care among components? In what ways are public health programs or non-medical services included in systems? How can integration be measured, and to what extent are systems integrated? What types of systems are most integrated?

(2) Processes to Improve Care. To what extent are systems using PCOR-based processes? What is the cause of variation in the adoption rates of PCOR-based processes (see, for example, Skinner and Staiger (2009) on tiger vs. tortoise hospitals). What types of processes do systems use to improve the care they provide? For example, what types of processes do they use to reduce hospital readmissions? To what extent do systems use nurse care managers to help patients with chronic illnesses? To what extent do they maintain disease registries? Provide decision support at the point of care for clinicians? Use the types of processes required for recognition as a patient-centered medical home? Use processes shown to be effective at reducing the rate of hospital-acquired infections?

(3) Incentives. What types of external incentives do policymakers and public and private payors provide for systems, and what is the impact of these incentives on the types or systems that are created, the processes they use, and the outcomes of the care they provide? Examples of incentives are pay for performance, public reporting of performance, private reports on performance, bundled payments, and sharing financial risks for the cost of care within delivery systems. Incentives may also include regulatory restrictions such as anti-trust enforcement or Stark regulations. What types of internal incentives do systems provide for their physicians and other providers? What is the impact of these internal incentives on the adoption of PCOR findings and on patient-centered outcomes? For example, what is the role of internal recognition of clinicians, organizational culture, and professionally-inspired incentives such as the Choosing Wisely effort? How are externally-derived financial incentives distributed within the system? What is the variation in incentive structures within systems?

(4) Environments. What environmental or market factors affect system performance and in what ways? Are particular system forms, processes, and incentives likely to be more successful in some regions (for example, traditionally low-utilization regions) than others? In urban areas versus rural areas? In areas of high market competition versus low market competition? Is system consolidation reducing market competition and raising prices? Are there system factors, processes, or incentives that are particularly better in promoting evidence-based medicine among the poor?

(5) Outcomes. What are the most significant outcomes to look for in order to compare systems, and how might we measure these? What patient-centered outcomes are best for evaluating the uptake of processes or treatments that PCOR has determined are effective in improving the quality of the care provided? What types of systems are best at reducing disparities in quality, patient experience, and patient-centered outcomes, especially among the poor? Attention should be given to quality and resource use, with particular focus on the cost-quality tradeoff, and to possible cost-offsets under integration, such as increased pharmacy costs that may contribute to decreased inpatient costs. For systems or classes of systems with sufficiently large patient populations, broader outcomes might be considered, such as avoidable deaths (see Nolte and McKee, 2008).

Projects should be aimed at comparing the outcomes of different types of systems, processes, incentives, and environments ideally analyzing all four together. Projects that simply evaluate the effectiveness of different types of pay for performance program, for example, without taking into account the types of system or types of process used, would be considered not responsive to the objectives of this FOA.

Coordinating Center

AHRQ plans to fund, through a contract mechanism, a Coordinating Center for the Centers of Excellence. The Coordinating Center will conduct activities such as convening a Technical Expert Panel (TEP), leading conversations across Centers of Excellence to harmonize methods and measures where possible and to foster cross-Center collaborations, organizing dissemination activities, and writing data and policy briefs.The Coordinating Center will also work with the Centers of Excellence to develop a system compendium of information, which will be shared among all participants in order to foster this collaborative work and to inform summative findings about the impact of system characteristics on successful PCOR dissemination and use. The grantees should be prepared early on in their grant to present methods and early findings to the TEP, and to publish early findings.

References

Burns, LR., JC. Goldsmith, A. Sen. 2013. Horizontal and Vertical Integration of Physicians: A Tale of Two Tails. Annual Review of Health Care Management: Revisiting the Evolution of Health Systems Organization. Advances in Health Care Management, Volume 15, 39-143.

Casalino, Lawrence P. Identifying Key Areas for Delivery System Research. Rockville, MD: Agency for Healthcare Research and Quality; 2014. AHRQ Publication No. 14-0024EF. http://www.ahrq.gov/professionals/systems/system/delivery-system-initiative/casalino/paper/casalino_idkeydsr.pdf

Cutler, D., J. Skinner, AD. Stern, D. Wennberg. Physician Beliefs and Patient Preferences: A New Look at Regional Variation in Health Care Spending. NBER Working Paper No. 19320 Cambridge, MA, August 2013

Eisenberg, John M. Health Services Research In A Market-Oriented Health Care System. Health Affairs. January/February, 1998. 17(1): 97-108.

Emanuel, Ezekiel. 2014. Reinventing American Health Care. Public Affairs: New York, NY. Pages 318-323.

Fisher, E., D. Staiger, J. Bynum, and D. Gottlieb. Creating Accountable Care Organizations: The Extended Hospital Medical Staff: A New Approach to Organizing Care and Ensuring Accountability. Health Affairs. 2007. 26(1): w44-w57.

Masica, AL., DJ. Ballard. The Protean Role of Health Care Delivery Organizations in Comparative Effectiveness Research. Mayo Clinic Proceedings. 2009. 84(12): 1062-1064.

McGlynn EA., et al. The Quality of Health Care Delivered to Adults in the United States. New Engl J Med. 2003. 348(26): 2635-2645.

Nolte, E., and C. McKee. Measuring The Health Of Nations: Updating An Earlier Analysis. Health Affairs, 27, no.1 (2008):58-71.

Robinson, JC. The Future of Managed Care Organization. Health Affairs. March/April, 1999. 18(2): 7-24.

Skinner. J., and D. Staiger. Technology Diffusion and Productivity Growth in Health Care

NBER Working Paper No. 14865. April 2009.

Timbie, JW., DS. Fox, KV. Busum, and EC. Schneider. Five Reasons That Many Comparative Effectiveness Studies Fail to Change Patient Care and Clinical Practice. Health Affairs. 2012. 31(10): 2168-2175.

42 U.S.C. 299b-37(a) - (c) authorizes AHRQ to make these awards to support the agency's dissemination of comparative clinical effectiveness research findings.

All applications submitted and AHRQ grants made in response to this FOA are subject to AHRQ's grant regulations at 42 CFR Part 67, Subpart A, the HHS Grants Policy Statement (see http://www.hhs.gov/asfr/ogapa/aboutog/grantsnet.html and http://www.hhs.gov/asfr/ogapa/aboutog/hhsgps107.pdf), and the terms and conditions set forth in the Notice of Award.

You may submit an application(s) if your institution/organization is a (an):

• Public or non-profit private institution, such as a university, college, or a faith-based or community-based organization;
• For-profit private institution;
• Units of local or State government;
• Eligible agency of the Federal government.
• Indian/Native American Tribal Government (Federally recognized)
• Indian/Native American Tribal Government (Other than Federally recognized);
• Indian/Native American Tribally Designated Organization.

Because the purpose of this program is to improve healthcare in the United States, foreign institutions may participate in projects as members of consortia or as subcontractors only. Applications submitted by foreign institutions will not be reviewed. Organizations described in section 501(c) 4 of the Internal Revenue Code that engage in lobbying are not eligible.

HHS grants policy requires that the grant recipient perform a substantive role in the conduct of the planned project or program activity and not merely serve as a conduit of funds to another party or parties. If consortium/contractual activities represent a significant portion of the overall project, the applicant must justify why the applicant organization, rather than the party(s) performing this portion of the overall project, should be the grantee and what substantive role the applicant organization will play. Justification can be provided in the Specific Aims or Research Strategy section of the PHS398 Research Plan Component sections of the SF424 (R&R) application. There is no budget allocation guideline for determining substantial involvement; determination of substantial involvement is based on a review of the primary project activities for which grant support is provided and the organization(s) that will be performing those activities.

Non-domestic (non-U.S.) Entities (Foreign Institutions) are not eligible to apply.
Non-domestic (non-U.S.) components of U.S. Organizations are not eligible to apply.

Applicant Organizations

Applicant organizations must complete and maintain the following registrations as described in the PHS 398 Application Guide to be eligible to apply for or receive an award. All registrations must be completed prior to the application being submitted. Registration can take 6 weeks or more, so applicants should begin the registration process as soon as possible. The NIH Policy on Late Submission of Grant Applications states that failure to complete registrations in advance of a due date is not a valid reason for a late submission.

• Dun and Bradstreet Universal Numbering System (DUNS) - All registrations require that applicants be issued a DUNS number. After obtaining a DUNS number, applicants can begin both SAM and eRA Commons registrations. The same DUNS number must be used for all registrations, as well as on the grant application.
• System for Award Management (SAM) (formerly CCR) Applicants must complete and maintain an active registration, which requires renewal at least annually. The renewal process may require as much time as the initial registration. SAM registration includes the assignment of a Commercial and Government Entity (CAGE) Code for domestic organizations which have not already been assigned a CAGE Code.
• eRA Commons - Applicants must have an active DUNS number and SAM registration in order to complete the eRA Commons registration. Organizations can register with the eRA Commons as they are working through their SAM or Grants.gov registration. eRA Commons requires organizations to identify at least one Signing Official (SO) and at least one Program Director/Principal Investigator (PD/PI) account in order to submit an application.

Program Director/Principal Investigator (PD/PI)

The PD/PI must have an eRA Commons account. The PD/PI should work with his/err organizational officials to either create a new account or to affiliate their existing account with the applicant organization in eRA Commons. If the PD/PI is also the organizational Signing Official, he/she must have two distinct eRA Commons accounts, one for each role. Obtaining an eRA Commons account can take up to 2 weeks.

Any individual(s) with the skills, knowledge, and resources necessary to carry out the proposed research as the Program Director/Principal Investigator (PD/PI) is invited to work with his/her organization to develop an application for support. Individuals from underrepresented racial and ethnic groups as well as individuals with disabilities are always encouraged to apply for AHRQ support.

The PD/PI is required to devote 15% annual effort in each year of the project.

Only one PD/PI may be designated on the application.

This FOA does not require cost sharing.

While there is no cost sharing requirement included in this FOA, AHRQ welcomes applicant institutions, including any collaborating institutions, to devote resources to this effort. An indication of institutional support from the applicant and its collaborators indicates a greater potential of success and sustainability of the project. Examples of institutional support would include: donated equipment and space, institutional funded staff time and effort, or other resource investments. Applicant institutions should indicate institutional support by outlining the specific contributions to the project and providing assurances that their organization and any collaborators are committed to providing these funds and resources to the project. This information can be included at the end of the budget justification section of the application, but institutional support dollars are not to be shown/included in the detailed budget request.

Applicant organizations may submit more than one application, provided that each application is scientifically distinct.

AHRQ will not accept any application that is essentially the same as one already reviewed within the past thirty-seven months, except for submission:

• To an RFA of an application that was submitted previously as an investigator-initiated application but not paid;
• Of an investigator-initiated application that was originally submitted to an RFA but not paid; or
• Of an application with a changed grant activity code.

Applicants are required to prepare applications according to the current PHS 398 application forms in accordance with the PHS 398 Application Guide.

It is critical that applicants follow the instructions in the PHS 398 Application Guide, except where instructed in this funding opportunity announcement to do otherwise. Conformance to the requirements in the Application Guide is required and strictly enforced. Applications that are out of compliance with these instructions may be delayed or not accepted for review.

Although a letter of intent is not required, is not binding, and does not enter into the review of a subsequent application, the information that it contains allows AHRQ staff to estimate the potential review workload and plan the review.

By the date listed in Part 1. Overview Information, prospective applicants are asked to submit a letter of intent that includes the following information:

• Descriptive title of proposed activity
• Name(s) of the PD/PI
• Names of other key personnel
• Participating institution(s)

The letter of intent should be sent electronically to:

Geralyn (Geri) Goins
Email: [email protected]

AHRQ encourages applicants to take advantage of a technical assistance conference call sponsored by AHRQ program staff. The purpose of the conference call is to provide potential applicants with background information and respond to questions about the preparation of an application in response to this FOA. The conference call will take place on August 22, 2014 at 2:00 p.m. Eastern time. To register to participate in the conference call, please send an e-mail request to [email protected] by August 19, 2014. All registrants will be sent an e-mail reply which will contain information on the call, including call-in information, by COB August 21, 2014. Potential applicants are encouraged to contact AHRQ staff with any questions (see Section VII, Agency Contacts, below).

The conference call is open to any organization intending to apply. Participation is not a prerequisite to applying. Participants are encouraged to submit questions via email prior to the conference call. Please submit up to five questions with your name and the name of your institution to [email protected] by August 19, 2014. Please include the phrase HEALTH CARE SYSTEMS FOA QUESTIONS in the subject line. Questions of a similar topic and nature may be grouped at the sole discretion of AHRQ staff. Notes from the conference call will be posted on the AHRQ web site.

Applications must be prepared using the PHS 398 research grant application forms and instructions for preparing a research grant application. Submit a signed, typewritten original of the application, including the checklist, and three signed photocopies in one package to:

Center for Scientific Review
National Institutes of Health
6701 Rockledge Drive, Room 1040, MSC 7710
Bethesda, MD 20892-7710 (U.S. Postal Service Express or regular mail)
Bethesda, MD 20817 (for express/courier service; non-USPS service)

At the time of submission, two additional paper copies of the application and all copies of the Appendix files must be sent to:

Geralyn (Geri) Goins, M.S.M.
Grants Manager
Agency for Healthcare Research and Quality (AHRQ)
540 Gaither Road
Center for Delivery, Organization, and Markets (CDOM)
Rockville, MD 20850
Telephone: (301)427-1419
Email: [email protected]

All page limitations described in the PHS 398 Application Guide and the Table of Page Limits must be followed unless otherwise stated in this FOA.

The following page limitations apply to the Research Strategy section of each component of the application.

• Data Core: 12 pages
• Inter-related Projects: 5 pages per project

The following section supplements the instructions found in the PHS 398 Application Guide, and should be used for preparing a multi-component application.

NOTE: AHRQ is not using the Modular Grant Application and Award Process. Applicants for funding from AHRQ should ignore application instructions concerning the Modular Grant Application and Award Process, and prepare applications using instructions for the Research and Related Budget Components of the PHS 398 Application Guide. Applications submitted in the Modular format will not be reviewed.

The application must consist of the following components:

• Overall
• Data Core
• Projects (maximum of 6 projects)

All instructions in the PHS398 Application Guide must be followed, with the following additional instructions, as noted.

Face Page (Overall)

All instructions in the PHS 398 Application Guide must be followed.

Description, Project/Performance Sites, Senior/Key Personnel, Other Significant Contributors, (Overall)

All instructions in the PHS 398 Application Guide must be followed.

Table of Contents (Overall)

All instructions in the PHS 398 Application Guide must be followed.

Detailed Budget for Initial Budget Period (Overall/Composite)

Do not use Form Page 4 of PHS Form 398 to create the composite budget. Instead, using the suggested format and example presented below, prepare a Composite Budget reflecting direct costs for All Proposed Years of Support. Note that direct costs INCLUDE any third party indirect costs. (Justification for budget elements should not be presented here, but in the individual budgets of the projects and cores.)

Component	Year 1 Direct Costs	Year 2 Direct Costs	Year 3 Direct Costs	Year 4 Direct Costs	Year 5 Direct Costs	Total All Yrs Direct Costs
Data Core	$1,100,000	$1,122,000	$1,346,400	$1,373,328	$1,400,795	$ 6,342,523
Project 1 (name)	$ 340,000	$ 346,800	$ 353,736	$ 360,811	$ 368,027	$ 1,769,374
Project 2 (name)	$ 300,000	$ 306,000	$ 312,120	$ 318,362	$ 324,730	$ 1,561,212
Project 3 (name)	$ 405,000	$ 413,100	$ 421,362	$ 429,789	$ 438,385	$ 2,107,636
Total Direct Costs	$2,145,000	$2,187,900	$2,433,618	$2,482,290	$2,531,937	$11,780,745

Detailed budgets and budget justifications FOR EACH YEAR OF SUPPORT are required within the descriptions of each project and core. The budget justifications must clearly explain and detail any budget fluctuations in future years. Please use PHS 398 Form Page 4 for each year of support requested for each project, and PHS 398 Form Page 5 for each individual project. Applications that do not include a detailed budget and budget justification for each year of requested support will not be reviewed.

Biographical Sketch (Overall)

All instructions in the PHS 398 Application Guide must be followed.

Resources (Overall)

All instructions in the PHS 398 Application Guide must be followed.

Research Plan (Overall)

All instructions in the PHS 398 Application Guide must be followed.

All instructions in the PHS398 Application Guide must be followed.

Face Page (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Description, Project/Performance Sites, Senior/Key Personnel, Other Significant Contributors, (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Table of Contents (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Detailed Budget for Initial Budget Period (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Detailed Budget and Budget Justification for each future year of support requested

Please use Form Page 4 for EACH year of support.

Budget for Entire Proposed Period of Support (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Biographical Sketch (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Resources (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Research Plan (Data Core)

All instructions in the PHS 398 Application Guide must be followed.

Each Project

All instructions in the PHS398 Application Guide must be followed.

Face Page (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Description, Project/Performance Sites, Senior/Key Personnel, Other Significant Contributors, (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Table of Contents (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Detailed Budget for Initial Budget Period (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Detailed Budget and Budget Justification for each future year of support requested (for each Project)

Please use Form Page 4 for EACH year of support for each project

Budget for Entire Proposed Period of Support (Project)

All instructions in the PHS 398 Application Guide must be followed.

Biographical Sketch (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Resources (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Research Plan (for each Project)

All instructions in the PHS 398 Application Guide must be followed.

Do not use the Appendix to circumvent page limits. Follow all instructions for the Appendix (please note all format requirements) as described in the PHS 398 Application Guide.

Part I. Overview Information contains information about Key Dates.

Information on the process of receipt and determining if your application is considered on-time is described in detail in the PHS 398 Application Guide.

Applicants may track the status of the application in the eRA Commons, NIH’s electronic system for grants administration.

This initiative is not subject to intergovernmental review.

For efficient grant administration, AHRQ grant administration procedures will be used and conducted in accordance with the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement. The Grants Policy Statement can be found at http://www.ahrq.gov/funding/grants/policies/hhspolicy.html.

Pre-award costs are not allowable.

NOTE: No Cost Extensions will not be allowed.

Applications must be received on or before the due dates in Part I. Overview Information. If an application is received after that date, it will not be reviewed.

Upon receipt, applications will be evaluated for completeness by the Center for Scientific Review, NIH, and for responsiveness by AHRQ. Applications that are incomplete and/or non-responsive will not be reviewed.

In order to expedite the review, applicants are requested to notify the AHRQ Referral Office by email [email protected] when the application has been submitted. Please include the FOA number and title, PD/PI name, and title of the application.

Institutional Review Board (IRB) approval of human subjects is not required prior to peer review of an application (see http://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html). However, initiation of IRB review, if necessary or applicable, is strongly encouraged to assure timely commencement of research.

Although there is no immediate acknowledgement of the receipt of an application, applicants are generally notified of the review and funding assignment within eight (8) weeks.

The PD/PI is required to devote 15% annual effort in each year of the project. Also, the leader of the Data Core and the leaders of each project must each devote at least 15% annual effort in each year of the project.

A planned project timeline must be included, identifying timing of major milestones such as planned release of early findings.

The PD/PI or his/her Center representative will participate in the Technical Expert Panel (TEP) meetings. The TEP will meet up to four times a year, in addition to telephone conferences and electronic exchanges. For budget purposes, applicants should assume that the TEP will meet in the Washington, D.C. area.

Under the cooperative agreement, the Centers will be expected to share data results with the coordinating center in order to co-produce a systems compendium.

AHRQ supports research, evaluations, and demonstration projects concerning the delivery of health care in inner-city and rural areas (including frontier areas), and health care for priority populations. Priority populations include low income groups; minority groups; women; children; the elderly; and individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care. 42 USC 299(c). Women and members of minority groups are included in all AHRQ-supported research projects involving human subjects, unless a clear and compelling rationale and justification is provided that inclusion is inappropriate, e.g. because of the lack of connection between the study and the health of women or particular minorities. Investigators should review the document entitled, AHRQ Policy on the Inclusion of Priority Populations, which is available http://grants.nih.gov/grants/guide/notice-files/NOT-HS-03-010.html. Applicants under this FOA should consider and discuss including priority populations in the research design as specified in this Notice.

The precise content of the data-sharing plan will vary, depending on the data being collected and how the investigator is planning to share the data with the public. Applicants who are planning to share data may wish to describe briefly the expected schedule for data sharing; the format of the final dataset; the documentation to be provided; whether or not any analytic tools also will be provided; whether or not a data-sharing agreement will be required and, if so, a brief description of such an agreement (including the criteria for deciding who can receive the data and whether or not any conditions will be placed on their use); and the mode of data sharing (e.g., under its own auspices by mailing a disk or posting data on its institutional or personal website or through a data archive or enclave). Investigators choosing to share under their own auspices may wish to enter into a data-sharing agreement. References to data sharing may also be appropriate in other sections of the application.

The reasonableness of the data sharing plan or the rationale for not sharing research data will be assessed by the reviewers. However, reviewers will not factor the proposed data sharing plan into the determination of scientific merit or the priority score.

The AHRQ confidentiality statute, 42 USC 299c-3(c), requires that information that is obtained in the course of AHRQ supported activities and that identifies individuals or establishments be used only for the purpose for which it was supplied. Information that is obtained in the course of AHRQ-supported activities and that identifies an individual may be published or released only with the consent of the individual who supplied the information or is described in it. There are civil monetary penalties for violation of the confidentiality provision of the AHRQ statute. 42 USC 299c-3(d). In the Human Subjects section of the application, applicants must describe procedures for ensuring the confidentiality of the identifying information to be collected. The description of the procedures should include a discussion of who will be permitted access to this information, both raw data and machine readable files, and how personal identifiers and other identifying or identifiable data will be restricted and safeguarded. Identifiable patient health information collected by grantees under this FOA will also be obtained and managed in accordance with the HIPAA Privacy Rule, 45 CFR Parts 160 and 164.

The grantee should ensure that computer systems containing confidential data have a level and scope of security that equals or exceeds that established by the HIPAA Security Rules, if applicable, and that established by the Office of Management and Budget (OMB) in OMB Circular No. A-130, Appendix III - Security of Federal Automated Information Systems. The applicability and intended means of applying these confidentiality and security standards to subcontractors and vendors, if any, should be addressed in the application.

Unless otherwise provided in grant awards, AHRQ grantees may copyright, or seek patents for, as appropriate, final and interim products and materials developed in whole or in part with AHRQ support, including, but not limited to, methodological tools, measures, software with documentation, literature searches, and analyses. Such copyrights and patents are subject to a royalty-free, non-exclusive, and irrevocable AHRQ license to reproduce, publish, use or disseminate for any purpose consistent with AHRQ’s statutory responsibilities and to authorize others to do so for any purpose consistent with AHRQ’s statutory responsibilities. In accordance with its legislative dissemination mandate, AHRQ purposes may include, subject to statutory confidentiality protections, making project materials, databases, results, and algorithms available for verification or replication by other researchers. In addition, subject to AHRQ budget constraints, final products may be made available to the health care community and the public by AHRQ or its agents if such distribution would significantly increase access to a product and thereby produce substantial or valuable public health benefits. Ordinarily, to accomplish distribution, AHRQ publicizes research findings but relies on grantees to publish research results in peer-reviewed journals and to market grant-supported products. AHRQ's Office of Communications and Knowledge Transfer (OCKT) wishes to be consulted in advance of publication in order to coordinate announcements of new AHRQ-supported research results with other AHRQ dissemination activities. Important legal rights and requirements applicable to AHRQ grantees are set out or referenced in AHRQ's grants regulation at 42 CFR Part 67, Subpart A (available in libraries and from the GPO's website at http://www.gpo.gov/fdsys/browse/collectionCfr.action?collectionCode=CFR) which incorporates additional applicable provisions on Rights in Data, including 45 CFR Part 74 and 37 CFR Part 401.

All grantees must agree to the "Cooperative Agreement Terms and Conditions of Award" in Section VI.2.A "Award Administration Information" below.

Applicants are required to follow the instructions for post-submission materials, as described in NOT-OD-13-030.

The mission of AHRQ is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable, and to work with the U.S. Department of Health and Human Services (HHS) and other partners to make sure that the evidence is understood and used.

AHRQ's priority areas of focus are:

• Improve health care quality by accelerating implementation of patient-centered outcomes research (PCOR).
• Make health care safer.
• Increase accessibility by evaluating Affordable Care Act (ACA) coverage expansions.
• Improve health care affordability, efficiency, and cost transparency.

As part of this mission, applications submitted to AHRQ to support health services research are evaluated for scientific and technical merit through the AHRQ peer review system.

Applications that are complete and responsive to the FOA will be evaluated for scientific and technical merit by an appropriate peer review group convened in accordance with standard AHRQ peer review procedures that are described in 42 CFR Part 67, Subpart A. Incomplete and/or non-responsive applications or applications not following instructions given in this FOA will not be reviewed. Only the review criteria described below will be considered in the review process.

Administrative Criteria: Upon receipt, applications will be evaluated for completeness by the Center for Scientific Review, NIH, and for responsiveness by AHRQ.

Merit Review Criteria: Merit Review Criteria, as described below, will be considered in the review process.

Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following review criteria and additional review criteria (as applicable for the projects proposed).

Peer review of the overall scientific and technical merit will emphasize a synthesis of two major aspects: 1) review of the Center of Excellence as an integrated effort focused on a central theme and 2) review of the merit of individual research projects and core components in the context of the proposed Center.

Reviewers will consider each of the review criteria below in the determination of scientific merit, and give a separate score for each. An application does not need to be strong in all categories to be judged likely to have major scientific impact. For example, a project that by its nature is not innovative may be essential to advance a field.

Significance

Does the project address an important problem or a critical barrier to progress in understanding and identifying new types of emerging delivery systems? Will the proposed project lead to a better understanding of current and emerging health care delivery system models in order to better target PCOR dissemination? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? What will be the effect of these studies on improving PCOR dissemination and improved uptake of CER findings?

Investigator(s)

Do the PD/PI, leader of the Data Core, and leaders of each program project each have at least 15% annual effort devoted to the project in each year of the application? Are the PD/PI, collaborators, and other researchers appropriately trained and well suited to carry out this work? Is the proposed leadership structure for the Data Core well justified? Are investigators for each proposed program projects qualified to conduct these studies? Is the work proposed appropriate to the experience level of the PD/PI and other researchers? Do the PD/PI and investigative team bring complementary and integrated expertise to the program projects? Are the leadership approach, governance and organizational structure appropriate for the overall project?

Innovation

Is the project original and innovative? For example: Does the overall project seek to improve our understanding of how to compare the performance of different types of health care systems, assess the extent to which these systems use PCOR evidence, assess the quality and cost of the care systems provide, seek to understand the characteristics of high-performing systems, propose innovative methods for disseminating PCOR evidence within health care systems, or address an innovative hypothesis or critical barrier to progress PCOR dissemination? Does the project develop or employ novel concepts, approaches or methodologies for this area, such as for dealing with selection issues or issues of assigning patients to systems?

Approach

Is a planned project timeline included, and does the timeline include availability of early findings? It is desirable for applicants to have plans for early findings. Are the conceptual or clinical framework, design, methods, and analyses adequately developed, well-integrated, well-reasoned, and appropriate to the aims of the program projects? Are the health care systems comparisons proposed in the application well justified? Does the applicant propose an appropriate budget for the Data Core activities? Are planned Data Core activities well justified? Does the application include a plan to share analytic results, tax identification numbers (TINs), and physicians national provider identifiers (NPIs) with the Coordinating Center for the systems compendium? Does the application include a well-developed plan for acquiring data? Does the Data Core include methodological work on how to statistically compare systems in terms of their characteristics and performance? Do primary data collection efforts have the appropriate methods?

Do proposed inter-related projects describe systems to be studied, system processes, types of incentives, and what environments and markets speed up the diffusion of PCOR findings into practice to produce patient-centered outcomes? How does the applicant plan to assess the impact of systems on PCOR findings?Are proposed projects well designed to have a major impact on health care in the United States? Has the applicant defined and justified the systems and geographic area to be studied? Do proposed projects compare the outcomes of different types of systems, processes, incentives and environments? Does the applicant acknowledge potential problem areas and consider alternative strategies? Are benchmarks for success presented?

Environment

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed Data Core activities and individual inter-related projects benefit from unique features of the scientific environment, or subject populations, or employ useful collaborative arrangements? Is there evidence of institutional support? How extensive are the capacities, experience, expertise and other resources of the applicant institution and participating organizations, institutions, and other entities? How well-aligned are they with the needs and aims of the proposed work? How likely are they to contribute to the project’s success? Have potential environmental obstacles and the means for avoiding or mitigating their effects been identified?

As applicable for the project proposed, reviewers will evaluate the following additional items while determining scientific and technical merit, and in providing an overall impact score, but will not give separate scores for these items.

Protections for Human Subjects

For research that involves human subjects but does not involve one of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate the justification for involvement of human subjects and the proposed protections from research risk relating to their participation according to the following five review criteria: 1) risk to subjects, 2) adequacy of protection against risks, 3) potential benefits to the subjects and others, 4) importance of the knowledge to be gained, and 5) data and safety monitoring for clinical trials.

For research that involves human subjects and meets the criteria for one or more of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate: 1) the justification for the exemption, 2) human subjects involvement and characteristics, and 3) sources of materials. For additional information on review of the Human Subjects section, please refer to the Guidelines for the Review of Human Subjects.

Degree of Responsiveness

Reviewers will assess how well the application addresses the purpose and objectives of this FOA. How responsive is the application to the special eligibility criteria, including the project requirements, noted in the FOA? []

Inclusion of Priority Populations

Reviewers will assess the adequacy of plans to address the needs of both genders, racial and ethnic minorities (and subgroups). Reviewers will assess the adequacy of attention to AHRQ priority populations (see above discussion on Priority Populations in section IV.6 Other Submission Requirements ).

Budget and Period of Support

The committee will evaluate whether the proposed budget is reasonable and whether the requested period of support is appropriate in relation to the proposed research. [

Privacy and Security Protections for Patients

Reviewers will assess the resources and processes to be used to address privacy and security issues in the development and implementation of the intervention.

Not Applicable

Applications will be evaluated for scientific and technical merit by (an) appropriate Scientific Review Group in accordance with AHRQ review policy and procedures, using the stated review criteria. Assignment to a Scientific Review Group will be shown in the eRA Commons.

As part of the initial merit review, all applications will:

- Undergo a selection process in which only those applications deemed to have the highest scientific merit will be discussed and assigned a priority score

- Receive a written critique

As part of the scientific peer review, all applications will receive a written critique.

Applications will be assigned on the basis of established AHRQ referral guidelines to the appropriate AHRQ Office or Center. Applications will compete for available funds with all other recommended applications submitted in response to this FOA. The following will be considered in making funding decisions:

• Scientific and technical merit of the proposed project as determined by peer review.
• Availability of funds.
• Responsiveness to goals and objectives of the FOA.
• Relevance and fit within AHRQ portfolio priorities, as well as overall programmatic balance.

After the peer review of the application is completed, the PD/PI will be able to access his or her Summary Statement (written critique) via the eRA Commons.

Generally, applicants should anticipate eight months between the application submission date and the earliest possible start date.

If the application is under consideration for funding, AHRQ will request "Just-In-Time" information from the applicant. Just-In-Time information generally consists of information on other support, any additional information necessary to address administrative and budgetary issues, and certification of IRB approval of the project's proposed use of human subjects. For details regarding IRB approval, applicants may refer to the "AHRQ Revised Policy for Institutional Review Board (IRB) Review of Human Subjects Protocols in Grant Applications" (http://grants.nih.gov/grants/guide/notice-files/not-hs-00-003.html).

Once all administrative and programmatic issues have been resolved, a formal notification in the form of a Notice of Award (NoA) will be provided to the applicant organization for successful applications. The NoA signed by the AHRQ grants management officer is the authorizing document and will be sent via email to the grantee’s business official.

Awardees must comply with any funding restrictions described in Section IV.5. Funding Restrictions.

Selection of an application for award is not an authorization to begin performance. Any costs incurred before receipt of the NOA are at the recipient’s risk. These costs may be reimbursed only to the extent considered allowable pre-award costs. See also Section IV.5., Funding Restrictions.

Any application awarded in response to this FOA will be subject to the DUNS, SAM Registration, and Transparency Act requirements as noted on the AHRQ web site at http://www.ahrq.gov/fund/postawrd.htm#terms.

All AHRQ grant and cooperative agreement awards are subject to AHRQ’s grants regulations at 42 CFR Part 67, Subpart A, the HHS Grants Policy Statement (see http://www.ahrq.gov/funding/policies/hhspolicy/index.html), and the terms and conditions set forth in the Notice of Award.

All applicants will be subject to a term and condition that applies the terms of 48 CFR section 3.908 to the award, and requires that grantees inform their employees in writing of employee whistleblower rights and protections under 41U.S.C. 4712 in the predominant native language of the workforce. A standard term and condition of award will be included in the notice of award.

As necessary, additional Terms and Conditions will be incorporated into the NoA.

Activities conducted under this award that involve the collection of information e.g., conducting surveys or requesting responses to uniform questions from nine or more persons, establishments or other entities, are currently required to be cleared by OMB under the Paperwork Reduction Act (PRA) (44 USC 3501-3521). PRA review and approval is required in cooperative agreements if AHRQ has significant input or control over the data collection activity. Detailed information on the PRA can be found at http://www.hhs.gov/ocio/policy/collection/. Submissions for clearance under PRA are through AHRQ and HHS. Therefore, affected grantees should include the time in their proposed timelines to develop materials and receive necessary clearances. It typically takes at least six months from date of initial submission to AHRQ to receive clearances, and sometimes much longer if submissions are incomplete or the justification for the proposed data collection plans are questioned during the clearance process. Information collection that requires PRA clearance may not begin until grantees receive written notification via e-mail from AHRQ that clearance has been obtained. Detailed information on the PRA can be found at http://www.hhs.gov/ocio/policy/collection/.

Terms and Conditions of Cooperative Agreement Award

The following cooperative agreement terms and conditions of award award are in addition to otherwise applicable OMB administrative guidelines: DHHS grant administration regulations at 45 CFR Parts 74 or 92 (Part 92 is applicable for State and local Governments), and other DHHS, PHS, and AHRQ grant administration policy statements. AHRQ will use these procedures in evaluating and administering the cooperative agreements under this FOA.

The administrative and funding instrument used for this program is a cooperative agreement. A cooperative agreements is an "assistance" mechanism (rather than an "acquisition mechanism), in which substantial AHRQ programmatic involvement with the grantee is anticipated during the performance of the activities. Under the cooperative agreement, the AHRQ purpose is to support and stimulate the recipient's activities by involvement in and otherwise working jointly with the award recipient in a partnership role; it is not to assume direction, prime responsibility, or a dominant role in the activities. Consistent with this concept, the dominant role and prime responsibility resides with the grantee and with the PD/PI for the project as a whole, although specific tasks and activities may be shared between a grantee and AHRQ as described below. Cooperative activities are intended to strengthen the individual grantee activities through the facilitation of data sharing, data access and communications.

All cooperative activities that include significant government involvement will require prior approval by the AHRQ Program Official.

Program Director/Principal Investigator (PD/PI) Rights and Responsibilities
The PD/PI will have the primary responsibility for conducting research in accordance with the terms and conditions of the Notice of Award, and cooperating with other key parties, including the AHRQ Program Official, other awardees, and AHRQ's contractor, the Coordinating Center (CC) on Systems.

As indicated in Section 2.1 of the FOA, AHRQ plans to fund a Coordinating Center under separate contract to assist in dissemination and data work. The Coordinating Center will set up and convene a Technical Expert Panel (TEP); organize dissemination activities, such as conferences and publications in special issues of journals; and write Data Briefs and Policy Briefs. The Coordinating Center will work to develop a systems compendium of systems across the U.S. This compendium will be shared with the grantees. The grantee will be required to share data results with the Coordinating Center for the Coordinating Center's purposes of strengthening the individual grantee activities through the facilitation of research collaborations.

Each PD/PI or his/her Center representative will participate in the TEP meetings. The TEP will meet up to four times a year, in addition to telephone conferences and electronic exchanges. For budget purposes, applicants should assume that the TEP will meet in the Washington, D.C., area. In response to the TEP meetings, the PD/PI will actively participate in the formulation of plans to promote generalizability across Centers of Excellence. Awardees will refine and revise, as necessary, selected methodologies in accord with plans developed collaboratively by the TEP.

AHRQ Responsibilities
AHRQ program staff will have substantial programmatic involvement that is above and beyond the normal stewardship role in awards, as described below.

AHRQ will have substantial scientific and programmatic involvement during the conduct of the project. AHRQ staff will provide technical assistance, advice, and support to the awardee, and will assist in the cooperative work of the project beyond the usual program stewardship for grants. The Program Official will assist with any data linkages needed between the grantee, AHRQ, and/or the Coordinating Center. Also, the Coordinating Center will share systems data, convene the TEP, set the agenda for TEP meetings, and preside over them. The Program Official will maintain contact with the PD/PI in between TEP meetings to consult on key project decisions to trouble-shoot problems and confirm that projects are proceeding according to plan. The Program Official will assist the Coordinating Center in selecting the outside experts to serve on the TEP, considering awardee nominations and other qualified individuals. The Program Official and the Coordinating Center, in coordination with the grantee, will also disseminate project findings to research and policy audiences.

The progress of work by each awardee will be reviewed at least annually.

Additionally, AHRQ program staff will be responsible for the normal scientific and programmatic stewardship of the award and will be named in the award notice.

Collaborative Responsibilities

The PD/PI and the Coordinating Center will work collaboratively to build a taxonomy and systems compendium of health care delivery systems found across the country. AHRQ will share its own potential data and results with the Coordinating Center's compendium, and the awardee is expected to share data results. For purposes of the collaborative research, the systems compendium will list systems by name and include identifiers e.g., tax identification numbers (TINs), hospital identifier (AHA id, Medicare provider number, etc.), and physicians national provider identifiers (NPIs) as appropriate - with outcome measures and performance of the system listed. Using the systems compendium, both AHRQ and the PD/PI will collaborate with the Coordinating Center so that the Coordinating Center can develop short Policy Briefs and Data Briefs on comparing systems, and can develop PCOR dissemination initiatives.

When multiple years are involved, grantees will be required to submit the Non-Competing Grant Progress Report (PHS 2590) annually.

The annual progress reports must include Sections 2.2.6 A through F as described in the general PHS form 2590 instructions. For details regarding progress report submission, refer to http://www.ahrq.gov/fund/noncomp.htm. If instructions on the AHRQ website are different from the PHS form 2590 instructions, follow the instructions on the AHRQ website.

The Progress Report is to include descriptive and evaluative comments on both completed activities and plans for the remainder of that year, including any changes foreseen in the future. At a minimum, the reports will include descriptive comments on: progress to date measured against project aims and the planned project timelines included in the application; methodological changes implemented; key preliminary findings; significant problems and resolutions; inclusion of priority populations; and project related publications, presentations, and dissemination activities. AHRQ does not currently limit the length of the progress report to 2 pages. AHRQ will provide the due date for submitting annual progress reports. In addition to the annual progress report, recipients may be required to submit quarterly progress reports to AHRQ. Detailed instructions on reporting requirements will be provided with the grant award.

Expenditure data is to be reported on the Federal Financial Report (FFR; SF 425). AHRQ requires annual financial expenditure reports for ALL grant programs as described in the HHS Grants Policy Statement (see http://www.ahrq.gov/fund/hhspolicy.htm). AHRQ implementation of the FFR retains a financial reporting period that coincides with the budget period of a particular project. However, the due date for annual FFRs will be 90 days after the end of the calendar quarter in which the budget period ends. Note that this is a change in due dates of annual FFRs and may provide up to 60 additional days to report, depending upon when the budget period end date falls within a calendar quarter. For example, if the budget period ends 4/30/2013, the annual FFR is due 9/30/2013 (90 days after the end of the calendar quarter of 6/30/2013).

A final Progress Report, final Federal Financial Report, and Final Invention Statement are required when an award ends. All final reports are due within 90 days of the project period end date. For further details regarding grant closeout requirements, refer to http://www.ahrq.gov/fund/closeout.htm.

We encourage inquiries concerning this funding opportunity and welcome the opportunity to answer questions from potential applicants.

eRA Commons Help Desk (Questions regarding eRA Commons registration, submitting and tracking an application, documenting system problems that threaten submission by the due date, post submission issues)
Telephone: 301-402-7469 or 866-504-9552 (Toll Free)
Finding Help Online: http://grants.nih.gov/support/index.html
TTY: 301-451-5939
Email: [email protected]

Direct your questions about general FOA issues, including information on the inclusion of priority populations to:

William Encinosa, Ph.D.
Center for Delivery, Organization, and Markets
Agency for Healthcare Research and Quality
Telephone: 301-427-1437
Email address: [email protected]

Direct your questions about peer review issues of grant application made in response to this FOA to:

Nghia M. Vo, M.D.Agency for Healthcare Research and Quality
Office of Extramural Research, Education and Priority PopulationsTelephone: 301-427-1191
Email address: [email protected]

Direct your questions regarding fiscal matters to:

Anna Caponiti
Office of Management ServicesAgency for Healthcare Research and Quality
Telephone: 301-427-1402
Email address: [email protected]

Recently issued AHRQ policy notices may affect your application submission. A full list of policy notices published by AHRQ is provided in the NIH Guide for Grants and Contracts. Notices can also be found at AHRQ Grants Policy Notices. All awards are subject to the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement.

This program is described in the Catalog of Federal Domestic Assistance at http://www.cfda.gov/ and is not subject to the intergovernmental review requirements of Executive Order 12372 or Health Systems Agency review. Awards are made under the authority of 42 USC 299 et seq. and, 42 CFR Part 67 and in accordance with 45 CFR Parts 74 or 92 and other referenced applicable statutes and regulations. All awards are subject to the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement. The HHS Grants Policy Statement can be found at http://www.ahrq.gov/fund/hhspolicy.htm.