and Human Services (HHS)
EXPIRED
EVALUATION OF DEMONSTRATIONS: "REWARDING RESULTS" RELEASE DATE: April 3, 2002 RFA: HS-02-006 Agency for Healthcare Research and Quality (AHRQ) (http://www.ahrq.gov) APPLICATION RECEIPT DATE: 05/20/2002 THIS RFA CONTAINS THE FOLLOWING INFORMATION: o Purpose of this RFA o Research Objectives o Mechanism(s) of Support o Funds Available o Eligible Institutions o Individuals Eligible to Become Principal Investigators o Special Requirements o Where to Send Inquiries o Letter of Intent o Submitting an Application o Peer Review Process o Review Criteria o Receipt and Review Schedule o Award Criteria o Required Federal Citations PURPOSE OF THIS RFA This Request for Application (RFA) expresses the interest of the Agency for Healthcare Research and Quality (AHRQ) in applications for grant support for a comprehensive evaluation and analysis of a major national initiative to demonstrate the impact of financial and nonfinancial incentives on the quality of health care. "Rewarding Results: Aligning Incentives with High-Quality Health Care" is a national initiative of The Robert Wood Johnson Foundation (RWJF) and other funding and technical assistance partners. Funding partners include AHRQ and The California Health Care Foundation (CHCF). Technical assistance partners include The National Health Care Purchasing Institute (NHCPI) and The Center for Health Care Strategies (CHCS). The "Rewarding Results" initiative is intended to develop, evaluate, and diffuse innovations in systems of provider payments and non- financial incentives that encourage and reward high-quality care. Each of several demonstration projects will include strong measurable goals for high quality care that (1) address the major needs of the applicant"s members, enrollees, beneficiaries, recipients, or employees, (2) align payment systems and non-financial incentives with the applicant"s quality goals, and (3) include a self-evaluation approach that will help the grantee and initiative partners continually improve their financial and non-financial incentives. The overall demonstration program includes substantial collaborative technical assistance to help grantees, applicants, and others to plan, implement, and continually improve their efforts to align payment and incentives with quality goals. Eligible applicants for the demonstration projects include employers, health plans, purchasing coalitions, Medicaid agencies, and Veterans Administration. The Call-for-Proposals (CFP) for demonstration projects is available at www.nhcpi.net. A major component of this initiative will be a comprehensive evaluation across all the demonstration projects, designed to draw critical findings and potential lessons from the combined experience of the projects, so that public and private payers can apply these lessons to the design of future incentive programs. This RFA expresses AHRQ"s interest in funding such an overarching evaluation project that would provide rigorous, objective, timely, and essential evidence about demonstrated innovations in systems of provider payments and non- financial incentives for improving the quality of care and about the consequences of such innovations. In conducting the overarching evaluation, the AHRQ awardee will work with funding and technical assistance partners and all the demonstration projects. The AHRQ- supported overarching evaluation will interact with and complement the self-evaluation activities done by each demonstration project. Examples of important questions to be addressed in the overarching primary evaluation include: In what ways does provider performance change in response to incentives, and how do responses vary among different performance measures and over time? Do providers respond primarily to financial or nonfinancial incentives, and what are the interactive effects among those incentives? Does the incentive system produce any measurable positive or negative economic consequences for providers? Do those consequences track provider performance appropriately, rewarding high quality providers relative to others? Does the burden of reporting increase or decrease for providers, and how does any change affect their satisfaction and the business case for improving quality? How do changes in provider behavior affect consumer satisfaction? What are the impacts on the costs of health plans and other payers? How do the nature of the health care market, the type of health plan or payer organization, and the underlying reimbursement system affect these outcomes? RESEARCH OBJECTIVES 1. Background The mission of AHRQ is to support and conduct research that improves the outcomes, quality, access to, and cost and utilization of health care services. AHRQ achieves this mission through health services research designed to (1) improve clinical practice, (2) improve the health care system"s ability to provide access to and deliver high quality, high-value health care, and (3) provide policymakers with the ability to assess the impact of payment and organizational changes on outcomes, quality, access, cost, and use of health care services. AHRQ"s research agenda is designed to be responsive to the needs of consumers, patients, clinicians and other providers, institutions, plans, purchasers, and Federal and State policymakers. AHRQ seeks to provide all of these decision makers with the evidence-based information that they need to improve quality and outcomes, control costs, and assure access to needed services. A 1999 report from the Institute of Medicine (IOM), "To Err is Human: Building a Safer Health System," provided documentation and analysis of the scope of patient safety issues in the American health care system. In a related 2001 report, "Crossing the Quality Chasm", the IOM noted that patient safety problems reflect "only a small part of the unfolding story of quality in American health care." According to this second report, there is a "chasm" between the health care we have and the health care we could have, and this gap to a large extent springs from two overarching system features: the way we pay for care, and the way we structure the organizations that provide it. The report calls for "action to improve the American health care delivery system as a whole, in all of its quality dimensions, for all Americans." These reports and similar calls for action have raised questions about what particular public and private sector changes in the health care system would most likely yield desired improvement. Changes under consideration include new methods of payment for providers, practical and effective risk adjustment methods, alternative approaches to address capital requirements for improving the delivery of health care, and changes in healthcare organizations and in the healthcare system that would result in more efficiently designed care processes, effective use of information technologies, the development of effective teams, and improved coordination of care across patient conditions, services, and settings. In the context of the IOM"s call for action, the initiative "Rewarding Results" will develop, evaluate, and diffuse innovations in systems of provider payments and non-financial incentives that encourage and reward high-quality care. A major component of this initiative will be the AHRQ-supported overarching evaluation of the demonstration projects, as discussed in this RFA. This RFA complements many other current AHRQ initiatives to improve the quality of health care. Particularly relevant to this RFA is a recently published Program Announcement (The Impact of Payment and Organization on Cost, Quality, and Equity, PA-01-125, NIH Guide, July 31, 2001) which expresses AHRQ"s highest priority interests in research that would provide rigorous, objective, and essential evidence required by public and private decision makers seeking to understand and improve the health care system, to make changes in health care delivery, insurance, and financing, and to manage the system in a manner that would induce efficient, effective, timely, equitable and accessible health care. Relevant issues under that initiative include (1) how different payment methodologies and financial incentives within the system affect health care quality, costs, and access and (2) the impact of purchaser and public sector initiatives on quality, costs, and access to health care and health insurance. Of particular interest is the impact of such financial factors on the quality of health care for vulnerable populations. Although past and current AHRQ initiatives have provided an expanded knowledge base about best practices in health care and about quality measurement and quality improvement, there is still considerable uncertainty about the most effective mechanisms especially payment mechanisms-- for improving quality. In practice, attempts by organizations to align incentives have often been linked with issues of efficiency, productivity, revenue, and profits rather than with health care quality. The "Rewarding Results" initiative is intended to provide large, sustained, innovative, and pragmatic demonstrations of how to make the link between incentives and quality a strong one. Consistent with the AHRQ mission, the AHRQ-supported primary evaluation project will provide rigorous, objective, timely, and essential evidence about such demonstrated innovations and their consequences. NHCPI, which serves as the National Program Office (NPO) for the "Rewarding Results" initiative, will offer substantial technical assistance for demonstration project applicants during their application process. NHCPI is making available technical assistance documents that provide information on case studies of existing efforts to align payment systems and nonfinancial incentives with quality goals, models for redesigning payment systems and nonfinancial incentives, approaches for estimating potential return on investment for changing provider rewards, and options for determining the impact of the rewards innovations on quality and cost. These documents and RWJF"s CFP are available at www.nhcpi.net. AHRQ applicants may contact NHCPI if assistance is needed in understanding these documents. These documents provide technical background for the potential nature, extent, and scope of the demonstration projects. Thus, these documents also serve as essential background for the objectives and methods of the overarching evaluation discussed below. AHRQ applicants are urged to make use of these documents in developing their applications. A single Primary Evaluation Team will be funded by AHRQ under a cooperative agreement, a core element of which is an Evaluation Advisory Committee (EAC), comprised of representatives from the funding and technical assistance partners, participating in the "Rewarding Results" initiative. The EAC will hold up to four meetings a year both to identify the questions, common measures, and methods which cut across all the demonstration projects and to focus on questions, measures, methods, and activities central to the overarching evaluation. The EAC will provide advice to the Primary Evaluation Team on the overarching evaluation and, in turn, provide coordination across the demonstration projects in all their evaluation activities. For budget purposes, applicants should assume that the EAC will meet in the Washington, D.C. area. Facilities will be arranged by the AHRQ project officer in consultation with the EAC. Evaluation activity in the "Rewarding Results" initiative will be conducted at two levels. First, each of the demonstration project grantees will conduct self-evaluation activities. Second, the Primary Evaluation Team will conduct an overarching primary evaluation across all demonstration projects. These two levels of evaluation are, however, interactive. On the one hand, each of the demonstration projects will (1) engage in its own self-evaluation activities, (2) provide information relevant to the overarching primary evaluation to the Primary Evaluation Team, and (3) be provided evaluation-related advice and assistance by the Primary Evaluation Team. Note that considerable non-evaluation-related technical assistance will be also be available to each demonstration project from NHCPI and CHCS. On the other hand, the Primary Evaluation Team will conduct the overarching primary evaluation and provide assistance and advice to funding and technical assistance partners on the demonstration projects" evaluation activities. Given the interactive nature of evaluation activities, AHRQ applicants should include information in their application demonstrating their ability to work collaboratively, and their agreement to do so. In particular, in the development of their applications, AHRQ applicants may consider the following: o As a condition of accepting RWJF funds, all demonstration project grantees will be required to participate in the AHRQ evaluation. Demonstration projects will be expected to meet RWJF requirements for the submission of narrative and financial reports and also will be required by RWJF to submit periodic information needed for overall project performance monitoring and management. Demonstration project directors may be asked to attend periodic meetings and give progress reports on their grants. At the close of each grant, each demonstration project grantee is expected to provide a written report on the project and its findings suitable for wide dissemination. o Successful RWJF applicants will have demonstrated that they currently possess the systems and procedures to provide the healthcare utilization and financial data that are necessary both for monitoring and evaluating the demonstration programs they have proposed and for the AHRQ-supported overarching evaluation of the entire "Rewarding Results" initiative. These data will be sufficient to compare the costs and results of the incentive demonstration with other payment approaches each successful applicant is using within a selected market and product line. RWJF applicants will have committed, in their proposals, to make such information available in a timely manner. For more on the relationship among participants, please see Cooperative Agreement Terms and Conditions of Award below under SPECIAL REQUIREMENTS. Applicants under this AHRQ initiative are cordially invited to a conference of applicants for RWJF demonstration projects. One purpose of the conference is to give background information to RWJF applicants and respond to their questions about the preparation of applications for demonstration projects under the "Rewarding Results" initiative. The conference will be held in Chicago, April 17-19, 2002. Attendance is not a prerequisite for potential applicants under the AHRQ-supported evaluation initiative. No technical assistance from AHRQ program staff will be provided at the conference. Attendees must pay for their own travel and accommodation costs. The conference will be open to any individual or organization intending to apply. For further information on the conference, see the NHCPI Website or contact Michael Hagan at the address listed below. 2. Objectives The main objective of the primary evaluation is to address a set of key issues which fall into three broad categories: formative and developmental issues, reporting standards, and outcomes. Some overlap necessarily exists among these categories. A. Formative and developmental issues: o Can the payers patients establish process and outcome goals for clinical practice, with the concurrence of their network of providers? Can they establish performance measures that address those goals, and validate the relationship between the measures and their goals? o Can the payers determine appropriate financial and non-financial incentives for providers to meet performance standards selected for study? Which incentive systems do they consider, and which do they actually select? What factors drive those decisions? What is the source, extent and role of technical assistance in those decisions? Can they develop the actuarial and information infrastructure to provide incentives and feedback to providers in a timely manner? In particular, can they collect the data necessary to support provider- level measurement? o Can the projects develop and implement a research design that enables them to explore the effects of alternative incentive structures? Can they provide feedback to allow for ongoing improvements to the incentive systems, while maintaining the integrity of the research design? If innovations in incentive systems are implemented over time or across multiple sites, how are such innovations diffused? Are they changed and shaped by the process of implementation itself? Can the projects establish baselines against which subsequent performance will be measured? o Do claims and encounter data meet all the data requirements for measuring performance, or are new reporting systems or codes needed? If so, can the projects develop and implement those new systems? B. Reporting standards issues: o Do the data provided by the projects meet audit standards, and support their claims for the outcomes of the initiative? o How reliable are the data received from individual providers, and do the projects have effective systems to monitor those data? C. Outcomes o In what ways does provider performance change in response to incentives, and how does that vary among different performance measures and over time? Do providers respond primarily to financial or nonfinancial incentives, or are the effects interactive? If the effects are interactive, how does that interaction work? o Does the incentive system produce any measurable positive or negative economic consequences for providers? Do those consequences track provider performance appropriately, rewarding high quality providers relative to others? Possible consequences include changes in unit prices for providers, or changes in caseload and casemix (both profitable and unprofitable). o Do spillover effects positive or negative occur in other areas of clinical practice that are not directly subject to incentives? o How does the incentive system affect provider satisfaction, through both tangible outcomes and changes in intangible professional and peer rewards? o Does the burden of reporting increase or decrease for providers, and how does any change affect their satisfaction, participation, and the business case for quality. o How do changes in provider behavior affect consumer satisfaction? o What are the impacts on the costs of health plans and other payers? o How do the nature of the health care market, the type of health plan or payer organization, and the underlying reimbursement system affect these outcomes? o Do assessments of outcomes vary with type of care provided, (e.g., primary care, speciality care), with settings of care (e.g., inpatient, ambulatory, rehabilitative, long term care), or by populations served (e.g., publicly or privately insured, the elderly or children)? 3. Demonstration projects" evaluations Each demonstration project"s evaluation team will conduct a qualitative and quantitative self-evaluation of the implementation and outcomes of the "Rewarding Results" demonstration for their particular organization"s project. The demonstration project"s evaluation team will also be responsible for working with the AHRQ-supported Primary Evaluation Team to develop and implement standard concepts and measures among the demonstration projects, and for reporting data, qualitative information, and self-evaluation findings back to the Primary Evaluation Team. In addressing formative, planning, and demonstration project development issues, each demonstration project"s self- evaluation staff will do the following: o Work with the project managers to develop a plan to design and implement a performance incentive system, with clear milestones, o Document the process that the organization uses to establish goals for clinical practice, and for obtaining input from its providers, document how the organization establishes performance measures for those goals, and obtains buy-in from providers. o Document any research or analyses that the organization undertakes to validate the relationship between goals and performance measures. o Work with the Primary Evaluation Team and evaluators from other projects to obtain standard measures for shared performance goals, and work with their own organization to revise their internal measures where necessary to meet the standard. o Document the process that the organization uses to determine financial and nonfinancial incentives, and the extent of provider involvement in those decisions. o Assess the readiness of the actuarial and information system (IS) infrastructure to support a performance incentive system and monitor any changes that are necessary. o Assess the ability of providers to meet the minimum reporting requirements, and the variation in the quality of the data received from different providers. o Identify the key leaders on all of these issues both within the organization and among the provider community, and the role that they play in changing the culture of the organization, document how lessons learned are incorporated into the incentive systems. o Provide regular feedback on the project development process to the primary evaluators, through a simple, web-based reporting system. This reporting system should give monthly feedback on progress toward the milestones, and factors contributing to success or failure in meeting those milestones. The demonstration projects will also conduct self-evaluations of program outcomes. In the context of those self-evaluations, demonstration projects will do the following: o Work with the primary evaluators to develop a research design that is appropriate for the specific project, and work with the project managers to implement that design, document the technical, practical, and political difficulties that conducting a reimbursement and feedback experiment entails. o Establish qualitative and quantitative baselines against which both implementation and outcomes can be measured. Those baselines might include: current reimbursement systems and their incentives or disincentives for clinical performance, current IS technology and the ability to collect and analyze provider-level data, current costs of care for specified chronic conditions, the extent of performance measurement and feedback to providers on their performance, comparisons of current provider performance with standards for appropriate care, and the extent of variation in performance among providers, and assessments of provider and consumer satisfaction, providers" views of the current reimbursement system, and their ability to perform high- quality, cost-effective care. o Work with the Primary Evaluation Team to develop strategies to assess the perspectives of both providers and consumers on the effectiveness of the reward system: those strategies might include focus groups, site visits, or surveys, and address such issues as satisfaction, the business case for rewarding performance, and changes in the administrative burdens that providers face. o Determine how provider performance changes in response to incentives, track trends in performance and in the variation of performance among providers, determine to the extent that the experimental design permits the provider response to different types of incentives, and the interactive effects of incentives. o Identify possible spillover effects in other areas of clinical practice that are not directly affected by incentives. o Determine the changes in administrative costs of the organization resulting from the implementation of the new systems, if possible, determine changes in the organization"s overall costs for the management of key chronic conditions. o Assess changes in the satisfaction of both providers and consumers, and in providers" perceptions of their ability to perform high-quality, cost-effective care. 4. Primary Evaluation Team The Primary Evaluation Team will work to produce overarching conclusions from the demonstration projects. The Primary Evaluation Team may: o Provide extensive technical assistance to each project"s self- evaluation activities as the project develops its performance plans. Topics for technical assistance include goal setting, performance measurement, experimental design, and developing baselines. o Organize group technical assistance meetings bringing all of the demonstration projects" evaluation teams together. Initially, those meetings will focus on the development of standard measures and survey instruments, and strategies that all of the projects will use. Subsequent meetings will allow the evaluators to participate in peer- to-peer technical assistance, and will give the Primary Evaluation Team the opportunity to discuss problems that the evaluators are facing. o Develop a simple web-based reporting system for the projects to report monthly activities and progress toward milestones, assess the quality of the data and information systems for each project, and the validity of the findings that they report. o Depending on the likelihood of successfully implementing well- designed experiments for each project, assess the feasibility and cost of using unfunded applicants or other demonstration projects as "controls." o Conduct a qualitative evaluation of how successful the projects are in developing and implementing new reimbursement and incentive systems, and changing the culture of both payers and provider organizations, the factors that contribute to successful implementation, and the factors that impede it. That evaluation will use information from the monthly reports from the projects, the group meetings with the demonstration projects" evaluation teams, structured site visits, and focus groups with providers. o To the extent possible, conduct an analysis of the effects of financial and nonfinancial incentives on provider performance and satisfaction, consumer satisfaction, costs to health plans, and the business case for providers. If the demonstration projects are able to design and implement successful experiments, the Primary Evaluation Team might conduct a meta-analysis of their findings. Otherwise, it might be necessary to try and obtain comparison data from unfunded demonstration project applicants. o For both the qualitative and the quantitative findings, assess the extent to which the nature of the health care marketplace, the type of health plan or payer, and the underlying reimbursement system may affect the observed outcomes. o Develop a series of formative and summative evaluation products for AHRQ, NHCPI, and RWJF that provide ongoing insights into both program development and the factors contributing to program successes and failures, to include an assessment of IS "best practices" to support quality initiatives. 5. Methods It is anticipated that the methods used in the project funded under this RFA will be both qualitative and quantitative. Applicants are expected to present a clear, cohesive and compelling evaluation design addressing both quantitative and qualitative methodological issues. These issues may include, for example: confounding factors that obscure or bias observation of the effects of the demonstrations, measurement issues (e.g., reliability and the extent to which measures reflect concepts they are intended to measure), potentially adverse effects of the proposed evaluation design and solutions to such problems, approaches to systematically aggregate multi-project, multi- level, multi-period data, strategies for analysis of data and presentation of results, potential quasi-experimental designs (e.g., the use of applicants not funded as demonstration projects as "controls"), generalizability (i.e., the extent to which assessments of demonstration activities and effects can be extrapolated to similar environments), potential existing sources of data, and the use of existing interview protocols. Applicants may address above-listed issues to the extent there is relevance to their research design. This list is for illustrative purposes and not to be taken as collectively exhaustive of all issues relevant to the evaluation. Applicants are expected to draw on their own experience, additional technical expertise, and relevant background literature in conceptualizing and presenting an evaluation design that addresses the objectives of the "Rewarding Results" initiative. [See, e.g., Rossi and Freeman, 1993, for a systematic approach to evaluation, and "Qualitative Methods in Health Services Research", 1998, for potentially relevant methodological discussions.] Proposed designs must be flexible enough to take into consideration the variety of approaches that demonstration projects might include in aligning their systems of payments and non-financial incentives with quality goals. [See CFP for a list of potential demonstration project approaches.] Note that the Paperwork Reduction Act requires that grantees participating in cooperative agreements generally need approval from the Office of Management and Budget (OMB) to collect certain data such as surveys from more than nine respondents. Accordingly, applicants should (1) provide a justified budget for appropriate expertise, (2) develop time lines reflecting the efforts necessary to gain approval, and (3) delineate contingency plans should the request be disallowed. Details of the Act and procedures to gain approval are available at: http://www.hhs.gov/oirm/infocollect/ MECHANISM OF SUPPORT This RFA will use the cooperative agreement (U01) award mechanism under which the Principal Investigator retains the primary responsibility and dominant role for planning, directing, and executing the proposed project, with AHRQ staff being substantially involved as a partner with the Principal Investigator, as described under the section "Cooperative Agreement Terms and Conditions of Award." This RFA is a one-time solicitation. Any future unsolicited, competing continuation application based on this project will compete with all investigator- initiated applications and will be reviewed according to the customary peer review procedures. The anticipated award date is 9/01/2002. AHRQ is not using the Modular Grant Application and Award Process. FUNDS AVAILABLE AHRQ intends to commit approximately $250,000 in total costs in FY 2002 to fund one grant in response to this RFA. Commensurate with the time line of the "Rewarding Results" initiative, it is anticipated that AHRQ applicants will request project periods of four years. Total costs over the life of the project should not exceed $1,500,000. Although the financial plans of AHRQ provide support for this program, awards pursuant to this RFA are contingent upon the availability of funds and the receipt of a sufficient number of meritorious applications. Funding beyond the first year is contingent on the availability of funds and satisfactory progress. ELIGIBLE INSTITUTIONS You may submit (an) application (s) if your institution has any of the following characteristics: o For-profit or non-profit organizations o Domestic and foreign o Public and private non-profit institutions, such as universities, clinics, colleges, and hospitals o Units of State and local governments o Faith-based organizations o Eligible Agency of the Federal Government Under recently enacted reauthorization legislation, AHRQ is authorized to enter into cooperative agreements with for-profit organizations as well as with public and not-for-profit entities. Thus, for-profit organizations may respond to this notice with research applications for cooperative agreements. Such applications will be administered in accordance with Subpart E of 45 CFR Part 74 and 42 CFR Part 67 Subpart A (except we note that the latter regulation has not yet been amended to reflect changes in Agency authority and Agency name.) (See December 6, 1999, AHRQ reauthorization at http://www.ahrq.gov/hrqa99a.htm). INDIVIDUALS ELIGIBLE TO BECOME PRINCIPAL INVESTIGATORS Any individual with the skills, knowledge, and resources necessary to carry out the proposed research is invited to work with their institution to develop an application for support. Individuals from under represented racial and ethnic groups as well as individuals with disabilities are always encouraged to apply for AHRQ programs. SPECIAL REQUIREMENTS Cooperative Agreement Terms and Conditions of Award The following special Terms of Award are in addition to and not in lieu of otherwise applicable OMB administrative guidelines, HHS grant administration regulations at 45 CFR Parts 74 and 92, 42 CFR Part 67 Subpart A, and other HHS, PHS grants administration policy statements. Applicants should be familiar with the Agency"s grant regulations, 42 CFR Part 67 Subpart A, and particularly sections 67.18-67.22. The cooperative agreement (U01) is an "assistance" mechanism (rather than an "acquisition" mechanism) in which substantial AHRQ scientific and/or programmatic involvement with the awardee is anticipated during performance of the activity. Under the cooperative agreement, the AHRQ purpose is to support and/or stimulate the recipient"s activity by involvement in and otherwise working jointly with the award recipient in a partner role, but it is not to assume direction, prime responsibility, or a dominant role in the activity. In the context of this initiative, cooperative activities are intended to strengthen individual demonstration evaluation activities and at the same time, generate generalizable results across sites and tasks in the overall evaluation project. Consistent with the concept of cooperative agreements, also described under MECHANISM OF SUPPORT, the dominant role and prime responsibility for project activity resides with the awardee for the project as a whole, although specific tasks and activities in carrying out the study will be shared among the awardee, the AHRQ project officer, and others. 1. Cooperative Activities The Agency for Healthcare Research and Quality, the Robert Wood Johnson Foundation, the California Health Care Foundation, the National Health Care Purchasing Institute, and the Center for Health Care Strategies will all have substantial scientific and programmatic involvement during the conduct of the demonstrations and their evaluation. These entities will provide technical assistance, advice, and support to all awardees, including the evaluation team, and guide the cooperative work of the projects. 2. Awardee Rights and Responsibilities The awardee will conduct evaluation and research in accordance with the terms and conditions of the Notice of Grant Award, and cooperate with other key parties named above. The Principal Investigator will participate with the AHRQ Project Officer, NHCPI, RWJF Program Officers, representatives of the RWJF "Rewarding Results" National Advisory Committee, other funders, and other technical assistance partners in the EAC, which will serve as the core mechanism for cooperative planning and coordination of the evaluation. The Principal Investigator will participate in EAC meetings. He or she will refine and revise, when necessary, aspects of the evaluation design and methodological procedures in accord with plans developed collaboratively in the context of the EAC. 3. AHRQ Staff Responsibilities The AHRQ Project Officer is responsible for coordination of the work of the evaluation awardee to ensure that the evaluation will complement the demonstration efforts and maximizes the generalizability of findings. The AHRQ Project Officer will arrange and coordinate EAC meetings. Facilities will be arranged by the AHRQ Project Officer in consultation with EAC. The AHRQ Project Officer will maintain contact with the Principal Investigator between EAC meetings to consult on key project decisions to trouble-shoot problems and confirm that the project is proceeding according to plan. The progress of work of the AHRQ awardee will be reviewed annually in the context of applications for continued funding of the project. 4. Advisory Committee The Evaluation Advisory Committee will meet four times a year in the Washington, D.C. area. 5. Arbitration Awards may be terminated in cases where the awardee has been given adequate notification about performance and fails to take corrective actions, including cases of documented under-performance, cases where there is a lack of participation in collaborative activities, and in cases involving human subject ethical issues. Any disagreement that may arise on scientific/programmatic matters (within the scope of the award), between the award recipient and AHRQ may be brought to arbitration. An arbitration panel will be composed of three members -- one selected by the individual awardee, a second member selected by AHRQ, and the third member selected by the two prior selected members. This special arbitration procedure in no way affects the awardee"s right to appeal an adverse action that is otherwise appealable in accordance with the PHS regulations at 42 CFR Part 50, Subpart D and HHS regulation at 45 CFR Part 16. Publication Transmittal: General AHRQ Requirements In keeping with the Agency"s efforts to translate the results of AHRQ- funded research into practice and policy, grantees and/or contractors are to inform the AHRQ Office of Health Care Information (OHCI) when articles from their studies are accepted for publication in the professional literature. Grantees and contractors should also discuss any ideas about other dissemination and marketing efforts with OHCI staff. The goal is to ensure that efforts to disseminate research findings are coordinated with other Agency activities to maximize awareness and application of the research by potential users, including clinicians, patients, health care systems and purchasers and policymakers. This is critical when outreach to the general and trade press is involved. Contact with the media will take place in close coordination with OHCI and the press offices of the awardee"s institution. In cases when products are created (such as annual or final reports, Web-based tools, CD-ROMs), grantees and contractors will be asked to submit to OHCI a brief plan describing how the product will be publicized. An OHCI staff person will be assigned to each product and will coordinate the implementation of the plan, especially issues related to printing and electronic dissemination, and outreach to the media. WHERE TO SEND INQUIRIES We encourage your inquiries concerning this RFA and welcome the opportunity to answer questions from potential applicants. Inquiries may fall into three areas: scientific/research, peer review, and financial or grants management issues: o Direct your questions regarding programmatic issues, including information on the inclusion of women, minorities, and children in study populations to: Michael Hagan Center for Organization and Delivery Studies Agency for Healthcare Research and Quality 2101 East Jefferson Street, 6W37 Rockville, MD 20852 Telephone: (301) 594-6818 FAX: (301) 594-2314 Email: [email protected] o Direct your questions about peer review issues to: Pat Thompson, Ph.D. Director, Division of Scientific Review Office of Research Review, Education and Policy Agency for Healthcare Research and Quality 2101 E. Jefferson Ave., Suite 401 Rockville, MD 20852 Telephone: (301) 594-1404 FAX: (301) 594-0154 Email: [email protected] o Direct your questions about financial or grant management matters to: George (Skip) Moyer Grants Management Specialist Division of Grants and Contracts Management Agency for Healthcare Research and Quality 2101 East Jefferson Street, 6W23 Rockville, MD 20852 Telephone: (301) 594-1842 FAX: (301) 594-3210 Email: [email protected] LETTER OF INTENT Prospective applicants may submit a letter of intent that includes the following information: o Descriptive title of the proposed research o Name, address, and telephone number of the Principal Investigator o Names of other key personnel o Participating institutions o Number and title of this RFA Although a letter of intent is not required, is not binding, and does not enter into the review of a subsequent application, the information that it contains allows AHRQ staff to estimate the potential review workload and plan the review. The letter of intent may to be sent at any time prior to submission of the application. The letter of intent should be sent to: Michael Hagan Center for Organization and Delivery Studies Agency for Healthcare Research and Quality 2101 East Jefferson Street, 6W37 Rockville, MD 20852 Telephone: (301) 594-6818 FAX: (301) 594-2314 Email: [email protected] AHRQ Data If applicable, AHRQ encourages research applications that will use data from the Medical Expenditure Panel Survey, or MEPS http://www.ahrq.gov/data/mepsix.htm the Healthcare Cost and Utilization Project, or HCUP-3 http://www.ahrq.gov/data/hcup/ and other AHRQ sources. MEPS is a rich data source for healthcare utilization, expenditure and insurance information. MEPS directly links data about persons and their families with information obtained from their employers, insurers and healthcare providers. It is the third in a series of nationally representative surveys of medical care use and expenditures in the U.S.. Unlike its predecessors, MEPS is an ongoing survey. MEPS collects data on the specific health services that American use, how frequently they use them, the cost and source of payment for services, and information on the types and costs of private health insurance held by and available to the U.S. population. It provides a foundation for estimating the impact of changes in sources of payment and insurance coverage on different economic groups or special populations of interest, such as the poor, elderly, uninsured, and racial and ethnic minorities. Current information on the availability of MEPS data is on the MEPS section of the AHRQ Web site (http://www.ahrq.gov). The HCUP includes databases covering 1988 -1999, with 2000 data available in 2002. These all-payer databases were created through a Federal- State-industry partnership to build a multistate healthcare data system. The main HCUP databases contain discharge-level information for inpatient hospital stays in a uniform format with privacy protections. The Nationwide Inpatient Sample (NIS) is a nationwide probability sample of about 1000 hospitals. The State Inpatient Databases (SID) contain inpatient records for all community hospitals in 29 states. Other HCUP databases contain ambulatory surgery data from 15 states. These databases can be directly linked to county- level data form the Health Resources and Services Administration"s Area Resource File and to hospital-level data from the Annual Survey of the American Hospital Association. SUBMITTING AN APPLICATION Applications must be prepared using the PHS 398 research grant application instructions and form (rev. 5/2001). The PHS 398 is available at http://grants.nih.gov/grants/funding/phs398/phs398.html in an interactive format. For further assistance contact GrantsInfo, Telephone 301-710-0267, Email: [email protected]. AHRQ is not using the Modular Grant Application and Award Process. Applicants for funding from AHRQ should ignore application instructions concerning the Modular Grant Application and Award Process, and prepare applications according to instructions provided in form PHS 398 . Applications submitted in the Modular format will be returned without review. USING THE RFA LABEL: The RFA label available in the PHS 398 (rev. 5/2001) application form must be affixed to the bottom of the face page of the application. Type the RFA number on the label. Failure to use this label could result in delayed processing of the application such that it may not reach the review committee in time for review. In addition, the RFA title and number must be typed on line 2 of the face page of the application form and the YES box must be marked. The RFA label is also available at: http://grants.nih.gov/grants/funding/phs398/label-bk.pdf. SENDING AN APPLICATION TO NIH and AHRQ: Submit a signed, typewritten original of the application, including the checklist, and three signed photocopies in one package to: Center for Scientific Review National Institutes of Health 6701 Rockledge Drive, Room 1040 - MSC 7710 Bethesda, MD 20892-7710 (20817 for express/courier service) At the time of submission, two additional copies of the application, labeled "Advanced Copy (s)" must also be sent to: Michael Hagan Center for Organization and Delivery Studies Agency for Healthcare Research and Quality 2101 East Jefferson Street, 6W37 Rockville, MD 20852 Telephone: (301) 594-6818 FAX: (301) 594-2314 APPLICATION PROCESSING: Applications must be received by the receipt date listed in the heading of this RFA. If an application is received after that date, it will be returned to the applicant without review. The CSR and AHRQ will not accept any application in response to this RFA that is essentially the same as one currently pending initial review unless the applicant withdraws the pending application. The CSR and AHRQ will not accept any application that is essentially the same as one already reviewed. This does not preclude the submission of substantial revisions of applications already reviewed, but such applications must include an Introduction addressing the previous critique. Applicants are encourage to read all PHS Forms 398 instructions prior to preparing an application in response to this RFA. The PHS 398 type size requirements (p.6) will be enforced rigorously and non-compliant applications will be returned. State and local government applicants may use PHS 5161-1, Application for Federal Assistance (rev. 5/96), and follow those requirements for copy submission. Prior to IRB Approval Beginning with applications for AHRQ submitted for the February 1, 2001 receipt date, Institutional Review Board (IRB) approval of human subjects is not required prior to peer review of an application unless otherwise indicated by the Agency (http://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html.) All investigators/applicants proposing research involving human subjects should pay particular attention to the instructions in the form PHS 398 regarding human subject involvement. AHRQ program staff listed under INQUIRIES should be notified of approvals as soon as possible, if approvals occur after grant submission. The "AHRQ Revised Policy for IRB Review of Human Subjects Protocols in Grant Applications" was published in the NIH Guide on September 27, 2000. (http://grants.nih.gov/grants/guide/notice- files/not-hs-00-003.html). The RFA is also available on AHRQ"s Web site, http://www.ahrq.gov, (under Funding Opportunities) and through AHRQ InstantFAX at (301) 594-2800. To use InstantFAX, you must call from a facsimile (FAX) machine with a telephone handset. Follow the voice prompt to obtain a copy of the table of contents, which has the document order number (not the same as the RFA number). The RFA will be sent at the end of the ordering process. AHRQ InstantFAX operates 24 hours a day, 7 days a week. For comments or problems concerning AHRQ InstantFax, please call (301) 594-6344. AHRQ expects grant recipients to keep the Agency informed of publications as well as the known uses and impact of their Agency- sponsored research. Applicants (or grant recipients) are to agree to notify AHRQ immediately when a manuscript based on research supported by the grant is accepted for publication, and to provide the expected date of publication as soon as it is known, regardless of whether or not the grant award is still active. Additionally, in carrying out its stewardship of research programs, the AHRQ, at some point in the future, may begin requesting information essential to an assessment of the effectiveness of Agency research programs. Accordingly, grant recipients are hereby notified that they may be contacted after the completion of awards for periodic updates on publications resulting from AHRQ grant awards, and other information helpful in evaluating the impact of AHRQ-sponsored research. To receive an award, applicants must agree to submit an original and 2 copies of an abstract, executive summary, and full report of the research results in the format prescribed by AHRQ no later than 90 days after the end of the project period. The executive summary should be sent at the same time on a computer disk which specifies on the label the format used (WP5.1 or WP6.0 is preferable). Application Preparation (for Using Center for Medicare and Medicaid Services (CMS) Data) For applications that propose to use Medicare and Medicaid data that are individually identifiable, applicants should state explicitly in the Research Design and Methods section of the Research Plan (form 398) the specific files, time periods, and cohorts proposed for the research. In consultation with the Center for Medicare and Medicaid Services (CMS), formerly Health Care Financing Administration (HCFA), AHRQ will use this information to develop a cost estimate for obtaining the data. This estimate will be included in the estimated total cost of the grant at the time funding decisions are made. To avoid double counting, applicants should not include the cost of the data in the budget. Applicants should be aware that for individually identifiable Medicare and Medicaid data, the Principal Investigator and his or her grantee institutions will be required to enter into a Data Use Agreement (DUA) with CMS to protect the confidentiality of data in accordance with standards set out in OMB Circular A-130, Appendix III-Security of Federal Automated Information Systems. The use of the data is restricted to the purposes and time period specified in the DUA. At the end of this time period, the grantee is required to return the data to CMS or certify that the data have been destroyed. For the sole purpose of assuring that data confidentiality is maintained, included in the DUA is the requirement that the User agrees to submit to CMS, a copy of all findings within 30 days of making such findings. The user agrees not to submit these findings to any third party (including but not limited to any manuscript to be submitted for publication) until receiving CMSs approval to do so. Grantees must also comply with the confidentiality requirements of Section 924(c) of the PHS Act. See the Data Confidentiality section below for details on these requirements as well as references to Circular A-130 and its implementation guides from the National Institute of Standards and Technology. In developing research plans, applicants should allow time for refining, approving and processing their data requests. Requests may take six months from the time they are submitted to complete. Applications proposing to contact beneficiaries or their providers require the approval of the CMS administrator and may require meeting(s) with CMS staff. CMS data are provided on IBM mainframe tapes using the record and data formats commonly employed on these computers. Applicants should either have the capability to process these tapes and formats or plan to make arrangements to securely convert them to other media and formats. Questions regarding CMS data should be directed to the AHRQ program official listed under INQUIRIES. PEER REVIEW PROCESS Upon receipt, applications will be reviewed for completeness and responsiveness to the RFA. Incomplete and/or non-responsive applications will be returned to the applicant without further consideration. Applications that are complete and responsive to the RFA will be evaluated for scientific and technical merit by an appropriate peer review group convened in accordance with standard AHRQ peer review procedures (codified at 42 CFR Part 67). As part of the merit review, all applications will: o Receive a written critique o Undergo a process in which only those applications deemed to have the highest scientific merit will be discussed and assigned a priority score. REVIEW CRITERIA The goal of this AHRQ-supported project is the evaluation of demonstration projects supported under the RWJF "Rewarding Results" initiative in order to provide important, accurate, and timely information to decision makers about how pragmatic changes in the nature of payment and non-financial incentives may be linked to desirable changes in the quality of health care. In their written comments, reviewers will be asked to discuss the following aspects of your application in order to judge the likelihood that the proposed evaluation and research project will have substantial impact on pursuit of this goal: o Significance o Approach o Innovation o Investigator o Environment The scientific review group will address and consider each of these criteria in assigning your application"s overall score, weighting them as appropriate for each application. It is not necessary that your application be especially strong in all categories to be judged likely to have a major scientific impact and thus deserve a high priority score. In particular, you may propose to carry out your project in a straightforward and standard manner that, while not innovative, very appropriately addresses the most significant aspects of the evaluation and research issues involved under this particular initiative. (1) SIGNIFICANCE: Does the proposed evaluation and research address the most significant aspects of relationships among payment, non- financial incentives, and health care quality improvement? If the articulated aims of the evaluation and research are achieved, will scientific and practical knowledge about the relationships among payment, non-financial incentives, and health care quality be advanced in an important way? (2) APPROACH: Are the conceptual framework, design, methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Have all aspects of data collection and management for the project been considered, with appropriate attention to the quality and timeliness of data collection? Will data management and analysis ensure that conclusions are valid? Do you acknowledge potential problem areas and consider alternative tactics? Is the proposed evaluation approach flexible enough to account for variance across demonstration projects in their specific changes in payment and in non-financial incentives, in their quality goals and performance measures, etc.? (3) INNOVATION: Does your project employ novel concepts, approaches or methods? Are the aims original and innovative? Does your project challenge existing paradigms or develop new methodologies or technologies? (4) INVESTIGATOR: Are you appropriately trained and well suited to carry out this work? Is the work proposed appropriate to your experience level as the Principal Investigator and to that of other researchers (if any)? Are the numbers and roles of staff for the study defined and justified? Do the investigators have experience with identifying and applying appropriate evaluation methodologies and research designs, especially in the context of health care payment, non-financial incentive systems, and health care quality improvement? Do the investigators have experience working with governmental agencies in the performance of evaluation and research studies? Do the investigators have appropriate experience in the design and implementation of qualitative methods? (5) ENVIRONMENT: Does the study environment in which your work will be done contribute to the probability of success? Do the proposed analyses take advantage of unique features of the environment or employ useful collaborative arrangements? Is there evidence of institutional support and capacity to undertake the proposed project? ADDITIONAL REVIEW CRITERIA: In addition to the above criteria, your application will also be reviewed with respect to the following: o PROTECTIONS: The adequacy of the proposed protection for humans or the environment, to the extent they may be adversely affected by the project proposed in the application. o INCLUSION: The adequacy of plans to include subjects from both genders, all racial and ethnic groups (and subgroups), and children as appropriate for the scientific goals of the research. Plans for the recruitment and retention of subjects will also be evaluated. (See Inclusion Criteria included in the section on Federal Citations, below) o BUDGET: The reasonableness of the proposed project budget and the requested period of support in relation to the proposed research. DATA SHARING Data Confidentiality Pursuant to section 924(c) of the Public Health Service Act (42 USC 299c-3(c)), information obtained in the course of any AHRQ-study that identifies an individual or entity must be treated as confidential in accordance with any promises made or implied regarding the possible uses and disclosures of such data. In the Human Subjects section of the application, applicants must describe procedures for ensuring the confidentiality of the identifying information to be collected. The description of the procedures should include a discussion of who will be permitted access to the information, both raw data and machine readable files, and how personal identifiers and other identifying or identifiable data will be restricted and safeguarded. The awardee should ensure that computer systems containing confidential data have a level and scope of security that equals or exceeds those established by the Office of Management and Budget (OMB) in OMB Circular No. A-130, Appendix III - Security of Federal Automated Information Systems. The National Institute of Standards and Technology (NIST) has published several implementation guides for this circular. They are: An Introduction to Computer Security: The NIST Handbook, Generally Accepted Principals and Practices for Securing Information Technology Systems, and Guide for Developing Security Plans for Information Technology Systems. The circular and guides are available on the web at http://csrc.nist.gov/publications/nistpubs/800-12/. The application of these confidentiality and security standards to subcontractors and vendors, if any, should be addressed. Rights in Data AHRQ grantees may copyright or seek patents, as appropriate, for final and interim products and materials including, but not limited to, methodological tools, measures, software with documentation, literature searches, and analyses, which are developed in whole or in part with AHRQ funds. Such copyrights and patents are subject to a worldwide irrevocable Federal government license to use and permit others to use these products and materials for government purposes. In accordance with its legislative dissemination mandate, AHRQ purposes may include, subject to statutory confidentiality protections, making research materials, data bases, results, and algorithms available for verification or replication by other researchers, and subject to AHRQ budget constraints, final products may be made available to the health care community and the public by AHRQ or its agents, if such distribution would significantly increase access to a product and thereby produce public health benefits. Ordinarily, to accomplish distribution, AHRQ publicizes research findings but relies on grantees to publish research results in peer-reviewed journals and to market grant-supported products. Important legal rights and requirements applicable to AHRQ grantees are set out or referenced in the AHRQ"s grants regulation at 42 CFR Part 67, Subpart A (Available in libraries and from the GPO"s Website http://www.access.gpo.gov/nara/cfr/index.html). RECEIPT AND REVIEW SCHEDULE Letter of Intent Receipt Date: April 20, 2002 Application Receipt Date: May 20, 2002 Peer Review Date: July/August 2002 Earliest Anticipated Start Date: September 1, 2002 AWARD CRITERIA Award criteria that will be used to make award decisions include: o Scientific merit (as determined by peer review) o Availability of funds o Programmatic priorities REQUIRED FEDERAL CITATIONS INCLUSION OF WOMEN, MINORITIES, AND CHILDREN IN RESEARCH STUDY POPULATIONS: It is the policy of AHRQ that women and members of minority groups be included in all AHRQ-supported research projects involving human subjects, unless a clear and compelling rationale and justification are provided that inclusion is inappropriate with respect to the health of the subjects or the purpose of the research. All investigators proposing research involving human subjects should read the UPDATED "NIH Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research," published in the NIH Guide for Grants and Contracts on August 2, 2000 (http://grants.nih.gov/grants/guide/notice-files/not-od-00-048.html). A complete copy of the updated Guidelines is available at http://grants.nih.gov/grants/funding/women_min/guidelines_update.htm. To the extent possible, AHRQ requires adherence to these NIH Guidelines. Investigators may obtain copies from the above sources or from the AHRQ Publications Clearinghouse, listed under INQUIRIES, or from the NIH Guide Web site http://grants.nih.gov/grants/guide/index.html. AHRQ Program staff may also provide additional information concerning these policies (see INQUIRIES). AHRQ also encourages investigators to consider including children in study populations, as appropriate. PUBLIC ACCESS TO RESEARCH DATA THROUGH THE FREEDOM OF INFORMATION ACT: The Office of Management and Budget (OMB) Circular A-110 has been revised to provide public access to research data through the Freedom of Information Act (FOIA) under some circumstances. Data that are (1) first produced in a project that is supported in whole or in part with Federal funds and (2) cited publicly and officially by a Federal agency in support of an action that has the force and effect of law (i.e., a regulation) may be accessed through FOIA. It is important for applicants to understand the applicability of the amendment. NIH has provided guidance at http://grants.nih.gov/grants/policy/a110/a110_guidance_dec1999.htm. It is not likely that data gathered under projects supported through this initiative will be used as a basis for federal regulation or action having the force and effect of law, but it is possible. Were that the case, the OMB Circular would then require disclosure of data underlying the evidence upon which Federal policy decisions were made, if someone requested this data under the Freedom of Information Act. However, the above-cited AHRQ confidentiality statute protects individually identifiable data and limits disclosures to those consented to by the subjects or suppliers of consent, underlying data gathered for this evaluation may only be made available in nonidentifiable form, aggregated to prevent identification of subjects or with all identifying information concealed. Should applicants wish to place data collected under this RFA in a public archive, which can provide protections for the data (e.g., as required by the confidentiality statute applicable to AHRQ supported projects, 42 U.S.C. 299c-3c) and manage the distribution of non- identifiable data for an indefinite period of time, they may. The application should include a description of any archiving plan in the study design and include information about this in the budget justification section of the application. In addition, applicants should think about how to structure informed consent statements and other human subjects procedures given the potential for wider use of data collected under this award. HEALTHY PEOPLE 2010: The Public Health Service (PHS) is committed to achieving the health promotion and disease prevention objectives of "Healthy People 2010," a PHS-led national activity for setting health improvement priorities for the United States. AHRQ encourages applicants to submit grant applications with relevance to the specific objectives of this initiative. Potential applicants may obtain a copy of "Healthy People 2010" at http://www.health.gov/healthypeople. AUTHORITY AND REGULATIONS: This program is described in the Catalog of Federal Domestic Assistance, Number 93.226. Awards are made under Title IX of the Public Health Service Act (42 USC 299-299c-7) as amended by P.L. 106-129 (1999). Awards are administered under the PHS Grants Policy Statement and Federal Regulations 42 CFR 67, Subpart A, and 45 CFR Parts 74 or 92. This program is not subject to the intergovernmental review requirements of Executive Order 12372 or Health Systems Agency review. The PHS strongly encourages all grant and contract recipients to provide a smoke-free workplace and promote the non-use of all tobacco products. In addition, Public Law 103-227, the Pro-Children Act of 1994, prohibits smoking in certain facilities (or in some cases, any portion of a facility) in which regular or routine education, library, day care, health care or early childhood development services are provided to children. This is consistent with the PHS mission to protect and advance the physical and mental health of the American people. REFERENCES Crossing the Quality Chasm: A New Health System for the 21st Century, Committee on Quality of Health Care in America, Institute of Medicine, Washington, D.C., National Academy Press, 2001. Rossi, Peter H. and Freeman, Howard E., Evaluation: A Systematic Approach--5, Newbury Park, California, Sage Publications, 1993. Qualitative Methods in Health Services Research (A Special Supplement), Kelly Devers, Shoshanna Sofaer, and Thomas Rundall, editors, Health Services Research, December 4, 1998.
Weekly TOC for this Announcement
NIH Funding Opportunities and Notices
|
| ||||||
|
|
|
Department of Health and Human Services (HHS) |
|
|||
|
NIH... Turning Discovery Into Health® |
||||||