TRANSLATING RESEARCH INTO PRACTICE II Release Date: December 16, 1999 RFA: HS-00-008 Agency for Healthcare Research and Quality (formerly AHCPR) Letter of Intent Receipt Date: March 10, 2000 Application Receipt Date: April 27, 2000 PURPOSE The Agency for Healthcare Research and Quality (AHRQ), formerly know as the Agency for Health Care Policy and Research (AHCPR), invites applications to conduct cooperative agreement demonstration projects which specifically focus on evaluating strategies for translating research into practice through the development of partnerships between researchers and health care systems and organizations (e.g., purchaser groups, integrated health service delivery systems, academic health systems, managed care programs including HMOs, practice networks, worksite clinics). The objective of such partnership arrangements is to help accelerate and magnify the impact of the research on clinical practice and patient outcomes in applied settings. The intent of the RFA is to support the evaluation of currently planned or recently initiated programs whose goal is to improve the outcomes, quality, effectiveness, efficiency and/or cost-effectiveness of health care based on findings derived from sound research. A second goal of the RFA is to demonstrate that the translation of research into practice leads to measurable and sustainable improvements in health care. The presence of strong partnership arrangements is essential to help accelerate and magnify the impact of the research on clinical practice and patient outcomes in applied settings. This RFA builds on the Translating Research Into Practice (TRIP) I and Assessment of Quality Improvement Strategies in Health Care RFAs by focusing on health care systems and organizations and supporting the development of sustainable and replicable models and tools for translating research into practice to improve the quality, outcomes, effectiveness, efficiency and cost-effectiveness of health care. Applicants will be required to address one or more conditions of high national priority based on a specified set of criteria: conditions/settings where the most improvement can occur, conditions/settings where wide variability in practice currently exists, conditions that are common and/or costly, conditions/settings for which wide disparities in care exist for racial/ethnic minorities, and conditions which account for a large burden of disease and poor quality of life. Of particular interest are interventions that use the strengths of information systems for implementing evidence-based strategies for health care improvement. A funding priority for this RFA will be a focus on at least one of the six specified areas of the President=s Race and Disparities Initiative (infant mortality, cancer screening and management, cardiovascular disease, diabetes, HIV infection/AIDS, and child and adult immunizations), mental health and pediatric asthma. In addition, a modest amount of funds will be set aside for applicants who have been funded by AHRQ (under an existing or prior grant or contract, e.g., PORTs, PORT-IIs, and other research projects), but have not implemented strategies for translating their findings into practice on a large scale. This set-aside will support partnerships with national organizations (e.g., professional organizations, patient organizations, purchasing coalitions, multi-state health care systems, or others) to disseminate and implement the findings. The projects should focus on applied research with the objective of developing sustainable and reproducible strategies to translate research into practice effectively and efficiently. Towards this goal, it is expected that successful applicants will evaluate their implementation programs using appropriate concurrent comparison groups, document the costs and resources associated with the programs, identify key components of the model and organizational conditions that resulted in successful implementation, and evaluate transferability to other settings. A clear measure of success will be that participating health care organizations absorb successful models as a continuing initiative beyond the end of the project. It is expected that the health care organization will provide in- kind contributions during the funded period of the project and provide evidence for the sustainability of the intervention after the period of the grant. In addition to the project-specific evaluation carried out by the grantee, an external organization will carry out an evaluation across all of the funded projects. Its focus will include examinations of the sustainability of efforts and the likelihood of their replication. One of the most important goals of the Agency in implementing the TRIP-II initiative is to facilitate collaboration and partnerships between organizations or entities with the responsibility for providing or purchasing health care and the clinical and health services research communities to improve the quality, efficiency, effectiveness and/or cost- effectiveness of health care. HEALTHY PEOPLE 2000 The Public Health Service (PHS) is committed to achieving the health promotion and disease prevention objectives of "Healthy People 2000," a PHS-led national activity for setting health improvement priorities for the United States. AHRQ encourages applicants to submit grant applications with relevance to the specific objectives of this initiative. Potential applicants may obtain a copy of "Healthy People 2000" (Full Report: Stock No. 017-001-00474-0) or "Healthy People 2000" (Summary Report: Stock No. 017-001-00473-1) through the Superintendent of Documents, Government Printing Office, Washington, DC 20402-9325 (telephone 202-512-1800). ELIGIBILITY REQUIREMENTS Applications may be submitted by domestic or foreign, public or private organizations, including American Indian/Alaska Native organizations, universities, clinics, units of State, Tribal and local governments, and eligible agencies of the Federal Government. Under recently enacted reauthorization legislation, AHRQ is authorized to enter into cooperative agreements with for-profit organizations as well as with public and not- for-profit entities. Thus, for-profit organizations are invited to respond to this notice with research applications for cooperative agreements. Such applications will be administered in accordance with Subpart E of 45 CFR Part 74 and 42 CFR Part 67 Subpart A. The latter regulation will be amended in the near future to reflect this change in Agency name and authority. AHRQ encourages investigators and new investigators (as defined in the PHS 398 application instructions) who are women, members of minority groups and persons with disabilities to apply as Principal Investigators. MECHANISM OF SUPPORT The administrative and funding instrument to be used for this program will be a cooperative agreement (U18), an "assistance" mechanism (rather than an "acquisition" mechanism), in which substantial AHRQ scientific and/or programmatic involvement with the awardee is anticipated during performance of the activity. Under the cooperative agreement, the AHRQ purpose is to support and/or stimulate the recipient's activity by involvement in and otherwise working jointly with the award recipient in a partner role, but it is not to assume direction, prime responsibility, or a dominant role in the activity. Details of the responsibilities, relationships and governance of the study to be funded under cooperative agreement(s) are discussed later in this document under the section ATerms and Conditions of Award. A The total project period for applications submitted in response to the present RFA may not exceed 3 years. The anticipated award date is September 30, 2000. This RFA is a one-time solicitation. AHRQ has not determined whether or how this solicitation will be continued beyond the present RFA. FUNDS AVAILABLE AHRQ expects to award up to $7.0 million in total costs (direct costs plus facilities and administrative costs) in fiscal year 2000 to support the first year of approximately 12-18 projects under this RFA. The actual number of applications funded is dependent on the number of high quality applications and their individual budget requirements; it is not the intent of AHRQ that the awards be equal in size. The expected size of each award is $300,000-1,500,000 in total annual costs. From the $7.0 million, the Agency will allocate up to $3.5 million for projects that translate research findings to improve the quality of care for minority populations, up to $1.2 million for the translation of research on pediatric asthma into practice and up to $1.05 million for implementation of previous AHRQ research (from an existing or prior grant or contract) findings on a national level. Applicants for set-aside funds should explicitly note their intent to have a specific project considered for these funds. For the purposes of this RFA, minority populations refer to racial and ethnic groups for whom disparities in health care and outcomes have been documented compared with the United States non-minority population. These include Blacks/African Americans, Hispanic Americans, American Indians, Alaska Natives, Asian Americans and Pacific Islanders. Also, applicants are encouraged to focus on pediatric asthma or one or more of the six clinical areas identified in the President=s Initiative on Race and the U.S. Department of Health and Human Services= Initiative to Eliminate Racial and Ethnic Disparities in Health: infant mortality, cancer screening and management, cardiovascular disease, diabetes mellitus, HIV infection and AIDS, and immunizations for both children and adults. (See http://raceandhealth.hhs.gov/ for further information on these initiatives.) Pediatric asthma refers to the care of patients under 18 years of age. Applicants who are interested in focusing on other conditions should justify their choice with respect to the following criteria: conditions/settings where most improvement can occur, conditions/settings where wide variability in practice exists, conditions that are common and/or costly, conditions/settings for which wide disparities in care exist for racial/ethnic minorities, and conditions which account for a large burden of disease and poor quality of life. Funding for implementation of AHRQ research findings will be targeted at the evaluation of large-scale dissemination and translation efforts for existing grants or previously funded AHRQ projects including, but not limited to, the Patient Outcomes Research Teams (PORTs), Evidence-Based Practice Center (EPC) evidence reports, the National Guideline ClearinghouseJ (http://www.guidelines.gov) and investigator-initiated research. Projects focused on improving care for minority populations may either focus exclusively on those populations or compare the relative effectiveness of alternative implementation strategies for different population subgroups. Although this program is provided for in the financial plans of AHRQ, awards pursuant to this RFA are contingent on the availability of funds for this purpose. Funding beyond the initial budget period will depend on annual progress reviews by AHRQ and the availability of funds. RESEARCH OBJECTIVES Background The translation of research findings into sustainable improvements in clinical practice and outcomes remains one of the largest hurdles in improving the quality, efficiency, effectiveness and cost-effectiveness of health care. For example, despite a plethora of randomized clinical trial data and national guidelines from the early 1990s supporting early administration of beta-blockers for acute myocardial infarction, McLaughlin and colleagues in 1996 documented that only 53 percent of eligible patients at 37 hospitals actually received beta-blockers (ref. #1). In additionally, these researchers estimated that 18,000 people die each year due, in part, to failure to receive beta-blockers after acute myocardial infarction (ref. #2). For pediatric asthma, despite findings that show that having a written management plan for parents lowers the risk of emergency room visits and hospitalizations, a community-based survey in inner city New York reported that only half of the children with asthma had received written instruction (ref. # 3 and 4). Similarly, in a study examining the conformance of current patterns care for persons with schizophrenia with treatment recommendations, it was found that the rates at which patients' treatment conformed to the recommendations were modest at best, generally below 50 percent (ref.# 5). More broadly, a recent review of published studies on the quality of care received by Americans found that approximately 60 percent of patients with chronic conditions received recommended care (ref. # 6). This translational hurdle exists despite a wide range of strategies for implementing research into practice: continuing medical education (CME), provider reminder systems, interactive educational meetings, audit and feedback, local opinion leaders, local consensus process, academic detailing, financial incentives, reimbursement arrangements, guidelines, organizational and structural changes, computer decision support systems, patient-mediated interventions, continuous quality improvement initiatives, and multifaceted interventions combining several strategies. It appears that different strategies work best in different contexts and are influenced by the health care setting, patient, administration and practitioner perceptions, and the desired behavior change. A recent review of these interventions concluded that there are no >magic bullets= for improving health care (ref. #7). Another systematic review of the effectiveness of health services interventions concluded that while most strategies are effective under some circumstances, none is effective under all circumstances (ref. #8). These realities compound the translational hurdle. For example, a recent paper reported that there may be up to seven unique barriers to guideline implementation that need to be addressed and overcome in order to obtain meaningful quality improvement via this strategy (ref. # 9). Despite the inherent difficulties in translating research into practice, the President=s Advisory Commission on Consumer Protection and Quality in the Health Care Industry, in its 1998 report, recommended a continual emphasis: AThe purpose of the health care system must be to continuously reduce the impact and burden of illness, injury and disability, and to improve the functioning of the people of the United States@ (ref. #10). The emphasis on the continuous nature of improving the quality of care speaks to the need for sustainable efforts based on evidence of effectiveness. Although there are published reports documenting the efficacy of implementation strategies, there remains a paucity of evidence regarding the long term sustainability, replicability and effectiveness in applied settings. The broad agreement of consumers, business, labor, health care providers, health plans, State and local governments, the Federal government, and health care quality experts on the need to accelerate the uptake of evidence-based high quality practice across the health care system is summarized by the Commission and the focus of this RFA: AMore complete and rapid dissemination of effective health care practices should be supported by encouraging public and private organizations to collaboratively share knowledge of innovations, synthesize evidence on effective health care practices, develop practice protocols, disseminate this information, and provide technical assistance to practitioners.@ Objectives and Scope External demands for improved health care through translation of research into practice have stimulated important but isolated examples of changes in organization and practice. There is broad agreement among purchasers, researchers, health plans and quality improvement organizations about an urgent need to accelerate the uptake of evidence-based findings and tools into practice, but there is considerable uncertainty about the most effective mechanisms for doing so. In addition, there is collective concern among all stakeholders that examples of success in translating research into practice are difficult to replicate or sustain over time. Projects funded under this Cooperative Agreement are expected to identify, refine and evaluate successful strategies for translation of research findings into practice such that the efforts are sustainable, can be rapidly transferred to or replicated in other settings, and can lead to demonstrable improvements in care quality, outcomes, efficiency, or cost- effectiveness. Sustainable efforts are those that continue beyond the period of funding support and will persist despite the rapid changes in staff and conditions that health care organizations are facing today. This RFA builds on the Translating Research into Practice (TRIP-I) RFA (https://grants.nih.gov/grants/guide/rfa-files/RFA-HS-99-003.html), published on January 8, 1999 and the Assessment of Quality Improvement Strategies in Health Care RFA (https://grants.nih.gov/grants/guide/rfa-files/RFA-HS-99-002.html), published on January 22, 1999. Projects funded under the first RFA (TRIP- I) address a variety of health care problems primarily through randomized controlled trials. TRIP II is aimed at applying and rigorously evaluating some of the techniques and methods developed in idealized practice settings or those based on theoretical constructs in research translation now being employed but never evaluated. The evaluation will serve to document the implementation techniques and identifying factors, such as organizational and clinical characteristics, associated with success. The assessment of translational efforts in institutions with a number of Acultures@ is of particular interest because of the heterogeneity within and across many organizations. Such institutions are those that consist of groups with varying approaches to practice organization, structure and policy because of their differences in size, mix of practitioners, staffing ratios, support staff, information technology, reimbursement and philosophy (e.g., solo practitioners, small group, large multi-specialty and hospital-based practices, and in-patient and community-based care). The success of TRIP II will depend, in part, on the presence or establishment of substantial relationships between health care systems and organizations, and researchers. The systems and organizations include but are not limited to purchaser groups, integrated health care delivery systems, large group practices, community health centers, managed care organizations, practice networks and worksite clinics. Such relationships will enable the rigorous evaluation of planned and ongoing implementation efforts which may not otherwise occur. These evaluations are necessary in determining whether research translation efforts are truly effective in applied settings, rather than being dependent on intuition and assumptions of success. For example, rigorous evaluation of dissemination-only strategies such as conferences and dissemination of unsolicited material demonstrated that, contrary to widespread belief, these techniques were ineffective (ref. # 11). Reciprocal relationships between researchers and decision-makers in health care organizations will potentially increase the impact of studies funded under this program by addressing and incorporating decision-makers= needs and ideas on practice improvement. It also indicates a level of commitment for the organizations to the study and its results, suggesting that the implementation strategies, if successful, will be sustained and replicated following the study=s completion. Finally, because decision-makers in different organizations often face similar challenges and communicate with each other, it is hoped that funded projects will together address how this program=s findings can spread rapidly across health care organizations and systems. Up to $3.5 million will be devoted to projects that translate research findings to improve the health care of minority populations. This RFA complements the Measures of Quality Care for Vulnerable Populations RFA (https://grants.nih.gov/grants/guide/rfa-files/RFA-HS-99-001.html), published on December 22, 1998 and the Understanding and Eliminating Minority Health Disparities RFA (https://grants.nih.gov/grants/guide/rfa-files/RFA-HS-00-003.html), published on October 20, 1999. The latter is aimed at strengthening the science base on which strategies can be developed to eliminate the disparities. In addition to evaluating the application of interventions proven to be effective in non minority populations to racial and ethnic minority populations, the TRIP II RFA may also use sub-group analyses to evaluate the impact of implementation methodologies on the health care and outcomes of minorities. Lessons learned from care of these groups may be applicable to the United States population as a whole. A particular interest for AHRQ in this RFA will be the use of information technology in translating research findings into policy and care. The potential for technology to address this need has long been recognized and has been developed to some extent but its use has yet to become widespread. Its use is attractive because the technology can be implemented rapidly, and may be more easily transferable and sustainable than implementation strategies that are dependent on individuals and personal interactions (ref. # 12). Such technology and systems may play an especially important part in transferring knowledge of Awhat works@ across organizations or separate components of a single organization, or enhancing the translation of research into practice for individuals whose care requires transitions across settings. While interest in translating research into practice on a national scale has intensified, successful health care improvement efforts have been largely local and infrequently replicated. As noted above, up to $1.05 million will be set aside to support partnerships between researchers and national organizations (e.g., professional organizations, patient organizations, purchasing coalitions, multi-state health care systems, multiple Quality Improvement Organizations, and others) for the purpose of translating AHRQ-funded research into practice. Applicants wishing to pursue this set-aside may do so in one of two ways: 1. develop a component of the overall research project that adopts and adapts strategies from their more focused effort for use by the project and supports further partnership with national partner organization(s) to foster implementation on a broader scale; or 2. start from existing models of effective translational efforts of research previously funded by AHRQ and replicate or transfer them to a national level. In either approach, applicants must develop a plan for measuring changes in care patterns at a national level as a result of the dissemination/replication strategy. Research Methods Because of this RFA=s major objective of refining and assessing sustainable and transferable strategies to implement research findings and evidence- based tools and information, study methods must be adequately rigorous, reflecting the research question and the evidence needs of organizations that might implement similar interventions, and ensuring internal and external validity, and reliability. Studies of clinical interventions will be expected to employ concurrent comparison groups rather than historical controls because of the threat that secular trends pose to internal validity, especially in this era of rapid changes in our health care system. Strategies for addressing selection bias among participants, such as randomization at the level of the patient, practice or other organizational unit, pseudo-randomization, or matching must be included. Studies of organizational and structural interventions, though less amenable to randomization and concurrent control groups, will also be expected to address challenges to internal validity. In developing their study methods, applicants should consider the types of research findings or evidence-based tools that others may need in order to decide whether and how to apply a certain intervention. To the extent possible, investigators are encouraged to focus on the primary endpoints of improvements in health and reduction in disease burden. Nevertheless, because many interventions to improve practice affect distal outcomes such as mortality, focus on proximal outcomes, in addition to provider knowledge and behavior, process measures, resource utilization, quality and patient satisfaction may be appropriate. Such evidence may be derived from studies ranging from intervention to description and include qualitative as well as quantitative results. Qualitative methods, including case studies, interviews and focus groups, may be valuable for identifying underlying reasons for the gap between the knowledge base and practice as well as factors associated with successful interventions. For example, in a study of the effectiveness of a computerized reminder system for prevention of complications associated with AIDS, potential users of the research may be interested in the structural and organizational characteristics of the practice settings, the expertise and attitudes of the providers, and the demographics of the patient population as well as the clinical and organizational processes and outcomes. Because of the multidimensional challenges involved in translating research into practice, research teams should include expertise from a variety of disciplines, from both clinical and social science fields. Assessment of cost and cost-effectiveness is of special importance. Sub-group analyses may be necessary to evaluate the effectiveness of interventions to improve care for racial and ethnic minorities. To the extent possible, applicants should submit proposals that build on available data, will generate early results, and are modest in terms of time, scale, and cost. The study of interventions to translate research findings into improving the care of children poses a number of challenges. These include the need for larger sample sizes because of the relative infrequency among children of some of the conditions and associated primary physical outcomes that are a focus of the Race and Disparities Initiative (e.g., infant mortality, cancer, diabetes, cardiovascular disease, HIV/AIDS); the relative paucity of validated methods for assessing case mix/disease severity and outcomes; the multi-layered system of health care for children (involving schools and public clinics as well as hospitals and physicians= offices); and the role of adult care givers in children=s access, outcomes and quality (ref. # 13). Potential Data Sources On June 17, 1999, a Request for Expressions of Interest (RFEI) was published in the Federal Register in order to identify health care systems and organizations that had developed or were planning to develop programs to translate research into practice in the six focus areas of the President=s Initiative to Eliminate Racial and Ethnic Disparities in Health, mental health and pediatric asthma, who did not have established relationships with a research team, and would be willing to partner with a research team to develop a proposal for this RFA. The objective of the RFEI was to facilitate development of relationships between researchers and health care systems and organizations for this as well as future efforts. Such relationships are instrumental to evaluate implementation in applied settings and to address research questions that may arise in the future. A list of organizations that responded to the RFEI and their locations is below: Alliance Continuing Care Network, New York, NY; American Speech-Language Hearing Association, Rockville, MD; Carolina Medical Review, Columbia, SC; Children=s National Medical Center, Washington, DC; Clinical Directors Network, New York, NY; Clinical Regional Advisory Network (CRAN), Inc., Kenbridge, VA; Florida Association of Homes for the Aging, Health and Services, Inc., Tallahassee, FL; Healthcare Education and Research Foundation, Inc., St. Paul, MN; High Plains Research Network, Denver, CO; Iowa Department of Public Health, Des Moines, IA; Office of Medicaid Management, New York State Department of Health, Albany, NY; Ramah Navajo School Board, Inc., Pine Hill, NM; Rural Wisconsin Health Cooperative, Sauk City, WI; U.S. Quality Algorithms, Inc. (Aetna U.S. Healthcare), Blue Bell, PA; Village Care of New York, Inc., New York, NY; and Medical Affairs Tenet Health System, Dallas, TX. Details including the organizations= contact information, and information about the patient population, provider settings, data collection capacity, data availability and specific conditions or populations of interest (if any) are listed at the following Web site: ( http://www.ahrq.gov/fund/partners.htm ). Investigators are also encouraged to develop or build on existing relationships with organizations and systems on their own. They are not required to work with RFEI respondents. Applicants are expected to demonstrate that they either routinely collect and maintain data necessary for such research, or have the capacity to collect such data within the time and budget limits of the cooperative agreement. The specific data required for any project will be determined by the study purpose. However, there are five general types of data that organizations may use to conduct research on translating evidence into practice and may already have available: 1) enrollment data (including some basic demographic data); 2) administrative claims or encounter-level data (e.g., diagnosis and procedure codes, charges); 3) more detailed clinical data, including data on laboratory and diagnostic tests, and medications; 4) survey data (e.g., satisfaction, health status etc.); and, 5) qualitative and quantitative data on characteristics of providers and organizations delivering care. The accuracy and completeness of these types of data and the extent to which they can be linked will influence their usefulness for research. Applicants must be able to identify the specific data that will be used or collected for the project, demonstrate that they are already available or can be collected, and describe how the data can be cleaned, linked, and developed into an analytic file capable of answering the study questions. Sample Study Questions Research questions should address how the implementation of research findings and evidence-based information in everyday health care affects the outcomes, quality, effectiveness, efficiency and/or cost-effectiveness of health care. Evidence-based information can be used as the basis for clinical and organizational policy, purchasing, and other managerial decisions. The effect of the use of research evidence at any of these levels of decision making is included in this solicitation. The evaluation should assess the effects in terms of changes in clinical behavior, quality of care, changes in outcomes and health status, and organizational changes. Outcome measures might include morbidity, functional ability, disease status, complications rates, health care costs and utilization, and consumer satisfaction. Consideration should be given to the effect of local circumstances such as specific populations, diverse health settings, resources constraints, and political context on both the implementation process and the outcomes of care that are achieved. Evidence-based interventions may include, but are not limited to, structural and organizational changes such as alterations in staffing and duties, changes to financial incentives, comprehensive quality management/improvement systems, computerized drug information and dosage, clinician reminders, audit and feedback methods, interactive systems to facilitate shared decision-making, computer systems to deliver educational materials at the point of clinical decision making, clinical practice guidelines and protocols, the introduction of new technologies, the limitation of types of technologies or equipment, and disease management systems. Settings include in-patient, acute care, primary care and specialty care, including physician offices, clinics, community nursing and the worksite, home health care, other community care sites and transitions between health care settings. Examples of research questions include: o Can integrating clinical practice guidelines for asthma management into an electronic medical record at the point of care reduce unexpected ED visits and improve patient outcomes for pediatric asthma? o Will providing Ajust in time@ evidence-based clinical practice guidelines in ambulatory clinic settings improve clinician compliance with recommendations and patient satisfaction, and improve outcomes? o Can clinical or organizational strategies known to be effective in one type of setting be transferred to different settings and/or conditions? o How can the use of evidence-based strategies enhance compliance throughout an entire health care system or region? o Can reassignment of roles improve rates of adult immunization and cancer screening? o What clinical and/or organizational components of disease management programs are most effective - and for which populations? o Can coupling of guidelines with computer decision support systems and provider reminder systems enhance implementation of known interventions to reduce the mortality rate of acute myocardial infarction? o What are the best ways to improve the adoption of efficacious mental health interventions across a variety of mental health service settings? o What strategies are most effective and/or cost-effective or cost- beneficial in increasing the appropriate use of screening mammography, particularly in older and minority populations? o Can patient kiosks focused on preventive care be used to improve rates of adult immunization and cancer screening in a racially and ethnically diverse patient population? o Will coverage for visits by home health nurses improve compliance with medications, environmental recommendations and self-management, and reduce emergency room visits for children with severe asthma? o Can admissions be reduced and patient compliance with heart failure recommendations be improved by training translators in elements of disease management in community health centers? o Are interactive educational meetings via the Internet and teleconferencing effective and cost-effective for the management of diabetes for geographically isolated populations such as Native Americans and Alaska Natives? In addition, studies may address how organizational research can be translated into practice, the impact of organizational variables on clinical translation and the organizational and structural context of successful interventions needed to facilitate replication. SPECIAL REQUIREMENTS Study Organization To promote the development of successful projects, some of which will be collaborative, multi-site, and multi-organizational, applicants should describe how their efforts will be organized and coordinated. Applicants should develop a time-line for project activities and include the percentage of effort for key staff, as part of the budget justification. The distribution of effort and funds across organizations should be clearly defined and justified with a discussion of the capabilities and strengths of each. Work with the Coordinating Center, Steering Committee and Evaluators AHRQ will house the Coordinating Center for these projects, which will be composed of representatives of three AHRQ Centers collaborating on this RFA: the Center for Outcomes and Effectiveness Research, the Center for Practice and Technology Assessment and the Center for Quality Measurement and Improvement. Other AHRQ Centers may be invited to join the Coordinating Center depending on the focuses of studies funded under this RFA. The Coordinating Center will undertake activities to strengthen individual studies, facilitate synergism between the studies and achieve the major objectives of this initiative: to accelerate and magnify the impact of research to implement evidence-based findings into applied settings, and to develop sustainable and reproducible strategies to translate research into practice effectively and efficiently. The Coordinating Center will organize semi-annual meetings of the initiative=s Steering Committee to discuss common issues such as methodologies in implementation and in evaluation of implementation efforts, the development of a common data set with which projects can be compared or pooled, and dissemination of results. Examples of categories of common data elements include: descriptors of the patients, providers, practice setting and organizations; measures of costs and benefits of the interventions; enumeration of the target population for the intervention (denominator); and measures of patient and provider satisfaction. The Coordinating Center will collect tools that can be adopted or adapted by organizations interested in replicating the efforts of sites involved with these projects. Such material may include time-lines and flow-sheets of steps, patient and provider material, methodologies for automated records review and data collection forms. In addition, the Coordinating Center will develop other means for disseminating the results of the studies and magnifying their potential impact. Finally, the Coordinating Center will organize a meeting at the end of the third year of funding at which investigators will present and discuss findings. Applicants should include funds necessary to attend meetings and work with the Coordinating Center in their budget requests; willingness to participate with the Center must be stated. A Steering Committee, composed of the principal investigator(s) of each grant, the co- investigator(s) representing the health care organization or system for each grant, and six AHRQ staff, representing the three collaborating Centers mentioned above will be the main governing board of the study and will have primary responsibility for developing common data elements across projects, facilitating interactions between investigators and promoting strategies to disseminate and translate findings from these studies. The principal investigators from each grant and the AHRQ Program Officials will have one vote. The chairperson will be selected by the Coordinating Committee. Subcommittees will be established by the Steering Committee, as it deems appropriate, with representatives from AHRQ and individual studies. The Coordinating Center will also develop a task order contract through which external contractors will evaluate the success of the RFA at fulfilling its main objectives. Specific factors that will be addressed in the evaluation include whether the relationships between researchers and health care organizations/systems are substantive and sustainable, whether changes in practice are sustainable, whether systems developed within an institution to facilitate improved care for one condition can be applied to other conditions, whether the systems and interventions can be applied externally and whether the tools that are developed or refined in the studies and disseminated by the Coordinating Center are useful to potential external users. Terms and Conditions of Award These special Terms of Award are in addition to and not in lieu of otherwise applicable OMB administrative guidelines, HHS grant administration regulations at 45 CFR Parts 74 and 92, 42 CFR Part 67 Subpart A, and other HHS, PHS grants administration policy statements. Applicants should be familiar with the Agency=s grant regulations, 42 CFR Part 67 Subpart A, and particularly sections 67.18-67.22. Consistent with the U18 concept defined under Mechanism of Support, the dominant role and prime responsibility for the activity resides with the awardee(s) for the project as a whole, although specific tasks and activities in carrying out the studies will be shared among the awardees and the AHRQ Program Official(s). 1. Awardee Rights and Responsibilities o Awardees will have primary and lead responsibilities for the project as a whole, including research design, and protocol and instrument development, participant recruitment and follow-up, data collection, quality control, interim data and safety monitoring, final data analysis and interpretation, preparation of publications, appropriate dissemination of project findings, as well as collaboration with other awardees, and AHRQ Program Official(s). o The project team will attend semi-annual meetings of the Steering Committee. o The Principal Investigator will provide access to tools and data in a timely fashion to the Coordinating Committee and External Evaluators as is needed to achieve their objectives. o The Principal Investigator will submit a quarterly progress report to the responsible AHRQ program official. Awardees will retain custody of and have primary rights to the data developed under these awards, subject to Government rights of access consistent with current HHS, PHS, and AHRQ policies. 2. AHRQ Staff Responsibilities The AHRQ Program Official(s) will have substantial scientific/programmatic involvement in development of a common data set with which projects can be compared or pooled, and in dissemination of results. The dominant role and prime responsibility for the activity resides with the awardee(s) for the project as a whole, although specific tasks and activities in carrying out the studies will be shared among the awardees and the AHRQ Program Official(s). AHRQ reserves the right to terminate or curtail a study in the event of substantial shortfall in participant recruitment, follow-up, data reporting, quality control, or other major breach of the protocol; human subject ethical issues that may dictate a premature termination, and unwillingness to participate in investigator meetings, submit quarterly progress reports; or provide data and instruments to the Coordinating Center or evaluators. 3. Collaborative Responsibilities Collaborative protocols will be developed by the Steering Committee. Data will be submitted centrally to the Coordinating Center. Protocols will define rules regarding access to data and publications. Awardees will be required to accept and implement common protocols and procedures approved by the Steering Committee. 4. Arbitration Any disagreement that may arise on scientific/programmatic matters (within the scope of the award), between award recipients and AHRQ may be brought to arbitration. An arbitration panel will be composed of three members -- one selected by the Steering Committee (with the AHRQ member not voting) or by the individual awardee in the event of an individual disagreement, a second member selected by AHRQ, and the third member selected by the two prior selected members. This special arbitration procedure in no way affects the awardee's right to appeal an adverse action that is otherwise appealable in accordance with the PHS regulations at 42 CFR Part 50, Subpart D and HHS regulation at 45 CFR Part 16. Data Privacy Pursuant to section 903(c) of the Public Health Service Act (42 USC 299a- 1(c)), information obtained in the course of any AHRQ-study that identifies an individual or entity must be treated as confidential in accordance with any promises made or implied regarding the use and purposes of the data collection. Applicants must describe in the Human Subjects section of the application procedures for ensuring the confidentiality of such identifying information. The description of the procedures should include a discussion of who will be permitted access to the information, both raw data and machine readable files, and how personal identifiers and other identifying or identifiable data will be safeguarded. The grantee should ensure that computer systems containing confidential data have a level and scope of security that equals or exceeds those established by the Office of Management and Budget (OMB) in OMB Circular No. A-130, Appendix III - Security of Federal Automated Information Systems. The National Institute of Standards and Technology (NIST) has published several implementation guides for this circular. They are: An Introduction to Computer Security: The NIST Handbook; Generally Accepted Principles and Practices for Securing Information Technology Systems; and Guide for Developing Security Plans for Information Technology Systems. The circular and guides are available on the web at http://csrc.nist.gov/publications/nistpubs/800-12/handbook.pdf. The application of these standards to subcontractors and vendors should be addressed as necessary. Important legal rights and requirements applicable to AHRQ grantees are set out or referenced in the Agency=s grants regulations at 42 CFR 67, Subpart A. Rights in Data In keeping with AHRQ's legislative mandates to make both research results and data available, copies of all products and materials developed under a cooperative agreement supported in whole or in part by AHRQ funds are to be made available to AHRQ promptly and without restriction, upon request by AHRQ. AHRQ grantees may copyright or seek patents, as appropriate, for final and interim products and materials including, but not limited to, methodological tools, measures, software with documentation, literature searches, and analyses, which are developed in whole or in part with AHRQ funds. Such copyrights and patents are subject to a Federal government license to use and permit others to use these products and materials for AHRQ purposes. In accordance with its legislative dissemination mandate, AHRQ purposes may include, subject to statutory confidentiality protections, making research materials, data bases, and algorithms available for verification or replication by other researchers; and subject to AHRQ budget constraints, final products may be made available to the health care community and the public by AHRQ, or its agents, if such distribution would significantly increase access to a product and thereby produce public health benefits. These purposes may also include the development of an electronic clearinghouse that would enable outside organizations to adopt or adapt methods, tools and instruments in their own implementation efforts. Ordinarily, to accomplish distribution, AHRQ publicizes research findings but relies on grantees to publish in peer- reviewed journals and to market grant-supported products. Important legal rights and requirements applicable to AHRQ grantees are set out or referenced in the AHRQ=s grants regulation at 42 CFR Part 67, Subpart A (Available in libraries and from the GPO=s Website http://www.access.gpo.gov/nara/cfr/index.html. INCLUSION OF WOMEN, MINORITIES, AND CHILDREN IN RESEARCH INVOLVING STUDY POPULATIONS It is the policy of the AHRQ that women and members of minority groups be included in all AHRQ-supported research projects involving human subjects, unless a clear and compelling rationale and justification is provided that inclusion is inappropriate with respect to the health of the subjects or the purpose of the research. All investigators proposing research involving human subjects should read the "NIH Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research," which have been published in the Federal Register of March 28, 1994 and in the NIH Guide for Grants and Contracts of March 18, 1994. To the extent possible, AHRQ requires adherence to these NIH Guidelines. Investigators may obtain copies from the above sources or from the AHRQ Publications Clearinghouse listed under INQUIRIES, or from the NIH Guide Website https://grants.nih.gov/grants/guide/index.html. AHRQ also encourages investigators to consider including children in study populations, as appropriate. AHRQ announced in the NIH Guide of May 9, 1997, that it is developing a policy and implementation plan on the inclusion of children in health services research. This Notice is available through the AHRQ Website http://www.AHRQ.gov (Funding Opportunities) and InstantFAX (see instructions under INQUIRIES). AHRQ Program staff may also provide additional information concerning these policies (see INQUIRIES). LETTER OF INTENT Prospective applicants are asked to submit, by March 10, 2000, a letter of intent that includes a descriptive title of the proposed research, name, address, and telephone number of the Principal Investigator, the identities of other key personnel and participating institutions, and number and title of the RFA in response to which the application may be submitted. Although a letter of intent is not required, is not binding, and does not enter into consideration of any subsequent applications, the information allows AHRQ staff to estimate the potential review workload and to avoid conflict of interest in the review. AHRQ will not provide responses to letters of intent. The Letter of Intent is to be sent to the Joanne Book (see address under INQUIRIES). APPLICATION PROCEDURES The research grant application form PHS 398 (rev. 4/98) is to be used in applying for these grants. State and local government applicants may use PHS 5161-1, Application for Federal Assistance (rev. 5/96), and follow those requirements for copy submission. Applicants are encouraged to read all PHS Form 398 instructions prior to preparing an application in response to this RFA. Applications kits are available at most institutional offices of sponsored research. They may also be obtained from the Division of Extramural Outreach and Information Resources, National Institutes of Health, 6701 Rockledge Drive, MSC 7910, Bethesda, MD 20892-7910, telephone 301/710-0267, E-mail/Internet: [email protected]. AHRQ applicants are encouraged to obtain application materials from the AHRQ Publications Clearinghouse (see INQUIRIES). The RFA label and line 2 of the application form (rev. 4/98) should both indicate the RFA number. The RFA label must be affixed to the bottom of the face page. Failure to use this label could result in delayed processing of the application such that it may not reach the review committee in time for review. In addition, the RFA title and number must be typed on line 2 of the face page and the YES box must be marked. The sample RFA label available at https://grants.nih.gov/grants/funding/phs398/label-bk.pdf has been modified to allow for this change. Please note this is in pdf format. The PHS 398 type size requirements (p.6) will be enforced rigorously and non-compliant applications will be returned and not reviewed. Submit a signed, typewritten original of the application, including the Checklist, and three signed, photocopies, in one package to: Center for Scientific Review National Institutes of Health 6701 Rockledge Drive Room 1040 - MSC 7710 Bethesda, MD 20892-7710 (20817 for express/courier service) At the time of submission, two additional copies of the application, labeled AAdvanced Copy(s)@ must be sent to: Joanne Book Center for Outcomes and Effectiveness Research Agency for Healthcare Research and Quality 6010 Executive Blvd., Suite 300 Rockville, MD 20852 Telephone: (301) 594-4039 FAX: (301) 594-3211 E-mail Address:[email protected] Applications must be received by April 27, 2000. An application received after that deadline may be acceptable if it carries a legible proof-of- mailing date, assigned by the carrier, and the proof-of-mailing is not later than 1 week prior to the deadline date. If an application is received after that date, it will be returned to the applicant without review. Conference for Prospective Applicants AHRQ plans to convene a special technical assistance workshop to assist potential applicants. The purpose of this conference is to give background information and respond to questions about the preparation of an application in response to this RFA. The workshop will be held in Rockville, MD on February 2, 2000. Attendance is not a prerequisite to applying. Attendees must pay for their own travel and accommodation costs. The workshop will be open to any individual or organization intending to apply. Attendees are encouraged to submit questions prior to the session. A synthesis of pertinent Questions and Answers discussed at the prospective applicants= meeting will be available from the AHRQ Publications Clearinghouse, listed under AInquiries@ and the agency=s web page. For further information on the conference, contact Joanne Book at the address listed above. Application Preparation (for Using HCFA Data) For applications that propose to use Medicare or Medicaid data that are individually identifiable, applicants should state explicitly in the Research Design and Methods section of the Research Plan (form PHS 398) the specific files, time periods, and cohorts proposed for the research. In consultation with the Health Care Financing Administration (HCFA), AHRQ will use this information to develop a cost estimate for obtaining the data. This estimate will be included in the estimated AHRQ cost of funding the grant and may affect the fundability of the application. To avoid double counting, applicants should not include the cost of the data in the budget. Applicants should be aware that for individually identifiable Medicare and Medicaid data, Principal Investigators and their grantee institutions will be required to enter into a Data Use Agreement (DUA) with HCFA to protect the confidentiality of data in accordance with standards set out in OMB Circular A-130, Appendix IIIBSecurity of Federal Automated Information Systems. The use of the data is restricted to the purposes and time period specified in the DUA. At the end of this time period, the grantee is required to return the data to HCFA or certify that the data have been destroyed. Grantees must also comply with the confidentiality requirements of Section 903(c) of the PHS Act. See the Data Privacy Section for details on these requirements as well as references to Circular A-130 and its implementation guides from the National Institute of Standards and Technology. In developing research plans, applicants should allow time for refining, approving, and processing their data requests. Requests may take 6 months from the time they are submitted to complete. Applications proposing to contact beneficiaries or their provider require the approval of the HCFA Administrator and may require meeting(s) with HCFA staff. HCFA data are provided on IBM mainframe tapes using the record and data formats commonly employed on these computers. Applicants should either have the capability to process these tapes and formats or plan to make arrangements to securely convert them to other media and formats. Questions regarding HCFA data should be directed to Joanne Book, whose contact information is listed under INQUIRIES. REVIEW CONSIDERATIONS Upon receipt, applications will be reviewed for completeness and responsiveness. Incomplete applications will be returned to the applicant without further consideration. Applications that are complete but are not responsive to the RFA will be returned to the applicant without further consideration; applicants may submit these applications as investigator- initiated R01 grant applications to AHRQ to compete with other unsolicited applications at the next review cycle. As part of the merit review, all applications will receive a written critique, and also may undergo a process in which only those applications deemed to have the highest scientific merit will be discussed and assigned a priority score. All applications will be judged on the basis of the scientific merit of the proposed project and the documented ability of the investigators to meet the RESEARCH OBJECTIVES of the RFA. Although the technical merit of the proposed project is important, it will not be the sole criterion for evaluation of a study. Other considerations, such as importance and timeliness of the proposed study and access to patients/data will be part of the evaluation criteria. General Review Criteria The goals of AHRQ-supported research are to enhance the outcomes, quality, and cost, use and access of our health care system. The reviewers will comment on the following aspects of the application in their written critiques in order to judge the likelihood that the proposed research will have a substantial impact on the pursuit of these goals. Each of these criteria will be addressed and considered by the reviewers in assigning the overall score, weighting them as appropriate for each application. Note that the application does not need to be strong in all categories to be judged likely to have a major scientific impact and thus deserve a high priority score. For example, an investigator may propose to carry out important work that by its nature is not innovative but is essential to move a field forward. 1. Significance. Does this study address an important problem as defined by the specific criteria above. If the aims of the application are achieved, how will scientific knowledge be advanced? What will be the effect of these studies on the concepts or methods that drive this field? How will health care and health outcomes be demonstrably improved? What is the potential for the intervention to improve health care for a large number of individuals? 2. Approach. Are the conceptual framework, design, methods, and analyses adequately developed, well-integrated, and appropriate to the aims of the project? Are the proposed data sources appropriate and adequate? Does the applicant acknowledge potential problem areas and consider alternative tactics? 3. Innovation. Does the project employ novel concepts, approaches or method? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies? 4. Investigator. Is the investigator appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the principal investigator and other researchers (if any)? Is the project (or workplace) well organized? 5. Environment. Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Is there evidence of institutional support? The initial review group will also examine: the appropriateness of proposed project budget and duration; the adequacy of plans to include both genders, children, and minorities and their subgroups as appropriate for the scientific goals of the research and plans for the recruitment and retention of subjects; the provisions for the protection of human and animal subjects; and the safety of the research environment. Special Review Criteria o Degree to which the research will improve our understanding of how we can narrow the gap between what is known and what is done in the real world of health care. o Appropriateness of methods to address research hypotheses and needs of those who may be interested in applying the results of the study. o Demonstration of substantial commitment by the health care organization/system to the implementation effort and to sustaining the intervention and specific improvements beyond the duration of this study in a constantly changing environment, including use of in-kind support and letters of support from clinical and administrative staff. o Extent to which results will be transferable or reproducible in other health care systems. o Likelihood of the development of systems that can be applied to implementation of research findings for a variety of conditions and in a range of settings, and the sustain ability of such systems. o Demonstration of ability to recruit an adequate number of patients and providers. o Extent to which the project builds-on or establishes meaningful linkages between researchers and health care systems/organizations with the expectations that such relationships will exist beyond the study period. o Development of new infrastructure including talent, relationships, tools, methods and data systems necessary to accelerate, magnify and track the impact of the research on clinical practice and patient outcomes. o What is the potential for the intervention to improve health care for a large number of individuals? o Plans for dissemination and implementation of findings within and outside of the project=s organization. o Development of additional ideas on how best to achieve the goals of this cooperative study. o Demonstrated willingness to work as part of the cooperative study, with the AHRQ Program Official(s), the Coordinating Center, the Steering Committee and external evaluators. For applicants interested in implementing AHRQ-funded findings nationally, the partnering organization must be either a co-investigator or a subcontractor to the researcher. Clear plans for tracking changes in care over time must be identified. AWARD CRITERIA Applications will compete for available funds with all other applications under this RFA. The following will be considered in making funding decisions: 1) quality of the proposed project as determined by peer review; 2) availability of funds; 3) program balance with respect to conditions, populations and strategies for improving implementation studied, regional representation, and targeting understudied areas with unmet evaluation needs; 4) compatibility with other awardees with respect to cooperative activities; 5) fulfillment of set-aside funding targets; and, 6) policy relevance. INQUIRIES Copies of this RFA are available from: AHRQ Publications Clearinghouse P.O. Box 8547 Silver Spring, MD 20907-8547 Telephone: 1-800-358-9295 The RFA is available on AHRQ=s Web site, http://www.AHRQ.gov under Funding Opportunities and through AHRQ InstantFAX at (301) 594-2800. To use InstantFAX, you must call from a facsimile (FAX) machine with a telephone handset. Follow the voice prompt to obtain a copy of the table of contents, which has the document order number (not the same as the RFA number). The RFA will be sent at the end of the ordering process. AHRQ InstantFAX operates 24 hours a day, 7 days a week. For comments or problems concerning AHRQ instant fax, please call at (301) 594-6344. AHRQ welcomes the opportunity to clarify any issues or questions from potential applicants who have read the RFA. Written and telephone inquiries concerning this RFA are encouraged. Direct inquiries regarding programmatic issues, including information on the inclusion of women, minorities, and children in study populations to: Joanne Book Center for Outcomes and Effectiveness Research Agency for Healthcare Research and Quality 6010 Executive Blvd., Suite 300 Rockville, MD 20852 Telephone: (301) 594-4039 FAX: (301) 594-3211 E-mail Address: [email protected] or Diane Brown Center for Practice and Technology Assessment Agency for Healthcare Research and Quality 6010 Executive Blvd., Suite 300 Rockville, MD 20852 Telephone: (301) 594-4019 FAX: (301) 594-4027 E-mail Address: [email protected] Applicants interested in focusing on mental health are encouraged to contact NIMH=s Dissemination Research Program. The program supports studies on the array of influences that affect the adoption of valid mental health research findings into clinical practice and mental health service system policy. This program supports research aimed at determining the impact, efficacy, quality, and effectiveness of knowledge dissemination interventions delivered across the spectrum of mental health service settings. Areas of support include: analysis of the strategies and factors influencing the adoption and utilization of mental health research knowledge; descriptive and comparative evaluations of individual and multi- faceted knowledge dissemination interventions and methodologies; longitudinal and follow-up studies to determine factors that contribute to, and militate against, efficacious short and long-term change in practice and policy; evaluation of the effectiveness of knowledge dissemination interventions and strategies on service delivery systems that address rural, minority, and other underserved populations. For further information, contact: Harold Goldstein, Ph.D. Chief, Dissemination Research Program National Institute of Mental Health 6001 Executive Blvd., Rm. 7133, MSC 9631 Bethesda, MD 20892-9631 Telephone: (301) 443-3747 FAX: (301) 443-4045 E-mail Address: [email protected] Direct inquiries regarding fiscal matters to: Joan Metcalfe Grants Management Specialist Agency for Healthcare Research and Quality 2101 East Jefferson Street, Suite 601 Rockville, MD 20852 Telephone: (301) 594-1841 FAX: (301) 594-3210 Email: [email protected] AUTHORITY AND REGULATIONS This program is described in the Catalog of Federal Domestic Assistance No. 93.226. Awards are made under authorization of Title IX of the Public Health Service Act (42 USC 299-299c-6). Awards are administered under the PHS Grants Policy Statement and Federal Regulations 42 CFR 67, Subpart A, and 45 CFR Parts 74 and 92. This program is not subject to the intergovernmental review requirements of Executive Order 12372 or Health Systems Agency review. The PHS strongly encourages all grant and contract recipients to provide a smoke-free workplace and promote the non-use of all tobacco products. In addition, Public Law 103- 227, the Pro-Children Act of 1994, prohibits smoking in certain facilities (or in some cases, any portion of a facility) in which regular or routine education, library, day care, health care or early childhood development services are provided to children. This is consistent with the PHS mission to protect and advance the physical and mental health of the American people. REFERENCES 1. McLaughlin TJ, Soumerai SB, Wilson DJ, Gurwitz JH, Borbas C, Guadagnoli E, McLaughlin B, Morris N, Cheng SC, Hauptman PJ, Antman E, Casey L, Asinger R, Gobel F. Adherence to National Guideline for Drug Treatment of Suspected Acute Myocardial Infarction: Evidence for Undertreatment in Women and the elderly. Arch Intern Med 1996;156:799-805. 2. Soumerai SB, McLaughlin TJ, Spiegelman D, Hertzmark E, Thibault G, Goldman L. Adverse Outcomes of Underuse of beta-blockers in elderly survivors of acute myocardial infarction. JAMA 1997; 277:115-21. 3. Lieu TA, Quesenberry CP, Capra AM, Sorel ME, Martin KE, Mendoza GR. Outpatient Management Practices Associated with Reduced Risk of Pediatric Asthma Hospitalization and Emergency Department Visits. Pediatrics 1997;100:334-41. 4. Warman KL, Silver EJ, McCourt MP, Stein RE. How Does Home Management of Asthma Exacerbations by Parents of Inner-City Children differ from NHLBI Guideline Recommendations? Pediatrics 1999;103:422-27. 5. Lehman A, Steinwachs D, and the Survey Co-Investigators of the PORT Project. Patterns of Usual Care for Schizophrenia: Initial Results from the Schizophrenia Patient Outcomes Research Team (PORT) Client Survey. Schizophrenia Bulletin, 24:11-20, 1998. 6. Schuster MA, McGlynn E, Brook RH. How Good is the Quality of Health Care in the United States ? Milbank Quarterly 1998;76:517-63. 7. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc. J, 1995;153:1423-31. 8. NHS Center for Reviews and Dissemination. Effective Health Care. Getting Evidence into Practice. February 1999;5(1). 9. Cabana MD, Rand CS, Power NR, Wu AW, Wilson MH, Abboud PC, Rubin HR. Why Don=t Physicians Follow Clinical Practice Guidelines? A Framework for Improvement. JAMA 1999;282:1458-65. 10. Http://www.hcqualitycommission.gov 11. Oxman AD, Thomson MA, Davis DA, Haynes RB (1995). No Magic Bullets: A systematic Review of 102 Trials of Interventions to Improve Professional Practice. Can Med Assoc J;153(10):1423-1431. 12. Hunt DL, Haynes RB, Hanna SE, Smith K (1998). Effects if Computer- based Clinical Decision Support Systems on Physician Performance and Patient Outcomes. A Systematic Review. JAMA;280:1339-46. 13. Forrest CB, Simpson L, Clancy C (1997). Child Health Services Research. Challenges and Opportunities. JAMA;277(22):1787-1793.
Weekly TOC for this Announcement
NIH Funding Opportunities and Notices
Office of Extramural Research (OER) |
National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892 |
Department of Health and Human Services (HHS) |
||||||||