Full Text HS-95-003

CONSUMER ASSESSMENTS OF HEALTH PLANS STUDY

NIH Guide, Volume 24, Number 9, March 10, 1995

RFA:  HS-95-003

P.T. 34

Keywords: 
  Surveys & Survey Research 
  Data Management/Analysis+ 
  HEALTH CARE 


Agency for Health Care Policy and Research

Letter of Intent Receipt Date:  May 20, 1995
Application Receipt Date:  June 20, 1995

PURPOSE

The Agency for Health Care Policy and Research (AHCPR) invites
applications for cooperative agreements to:  (1) produce reliable,
valid and rigorously tested survey protocols for collecting
information from consumers regarding their assessments of health
plans and services; (2) develop and test the effectiveness of
different formats for conveying resulting information to consumers;
(3) demonstrate the resulting survey protocols in real world
settings; and (4) evaluate the usefulness of this information in
assisting consumers, and purchasers acting on their behalf, in making
informed selections of health care plans and services.  The long term
goal of this project is to strengthen the science base underlying the
evolution and the use of consumer surveys within the health care
industry.

HEALTHY PEOPLE 2000

The Public Health Service (PHS) is committed to achieving the health
promotion and disease prevention objectives of "Healthy People 2000,"
a PHS-led national activity for setting priority areas.  This RFA,
Consumer Assessments of Health Plans Study (CAHPS), is related to the
priority areas of access to and use of clinical preventive services,
maternal and child health services, and health services related to
major disease categories.  Potential applicants may obtain a copy of
"Healthy People 2000" (Full Report:  Stock No. 017-001-00474-0 or
Summary Report:  Stock No. 017-001-00473-1) through the
Superintendent of Documents, Government Printing Office, Washington,
DC 20402-9325 (telephone 202-783-3238).

ELIGIBILITY REQUIREMENTS

Applications may be submitted by public and private non-profit
private domestic organizations or associations, including
universities, units of State and local governments, non-profit firms
and foundations; or a consortium of organizations.  If the
application is submitted by a domestic, non-profit, public or private
organization, a consortium may include other types of organizations,
such as for-profits.  Racial/ethnic minorities, women, and persons
with disabilities are encouraged to apply as Principal Investigators.

MECHANISM OF SUPPORT

The administrative and funding instrument will be the cooperative
agreement (U18), an "assistance" mechanism in which substantial AHCPR
scientific and programmatic involvement with the awardee(s) is
anticipated during the performance of the project.  The total project
period for each application submitted in response to this RFA may not
exceed five years.  The anticipated award date is September 30, 1995.
Funding beyond the initial budget period will depend upon annual
progress reviews by AHCPR and the availability of funds.

FUNDS AVAILABLE

The AHCPR expects to award up to $2 million for two projects not to
exceed $1 million in total costs for each project for the first year.
The number of awards is dependent on the number of high quality
applications responding to this RFA.  This is a one time
solicitation.

RESEARCH OBJECTIVES

Background

Public and private organizations are surveying consumers to collect
information on access to care, use of health services, health
outcomes, and patient satisfaction.  The results of these surveys are
being used by: consumers to inform their choices about health care
plans; purchasers to assess the value of the services they buy; and
health insurers, quality assurance managers and policy makers, to
plan programs and services.

Existing tools for obtaining information from health plan enrollees
vary on several dimensions:  content of the questions;
characteristics of intended respondents; and fit to particular health
care settings.  The extent to which these instruments have been
tested and the methods used to accomplish the testing also differ
across surveys.  Because of these variations, it is unlikely that any
single existing instrument will produce accurate and useful
assessments from (and for) consumers across different types of plans
and health care settings.

Potential survey users require responses to a range of critical
questions including, but not limited to, the following:  Are existing
instruments capable of producing reliable and valid assessments of
consumer evaluations of health care services?  Do they include the
full range of topics important to consumers?  Can they be used to
obtain information from the full range of audiences of interest,
including high users of health care services; people insured under
managed care as well as fee-for service plans; consumers with poor
literacy skills; plan disenrollees; participants in publicly
subsidized programs; and relatively healthy middle-class health care
consumers?

Even if reliability and validity can be demonstrated, will potential
users, particularly inexperienced ones, know how to conduct a valid
survey?  Will different users collect data in a manner that allows
valid comparison across a variety of health plans?

And most importantly, if these efforts can be accomplished, will they
yield the expected pay-off; that is, the production of accurate,
understandable information that enables consumers and purchasers
acting on their behalf to identify and select appropriate,
high-quality health care coverage?

In September 1994, AHCPR co-sponsored a conference with the Robert
Wood Johnson (RWJ) Foundation, "Consumer Survey Information in a
Reformed Health Care System."  Conference papers and discussions
identified no single instrument that could be used across a wide
variety of consumer groups and within multiple service settings to
obtain assessments of health plans and services.  Some of these
surveys are effective, though, for a specific purpose, population or
service.  Thus, these various instruments offer the possibility of
constructing "question modules" -- sets of questions drawn from a
variety of sources and relating to diverse concepts, consumers and
delivery settings -- to use as the basis for activities to be
performed under this agreement.

The AHCPR is currently supporting a project with that goal:  to
construct question modules from existing survey instruments.  This
Survey Design Project, through contract to the Research Triangle
Institute, is developing modules of consumer survey questions.  These
modules ask about the experiences of consumers regarding access to
care, use of specific health services, health outcomes, and
satisfaction with the care received.  The goal of the project is to
produce modules that are adaptable to a variety of health care
settings.

The Survey Design Project began with an extensive search for question
modules and items.  This search was guided by recent published and
unpublished studies on consumer information needs.  The contractor
then developed an Adult Health Care Survey and began developing
modules and items related to other areas of interest.

Although validity and reliability studies were not within the scope
of work, items in the modules were subjected to rigorous cognitive
testing to ensure appropriate interpretation by respondents.  Items
in the modules were then revised as necessary to be understandable to
consumers who receive care through different benefits arrangements
(e.g., fee for service as well as managed care plans).

Finally, this project's methods and products were reviewed by as many
interested parties as could be identified.  This review included two
public meetings, one held at the beginning of the contract and one at
the end.  Draft modules and materials were circulated for comments to
over 100 researchers, consumers and potential users.  The project was
also informed by a technical panel of researchers and representatives
of health insurance organizations, managed care organizations,
clinical associations and other groups.

Because AHCPR wishes to provide rapid access to products from the
Survey Design Project and to products resulting from CAHPS, we plan
to disseminate both through the AHCPR User Group Initiative.  This
effort is being developed to respond to requests from potential users
who wish to begin using consumer question modules as early as
possible, even before the completion of the systematic demonstration
called for in this RFA.  Points at which grantees are expected to
coordinate activities with the User Group Initiative are noted under
RESEARCH OBJECTIVES, Study Design.

The results of the Survey Design Project, including limited technical
and operational instructions, will be placed in the public domain no
later than March 30, 1995.  The results of the Survey Design Project,
the AHCPR/RWJ conference and other relevant efforts are being made
available to potential applicants and others (see INQUIRIES).

Objectives and Scope

The objectives of this RFA are to:  (1) produce reliable, valid and
rigorously tested survey protocols for collecting information from
consumers regarding their assessments of health plans and services;
(2) develop and test the effectiveness of different formats for
conveying resulting information to consumers; (3) demonstrate the
resulting survey protocols in real world settings; and (4) evaluate
the usefulness of this information in assisting consumers, and
purchasers acting on their behalf, in making informed selections of
health care plans and services.

To save time and resources, applicants should select the best
existing set of questions currently in the public domain and use it
to accomplish objectives 1 and 2.  For purposes of this RFA, the
"best existing set of questions" is that which most successfully
addresses the following questions and concerns:

1.  Who are the critical consumers from whom to collect assessments
of health care plans and services?  Successful applicants will
develop and test a core question module designed to obtain a basic
assessment of plan features and services from relatively healthy,
adult, middle-level socioeconomic status (SES) Americans who receive
care through the predominant types of benefits plans and delivery
settings (e.g., indemnity plans, managed care plans, etc).  Grantees
should develop and test additional modules, as time and budget
permit, to obtain assessment data from some of the following types of
consumers:

a.  Parents of young children, especially those of low SES,
b.  Racial and ethnic minorities,
c.  Participants in publicly subsidized health programs, such as
Medicaid or State health programs,
d.  High users of health care services, such as the chronically ill,
those suffering severe acute episodes of illness, and persons with
disabilities,
e.  Persons living in rural areas,
f.  Persons with poor literacy skills, and
g.  Disenrollees from health plans.

For which of these potentially important consumer groups should
question modules be developed?  After the core module is completed,
what should be the sequence of module development?  What rationale
supports this prioritization?  Applicants should address these
questions as part of their application.  If applicants discover that
items or question modules for use with a consumer group perceived as
high- priority do not exist, they should propose development of those
modules as part of this project.

2.  What information on access and quality do consumers want and need
to assist in selecting health care plans and services?  How do
information needs vary by type of audience?  Although a large-scale,
fully developed needs assessment covering the entire range of topics
and consumers may not exist, a number of qualitative (e.g., focus
group studies by AHCPR and the National Committee on Quality
Assurance in 1994) and quantitative (e.g., a survey by Consumers
Union 1995) studies conducted recently  provide information related
to this question.  If an applicant discovers an access or
quality-related information need for which items/questions modules do
not exist, the applicant may propose development of those modules as
part of this project.

3.  Through what types of benefits arrangements and delivery settings
are the consumer groups of interest likely to receive services?
Consumers may conceivably obtain health care services through fee for
service plans, diverse types of managed care plans, public assistance
programs or other types of benefits arrangements.  These benefits may
be offered through public or private employers, purchasing
cooperatives, multi-state health plans, or public health clinics.
This diversity of benefits and settings means that items, or the
wording of items, from any given set of questions may not be
appropriate for all consumers in all situations.

In selecting their core questions module, applicants should consider
that the overall goal of CAHPS is to produce modules (especially a
core module) that will allow comparability across the greatest
possible range of plans and benefits arrangements.  Since it is
unlikely that any existing module was designed with application to
multiple settings in mind, applicants should discuss in their
application the extent to which their selected module will require
post-award testing to meet this goal.

4.  What evidence exists to ensure that consumers interpret proposed
survey items as the developers intended?  As mentioned under RESEARCH
OBJECTIVES, Background, a variety of instruments exist with which to
obtain consumer perceptions about health care services.  A limitation
of some of these instruments is their lack of cognitive testing of
survey items with members of the target audience.  In selecting
question modules for use in this RFA, applicants should obtain
information about the extent to which item interpretation was tested
and the methods used to accomplish this.

Because the AHCPR wishes to build on existing work to the greatest
extent possible and to accomplish objectives 1 and 2 rapidly, it is
recommended that the results of the Survey Design Project form the
basis for activities to be performed under this agreement.  The
modules developed in the Survey Design Project, to AHCPR's awareness,
best satisfy the concerns discussed above.

Although use of the modules from the Survey Design Project is
recommended, itis anticipated that applicants may make revisions,
additions, or other alterations to them.  Changes or alternatives to
the modules are acceptable and appropriate to the extent that they
address the concerns discussed above.

In any case, each applicant must:  (a) specify instruments or
question modules and (b) discuss their suitability for accomplishment
of study objectives.  Applicants' responses to the four preceding
questions should form the basis for Phase 1 of this project as
described in the next section.

Study design

Grantees will need to accomplish a number of objectives in two
phases, a project planning phase (Phase 1) and a demonstration and
evaluation phase (Phase 2).  During both phases, grantees will
collaborate with the AHCPR project officer (and the advisory
committee, when appropriate) to define and respond to key questions
and issues that will guide the development of CAHPS products.

Grantees are expected to prepare interim reports and a final report
that summarizes all Phase 1 and 2 activities. Elements of these
reports will be determined jointly by the grantees and AHCPR.

Phase 1:  Project planning

The major goals of Phase 1 include:  (a) development and initial
implementation of a project work plan, (b) production of survey
protocols suitable for use in real-world settings, (c) development
and testing of report formats for survey information, (d) development
and initial implementation of a process and outcome evaluation plan,
and (e) recruitment of demonstration sites and development of a
demonstration time table.  Phase 1 is expected to be completed within
15 months of the award date.

Applicants should provide information that demonstrates their ability
to accomplish the activities listed below.  Applicants may include in
appendices descriptions of prior projects that demonstrate their
experience in performance of similar activities.

1.  Develop a work plan.  This should include specification of
project activities, linked to a listing of products and a time table
for their execution or development.

2.  Propose advisory committee (see SPECIAL REQUIREMENTS).
Applicants should discuss proposed composition of the advisory
committee (i.e., types of expertise required) and ways in which the
advisory committee may be used most effectively during the course of
the project.

3.  Characterize focal consumer populations.  Applicants will have
laid the groundwork for this step as part of their application.  In
Phase 1, grantees and AHCPR (with input from the advisory committee)
should complete specification of focal consumer groups, as well as
determine their quality-related information needs.  As they make
these decisions, the project team should keep in mind that products
resulting from CAHPS must be appropriate for use in a variety of
settings as discussed in RESEARCH OBJECTIVES, Objectives and Scope,
number 1.

4.  Refine the question modules.  Applicants will have selected
existing modules and identified items/modules that need to be
developed as part of the application.  As the grantees develop new
items and modules, they should make critical decisions jointly with
AHCPR and with input from the advisory committee.

5.  Perform all activities related to production of survey protocols
that are ready for use in real world settings.  This includes:
testing the validity and reliability of question modules for each
consumer group of interest; performing cognitive testing, as
necessary, to ensure that respondents will interpret items as
intended; resolving all issues related to sampling strategies;
preparing alternative versions of survey instruments for
administration by telephone or mail; developing and implementing
plans for data analysis and processing, and other issues.

As part of the application, applicants should identify criteria for
site selection for Phase 1 reliability, validity and cognitive
testing and obtain letters of commitment from actual sites.

Because the user demand for rigorously tested survey modules is
great, grantees are expected to perform reliability, validity and
cognitive testing on a module-by-module schedule, with priority given
to modules that have the widest applicability.  As each module is
tested and revised, it will be released to the public through the
AHCPR User Group Initiative.  Though many important activities need
to be accomplished within the 15 month Phase 1 time frame, grantees
should sequence activities so that the testing of questionnaire
modules occurs as early as possible.

6.  Coordinate activities as appropriate with AHCPR User Group
Initiative.  The purposes of the User Group Initiative are to
disseminate modules and segments of the user manual as they are
completed, track efforts at implementing the modules at sites other
than those included in this cooperative agreement, and provide
technical assistance to these users.

7.  Develop/revise user manual.  The user manual should provide
thorough, comprehensive documentation to guide users, whether
unsophisticated or highly experienced, through all activities
associated with survey administration, including mail and telephone
administration.

8.  Develop and test sample reporting formats.  Many different types
of consumers will ultimately use reports based on data collected
through these instruments.  To be optimally useful to consumers,
characteristics of the data-based reports should be tailored to fit
characteristics of the intended user.

On the other hand, some potential survey administrators will not have
the resources required to develop reports specifically tailored to
consumer subgroups.  These users will need to know how to select
appropriate formats from a stock of sample reports and how to design
and test their own materials for a heterogeneous group of users.

9.  Develop a process and outcome evaluation plan.  One of the
grantee's most important responsibilities under this agreement will
be the development of a comprehensive plan to evaluate CAHPS
products, procedures, and outcomes.  This involves at least three
components:  (a) developing and implementing a plan for the
evaluation of report formats; (b) developing and implementing a plan
for the process evaluation of CAHPS procedures (e.g., the usefulness
of the operations manual to staff at demonstration sites); and (c)
developing and implementing a plan for the evaluation of the
effectiveness of data-based reports to consumers.  Though all of the
evaluation planning must occur in Phase 1, the only evaluation data
expected to be collected in Phase 1 is that related to step "a."

Applications should demonstrate the applicant's ability to:

o  Specify appropriate evaluation questions, identify appropriate
outcome measures and appropriate respondents for each question, and
specify times at which these questions should be asked.

o  Develop an appropriate evaluation design.  The design should:  (a)
eliminate as many threats to validity as possible; and (b) result in
a clear demonstration of usefulness of the data-based reports in
assisting consumer decision-making in an open enrollment situation,
as well as identification of factors related to product effectiveness
or ineffectiveness.

o  Identify and use appropriate qualitative and quantitative data
collection instruments.

o  Analyze data appropriately.  Applicants should pay particular
attention to strategies for analysis of qualitative data, as well as
the issue of conveying information from quantitative evaluation
analyses in a manner that is understandable by and useful to
personnel at sites implementing the survey protocols.

o  Develop recommendations based on evaluation data that can be used
to modify CAHPS projects or procedures.  One component of the CAHPS
evaluation plan should be the development of clearly written,
user-friendly recommendations designed to promote data-based changes
to products and procedures.

10.  Recruit sites and develop a time table for Phase 2
demonstrations.

o  Identify selection criteria for and propose demonstration sites.
AHCPR is interested in a range of diverse demonstration sites as
discussed above in RESEARCH OBJECTIVES, Objectives and Scope, numbers
1 and 3.  Applicants are not required to include all of those types
of settings in their applications, but should provide examples of the
types of settings they intend to include, a rationale for their
choice, and evidence of recent experience in undertaking large-scale,
multi-site demonstrations.  If necessary, AHCPR will work with the
awardee to identify appropriate organizations.  Final selection of
sites will take place in consultation with the AHCPR Project Officer.

o  Identify key personnel from each potential site and specify their
roles.  Also, as sites are selected, grantees will need to identify
appropriate demonstration site staff to be added to the advisory
committee (e.g., the lead contact at each site, consumer
representatives, etc.).

o  Building on the work plan discussed above, develop a demonstration
time table.  This document should contain all the information sites
will need to know in order to commit themselves to the demonstration.

Phase 2:  Demonstration and evaluation

The goals of Phase 2 are to:  (a) work with demonstration sites to
implement the tested survey protocols; (b) assess the effectiveness
of data obtained through these surveys in assisting consumers with
their health plan selections; (c) continue implementation of the
process and outcome evaluation; and (d) revise CAHPS products and
procedures based on evaluation data.

Applicants should provide information which illustrates their ability
to plan and implement large-scale, multi-site demonstrations,
including their ability to accomplish the following activities.

11.  Develop a demonstration management plan.  After each site
commits to the demonstration, the grantee should develop an overall
management plan which will serve as the master plan to guide all
demonstration activities.  The groundwork for this plan will have
been laid by development of demonstration time tables discussed at
the end of Phase 1 activities.  The management plan should clearly
specify both grantee and site personnel roles and responsibilities
and should include a time table listing key decisions related to and
dates for all demonstration components.

12.  Assist sites in all phases of demonstration activity as
specified in the management plan, including:  selecting appropriate
survey modules; collecting data; using the operations manual;
developing, testing, and disseminating reports in formats appropriate
for the audiences of interest; and other activities.

13.  Coordinate activity among all demonstration sites and work with
the AHCPR User Group Initiative.

14.  Collect process and outcome evaluation data at all appropriate
points.  The grantee should analyze evaluation data and produce
reports which include recommended revisions to CAHPS products and
procedures.

15.  Revise products and procedures based on data-based
recommendations.  Before making these revisions, the grantee should
consult with AHCPR and obtain input from the advisory committee.

Timetable

Phase 1 of CAHPS should be completed 15 months from the date of
award.  The entire project may take four to five years.

SPECIAL REQUIREMENTS

To promote the development of this multi-site collaborative project,
a number of additional issues must be addressed in applications
responding to this RFA, as discussed below.

Budget

Applicants must develop a timeline for project activities, including
percentage of effort for key staff for all years, as part of the
budget justification.

Project Organization

Applicants, or principal members of the consortium qualified to
participate in this agreement, must have experience in: field survey
operations; testing and improvement of survey instruments; management
of multiple collaborators; program planning and evaluation; and
development and evaluation of health-related materials for consumer
audiences.

If a consortium of institutions responds to this RFA, the application
must describe a practical structure for consortium decision-making
and governance, and the mechanisms designed to ensure that effective
collaboration will occur among sites.  Unanticipated disagreements
about methods, resource allocation, standardization, authorship,
etc., may arise during the course of any project.  The consortium
must be able to make unified decisions on the merits of these issues,
without dissolving or routinely relying upon outside arbitration.

Confidentiality of Data

Information obtained in the course of this study that identifies an
individual or entity must be treated as confidential in accordance
with section 903(c) of the Public Health Service Act.  Applicants
must describe in the Human Subjects section number 5 procedures for
ensuring the confidentiality of information.  This should include a
discussion of who will be permitted access to the information, both
the raw data and machine readable files, and how personal identifiers
will be safeguarded.

Terms and Conditions of Award

This cooperative agreement anticipates substantial AHCPR scientific
and programmatic involvement with the awardees throughout the
planning, implementation, and close-out of CAHPS.

1.  Awardee Responsibilities

The awardee will have primary and lead responsibility for all
activities including:

o  Formation of the advisory committee, including convening meetings
and documenting activities, with committee membership approved by the
AHCPR project officer;

o  Development and implementation of a comprehensive process and
outcome evaluation design;

o  Selection, in collaboration with the AHCPR project officer, of
items or question modules from the consumer survey instrument that
will be tested;

o  Identification and implementation of techniques to test the
modules for reliability, validity, consistency in interpretation, and
comparability across different sites/subpopulations;

o Development and implementation of a module-by-module testing
schedule;

o  Identification and recruitment of Phase 1 testing sites for
reliability, validity, etc.  Applicants should submit letters of
commitment with the application;

o  Identification of characteristics of appropriate Phase 2
demonstration sites and selection, in collaboration with the project
officer, of actual sites possessing these characteristics;

o  Development of sample selection strategy and identification of
potential associated methodological issues;

o  Identification of methodological issues associated with
administration of the survey to the subpopulations and sites of
interest;

o  Identification of potential confidentiality concerns of test sites
and individual respondents and a strategy for implementing safeguards
that address these concerns;

o  Identification of methodological issues associated with mode of
administration of the questionnaire;

o  Identification and implementation of procedures that enhance
response rates;

o  Development and implementation, with project officer approval, of
an appropriate data coding, preparation and analysis plan;

o  Development and documentation of strategies to address all
outlined methodological issues;

o  Development, testing and revision of a comprehensive survey
operations, or "user", manual suitable for use by organizations with
minimal as well as extensive experience in fielding large-scale,
multi-site surveys;

o  Provision of training and technical assistance to all Phase 2
demonstration sites;

o  Preparation and testing, in collaboration with project officer, of
understandable, usable reports that summarize survey results and
which are tailored to the subpopulations of interest;

o  Preparation of agreed-upon interim and final reports, including a
summary report to inform AHCPR of any difficulties encountered in the
course of planning, implementing and evaluating the surveys;

o  Provision of AHCPR access to all data generated under this award
as it is collected; and

o  Cooperation with other key parties in this project, particularly
the AHCPR User Group; other CAHPS grantees; and other complementary
projects.

2.  AHCPR Staff Responsibilities

The AHCPR Project Officer and other AHCPR staff will have substantial
scientific and programmatic involvement during the conduct of this
activity, through technical assistance, advice, and coordination
beyond the usual program stewardship for grants.  Collaboration on
the development of the testing plan, evaluation design and survey
protocol will occur after the award is made.  Specifically, AHCPR's
role in the cooperative agreement is to provide technical assistance,
advice, and support to the PI and to ensure that all related
AHCPR-supported projects complement one another. AHCPR will:

o  Work with the grantee to structure composition of the advisory
committee;

o  Approve composition of advisory committee, management plan and
plans for data collection and analysis;

o  Participate in all key project decisions, including but not
limited to:  identification of focal consumer groups, assessment of
consumer information needs, selection of items/question modules to be
tested, identification and resolution of methodological issues;

o  Work with the grantee to identify and negotiate with Phase 2
demonstration sites, with the AHCPR project officer retaining right
of final approval;

o  Establish collaboration with and obtain project-related
information from other Federal agencies and programs;

o  Disseminate tested question modules and other research findings as
they become available; and

o  Participate in or coordinate strategy sessions with the grantees
on at least four occasions in the first year and up to every four
months thereafter to review progress;

The AHCPR will require prior written approval for the addition or
deletion of a participating or collaborating institution, site, or
other organizational component.

The AHCPR reserves the right to terminate or curtail the study in the
event of:

o  Insufficient progress towards completion of study goals within an
agreed-upon time frame;

o  Inability to identify and recruit demonstration sites that include
members of the subpopulations and settings of interest;

o  Inability to work collaboratively with demonstration sites, the
advisory committee, or other necessary organizations;

o  Inability to successfully address key methodological issues; and

o  Substantive changes in the agreed-upon protocol with which the
AHCPR does not agree.

These special Terms of Award are in addition to and not in lieu of
otherwise applicable PHS grants policies and Federal regulations.

Collaborative Responsibilities

As discussed under Phase 1 activities, grantees must establish an
Advisory Committee to provide overall policy guidance and technical
expertise, and assist in conflict resolution.  The membership of the
Advisory Committee may include, in addition to the Principal and
Co-Investigators, the AHCPR project officer, consumers, methodology
experts in the fields of health care consumer survey research, social
marketing (with experience in developing health information for
consumers), health insurance and other health-related research areas;
representatives of public and private groups from Phase 2
demonstration sites, including employers, unions, health plan
administrators, and other health insurance providers; and local and
national health service providers and institutions.

SPECIAL INSTRUCTIONS TO APPLICANTS CONCERNING INCLUSION OF WOMEN AND
MINORITIES IN RESEARCH INVOLVING HUMAN SUBJECTS

It is the policy of the AHCPR that women and members of minority
groups be included in all AHCPR supported research projects involving
human subjects, unless clear and compelling rationale and
justification are provided that inclusion is inappropriate with
respect to the health of the subjects or the purpose of the research.

A new NIH policy resulting from the NIH Revitalization Act of 1993
(Section 492B of Public Law 103-43) supersedes and strengthens NIH's
previous policies (Concerning the Inclusion of Women in Study
Populations, and Concerning the Inclusion of Minorities in Study
Populations), which were in effect since 1990 and which AHCPR had
adopted.  The new NIH policy contains provisions that are
substantially different from the 1990 policies.  All investigators
proposing research involving human subjects should read the "NIH
Guidelines for Inclusion of Women and Minorities as Subjects in
Clinical Research," which have been published in the Federal Register
of March 28, 1994 (FR 59 14508-14513), and reprinted in the NIH GUIDE
FOR GRANTS AND CONTRACTS of March 18, 1994, Volume 23, Number 11.

Investigators may obtain copies from these sources or from the AHCPR
contractor, Global Exchange listed under INQUIRIES. AHCPR staff may
also provide information concerning this policy.

LETTER OF INTENT

Prospective applicants are asked to submit, by May 20, 1995, a letter
of intent that includes the name, address, and telephone number of
the proposed Principal Investigator and other key personnel; the
identities of proposed consortia members, including any other
participating organizations or institutions; a descriptive title of
the proposed demonstration project; and the number and title of the
RFA.

Although a letter of intent is not required, is not binding, and does
not enter into the consideration of any subsequent application, the
information allows AHCPR staff to estimate the potential review
workload and avoid conflicts of interest in the review.

The letter of intent is to be sent to the CAHPS project officer at
the address listed under INQUIRIES.

APPLICATION PROCEDURES

Applications are to be submitted on research grant application form
PHS 398 (rev. 9/91).  State and local government applicants may use
form PHS 5161, Application for Federal Assistance.  These forms are
available at most institutional offices of sponsored research; the
Office of Grants Information, Division of Research Grants, National
Institutes of Health, Westwood Building, Room 449, Bethesda, MD
20892, telephone (301) 710-0267.  For AHCPR, applications are
available from Global Exchange Inc., 7910 Woodmont Avenue, Suite 400,
Bethesda, MD 20814-3015, telephone (301) 656-3100, Fax (301)
652-5264.

The RFA label available in the PHS 398 (rev. 9/91) application form
must be affixed to the bottom of the face page of the application.
Failure to use this label could result in delayed processing of the
application such that it may not reach the review committee in time
for review.  In addition, the RFA title and number must be typed on
line 2a on the face page of the application form and the YES box must
be marked.

Complete information must be submitted with the application.
Consortium arrangements typically take the form of a formal agreement
between an applicant and other organization(s).  In the grant
application, a separate budget page and budget justification must be
included for each organization involved in the proposed consortium
arrangement.

Submit a signed, typewritten, original of the application, including
the Checklist, and three signed legible copies (two copies when using
the PHS 5161) in one package to:

Division of Research Grants
National Institutes of Health
6701 Rockledge Drive, Room 1040 - MSC 7710
Bethesda, MD  20892-7710
Bethesda, MD  20817 (for express mail or courier)

Applications submitted under this RFA must be received by June 20,
1995, by the Division of Research Grants, NIH.  If an application is
received after that date, it will be returned to the applicant.  At
the time of submission, two additional copies of the application (one
copy when using the PHS 5161) must be sent to the CAHPS project
officer at the address under INQUIRIES.

Conference for Prospective Applicants

The AHCPR plans to convene a conference for prospective applicants on
April 5, 1995 in Rockville, Maryland.  Attendance is not a
prerequisite to applying.  Attendees must pay for their own travel
and accommodation costs.  For further information, contact (301)
594-1357 ext 105.

REVIEW CONSIDERATIONS

All applications will be judged on the basis of the scientific and
technical merit of the proposed project and the documented ability of
the investigator to meet the objectives of the RFA.  Upon receipt,
applications will be reviewed for completeness by the Referral
Office, Division of Research Grants, NIH, and by AHCPR staff for
responsiveness to the RFA.  Incomplete and nonresponsive applications
will be returned to applicants without further consideration.  The
determination of any application as nonresponsive will be the sole
responsibility of AHCPR.

All accepted applications will undergo peer review for scientific and
technical merit by a special review group convened by the AHCPR.
Review criteria for AHCPR grant applications are:  significance and
originality from a scientific and technical viewpoint; adequacy of
the method; availability of data or proposed plan to collect data
required for the project; qualifications and experience of the
Principal Investigator and proposed staff; adequacy of the plan for
organizing and carrying out the project; reasonableness of the
proposed budget; and adequacy of the facilities and resources
available to the applicant.  Although the technical merit of the
proposed protocol is important, it is not the sole criterion for
selection of the awardee(s).

Applications may undergo triage by the peer review group on the basis
of relative scientific and technical competitiveness.  The AHCPR will
withdraw from further consideration those applications judged to be
non-competitive for award and notify the applicant Principal
Investigator and institutional official.  When an application is
reviewed, the peer review committee may recommend further
consideration or no further consideration.  The committee also
assigns priority scores to the applications for which further
consideration is recommended.  Recommendations of the peer review
committee may be reviewed subsequently by AHCPR's National Advisory
Council for Health Care Policy, Research, and Evaluation.  The peer
review process is rigorous, and only those applications judged to be
of greatest merit will be recommended for further consideration.

Special Review Criteria

In addition to the review criteria noted above, the review committee
will evaluate each application in response to this RFA against the
following special scientific and technical review criteria:

1.  Understanding of issues related to completing the activities
outlined in OBJECTIVES AND SCOPE and STUDY DESIGN.

2.  Understanding of all issues related to the measurement of
consumer assessments of health care plans and services.

3.  Understanding of the needs of various audiences who may use
health care information (e.g., consumers, purchasers, plans,
providers, payers, State and local-level health care organizations)
and the ways in which they may use this information (selection of
plans and providers; comparison of plan/provider performance;
identification of local or multi-site quality problems).

4.  Demonstrated experience in the performance of all aspects of
fielding a large-scale, multi-site survey, including but not limited
to:  formulation of a methodologically sound design; performance of
validity and reliability studies on large-scale survey instruments;
identification and resolution of methodological issues associated
with sample selection and sample size, and with administration of
surveys to the subpopulations of interest and within a variety of
health care delivery settings; identification and resolution of
methodological issues associated with mode of administration.

5.  Demonstrated ability to produce detailed survey operations
manuals suitable for use by groups with minimal as well as extensive
experience in fielding large-scale, multi-site surveys.

6.  Demonstrated ability to cooperate with, provide technical
assistance or training to, and/or monitor the progress of potential
collaborating organizations or test sites; particularly, other groups
such as consortium members, subgrantees, subcontractors,
community-based organizations, data supplying organizations,
respondents, and site representatives.

7.  Demonstrated ability to collaborate with government agencies in
the performance of research studies and demonstrations.

8.  Demonstrated ability to develop and test attractive,
understandable, usable health-related materials for a variety of
consumers.

9.  Demonstrated ability to design and implement process and outcome
evaluations, including:  specification of evaluation questions and
outcome measures; use of quantitative and qualitative data collection
methods; data analysis; development of data-based recommendations and
use of the recommendations to improve products and procedures.

AWARD CRITERIA

Applications will compete for available funds with all other
applications for this RFA.  The following will be considered in
making funding decisions:  quality of the proposed project as
determined by scientific and technical merit; program balance,
including the likelihood of successful collaboration based upon
sufficient compatibility of features; and availability of funds.  The
earliest anticipated date of award is September 30, 1995.

INQUIRIES

Written and telephone inquiries concerning this RFA are encouraged.
Copies of this RFA and background documents are available from:

Global Exchange Inc.
7910 Woodmont Avenue, Suite 400
Bethesda, MD  20814-3015
Telephone:  (301) 656-3100
FAX:  (301) 652-5264

Copies of the RFA without background materials can also be requested
by e-mail at cahpsrfa@PO3.AHCPR.GOV, the NIH GOPHER (gopher.nih.gov),
and AHCPR InstantFAX at (301) 594-2800, AHCPR Pub. No. 95-0044.  To
use InstantFAX, you must call from a fax machine with a telephone
handset.  Use the key pad on the receiver when responding to prompts
from InstantFAX.  The RFA will be sent at the end of the ordering
process.  AHCPR InstantFAX operates 24 hours a day, 7 days a week.
For questions about this service, call AHCPR's Division of
Communications at (301) 594-1364 ext. 159.

Direct inquiries regarding programmatic issues, including information
on the policy of inclusion of women and minorities in study
populations, to:

Christine Crofton, Ph.D.
CAHPS Project Officer
Agency for Health Care Policy and Research
2101 East Jefferson Street, Suite 603
Rockville, MD  20852-4908
Telephone:  (301) 594-1357 ext 105

Direct inquiries regarding fiscal matters to:

Mr. Ralph L. Sloat
Grants Management Officer, Office of Management
Agency for Health Care Policy and Research
2101 East Jefferson Street, Suite 601
Rockville, MD  20852-4908
Telephone:  (301) 594-1447

AUTHORITY AND REGULATIONS

This program is described in the Catalog of Federal Domestic
Assistance No. 93.226.  Awards are made under authorization of the
Public Health Service Act, Title IX (42 U.S.C. 299-299c-6).  Awards
are administered under the PHS Grants Policy Statement and Federal
Regulations 42 CFR 67, Subpart A, and 45 CFR Parts 74 and 92.  This
program is not subject to the intergovernmental review requirements
of Executive Order 12372.

The PHS strongly encourages all grant and contract recipients to
provide a smoke-free workplace and promote the non-use of all tobacco
products.  In addition, Public Law 103-227, the Pro-Children Act of
1994, prohibits smoking in certain facilities (or in some cases, any
portion of a facility) in which regular or routing education,
library, day care, health care or early childhood development
services are provided to children.  This is consistent with the phs
mission to protect and advance the physical and mental health of the
american people.

.

Return to RFAs Index

Return to NIH Guide Main Index

	Office of Extramural Research (OER)			National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892			Department of Health and Human Services (HHS)
Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, Audio or Video files, see Help Downloading Files.