Release Date:  August 28, 2001

RFA:  RFA-HS-02-001

Agency for Healthcare Research and Quality
Letter of Intent Receipt Date:  September 24, 2001
Application Receipt Date:       November 14, 2001

The Agency for Healthcare Research and Quality (AHRQ) invites applications 
for cooperative agreements to build on and expand the project known as 
Consumer Assessment of Health Plans Study (CAHPS).  AHRQ funded the initial 
CAHPS cooperative agreements from October 1995 through September 2001.  In 
this document, we refer to that project period as “CAHPS I.” The current RFA 
will be referred to as “CAHPS II.”  The purposes of CAHPS I  were to conduct 
research to: 1) produce reliable, valid and rigorously tested survey 
protocols for collecting information from consumers regarding their 
assessments of health plans and services; 2) develop and test the 
effectiveness of different formats for conveying resulting information to 
consumers; 3) evaluate the use of resulting survey protocols in applied 
settings; and 4) evaluate the usefulness of this information in assisting 
consumers, and purchasers acting on their behalf, in making informed 
selections of health care plans and services.  Detailed information on CAHPS 
I including all products developed under CAHPS I are available at

The present RFA builds upon the earlier work to enrich the evidence base 
concerning consumer assessment and reporting and enhance the usefulness of 
CAHPS products.  Its specific purposes are to conduct research to 1) maintain 
existing CAHPS products, 2) build upon three existing CAHPS products relating 
to nursing homes, group practices and persons with mobility impairment, 3) 
develop new products, including innovative ways of reporting CAHPS data, 4) 
assess the usefulness of CAHPS data for quality improvement purposes and 5) 
evaluate the effectiveness of CAHPS in applied settings.

The Public Health Service (PHS) is committed to achieving the health 
promotion and disease prevention objectives of "Healthy People 2010," a 
PHS-led national activity for setting health improvement priorities for the 
United States.  AHRQ encourages applicants to submit grant applications with 
relevance to the specific objectives of this initiative.  Potential 
applicants may obtain a copy of "Healthy People 2010" at


AHRQ strongly encourages all eligible and qualified organizations to respond 
to this RFA.  Status as a CAHPS I grantee is not a prerequisite for this RFA.  
Though each applicant organization or team  must have experience in all areas 
described in the “Objectives” section, this experience need not be based on 
CAHPS products.  We have referenced as many published and unpublished 
documents and reports as possible for the purpose of describing the CAHPS I 
experience to potential new CAHPS to this RFA.  This RFA includes areas of 
effort not addressed in CAHPS I: the use of CAHPS data for quality 
improvement purposes and a greatly expanded version of the reporting task 
(developing a comprehensive research approach to the development and testing 
of reporting formats).  Both these areas call for different types of 
expertise above and beyond those required by CAHPS I.
Applications may be submitted by domestic and foreign, public and private 
non-profit organizations, including universities, clinics, units of State and 
local governments, and eligible components of agencies of the Federal 
Government.  Under recently enacted reauthorization legislation, AHRQ is 
authorized to enter into cooperative agreements with for-profit organizations 
as well as with public and not-for-profit entities.  Thus, for-profit 
organizations are invited to respond to this notice with research 
applications for cooperative agreements.  Such applications will be 
administered in accordance with Subpart E of 45 CFR Part 74 and 42 CFR Part 
67 Subpart A.  The latter regulation has not yet been amended to reflect 
these changes in Agency name and authority.  (See December 6, 1999 
reauthorization at  AHRQ encourages women, 
members of racial and ethnic minority groups, and persons with disabilities 
to apply as Principal Investigators.


The administrative and funding instrument to be used for this program will be 
a cooperative agreement (U18), an "assistance" mechanism (rather than an 
"acquisition" mechanism), in which substantial AHRQ scientific and/or 
programmatic involvement with the awardee is anticipated during performance 
of the activity.  Under the cooperative agreement, the AHRQ purpose is to 
support and/or stimulate the recipient's activity by involvement in and 
otherwise working jointly with the award recipient in a partner role, but it 
is not to assume direction, prime responsibility, or a dominant role in the 
activity.  Details of the responsibilities, relationships and governance of 
the study to be funded under cooperative agreement(s) are discussed later in 
this document under the section “Terms and Conditions of Award.”

The total project period for applications submitted in response to the 
present RFA may not exceed 5 years.  The anticipated award date is Spring 

This RFA is a one-time solicitation.  AHRQ has not determined whether or how 
this solicitation will be continued beyond the present RFA.

We have chosen the cooperative agreement mechanism for CAHPS because it 
allows AHRQ to have a greater involvement in facilitating the collaborative 
efforts of the consortium members.  Continuing involvement in CAHPS research 
also enables AHRQ staff to serve more effectively as liaisons with other 
governmental and non-governmental agencies who are interested in using CAHPS 


AHRQ expects to award from $3 million to $4.5 million total costs (direct 
costs plus facilities and administrative costs) in fiscal year 2002 to 
support the first year of $1 million to $1.5 million to approximately three 
grantees under this RFA.  The total award amount is dependent on the final 
fiscal year 2002 budgets for both AHRQ and the Center for Medicare and 
Medicaid Services (CMS, formerly the Health Care Financing Administration).

The actual number of applications funded is dependent on the number of high 
quality applications and their individual budget requirements; it is not the 
intent of AHRQ that awards be equal in size.  Should additional funds become 
available and if AHRQ receives a sufficient number of meritorious 
applications, AHRQ reserves the right to fund additional applications under 
this RFA.

Although this program is provided for in the financial plans of AHRQ, awards 
pursuant to this RFA are contingent upon the availability of funds for this 
purpose.  Funding beyond the initial budget period will depend upon annual 
progress reviews by AHRQ and the availability of funds.


The work of the CAHPS I grants and the Survey User Network (SUN) contract has 
been performed by dividing team members into groups that perform specific 
tasks or work towards defined goals.  Since the names of these working groups 
are mentioned in resource documents referenced in this RFA, we identify these 
groups by name and describe their composition in this section. 

Since the division of team members as described in this section served us 
well in CAHPS I, it makes sense to discuss their use in CAHPS II.  After 
awards are made, the grantees will meet with the Project Officers to discuss 
how the work can best be conducted.  They will then decide together on the 
best way to subdivide the grantee teams to facilitate the completion of 
agreed-upon goals. 

The SUN or Survey User Network contract is a five year contract for the 
provision of various types of support to the CAHPS consortium.  This support 
includes preparing CAHPS products for dissemination to potential users; 
providing technical assistance to users; providing communication support to 
facilitate the work of the CAHPS consortium; providing technical and 
conference support for meetings; and providing support for a National CAHPS 
Benchmarking Database (NCBD) described below.  During CAHPS I, the SUN 
contractor played a critical role in identifying the needs of current users 
of CAHPS products, responding quickly and efficiently to those needs, and 
developing effective strategies for communicating with users on a regular 
basis.  The ways in which the grantees will interact with the SUN contractor 
will be described in greater detail in the Study Organization and Budget and 
Related Issues sections. Additionally, details regarding the scope of work of 
the SUN contract can be found on the AHRQ website at

The NCBD contains two major data sets: CAHPS survey response data and health 
plan descriptive data.  The primary purpose of the NCBD is to facilitate 
comparisons of CAHPS results among various types of CAHPS survey sponsors, 
including Medicaid agencies, public and private employers, and individual 
health plans. These comparisons are intended to provide both purchasers and 
health plans with benchmarking information useful for evaluating and 
improving performance as measured by the CAHPS survey. The NCBD also offers 
an important source of primary data for conducting specialized research that 
will enable improvements in future survey design and a better understanding 
of health plan and enrollee characteristics that influence performance. All 
sponsors of CAHPS surveys that are administered independently according to 
CAHPS specifications are invited to participate in the NCBD. Survey sponsors 
include public and private purchasers (employers, state Medicaid agencies, 
coalitions) and individual health plans.  The confidentiality of individual 
respondent information is maintained.  The NCBD also has several working 
groups including the NCBD Advisory Board, Technical Advisory Panel, and 
Executive Research Committee.  Grantees will participate in each of these 
workgroups.  Further information regarding grantee responsibilities will be 
presented in the Budget and Related Issues section. Additional information 
about the NCBD can be found at

“The CAHPS consortium” refers to the entire line-up of people who work on 
CAHPS, direct its work or advise the grantees.  This includes all grantees, 
AHRQ CAHPS staff (Project Officers and others), the SUN contractor, the CAHPS 
Steering Committee (to be described in the Study Organization section) and 
the CAHPS Grantee Advisory Committee (to be described in the Study 
Organization section).  Throughout this RFA, the term “CAHPS consortium” is 
used interchangeably with “the CAHPS team.”
Each grantee and their consortium/contractual partners/collaborators are 
referred to as a “grantee team.”  However, when the term “grantee team” is 
used throughout this solicitation, it is understood that the primary grantee 
is responsible for the project, which includes the programmatic and fiscal 
management of third parties.

We refer to the CAHPS Survey and Reporting Kit, or CSRK, throughout this RFA.  
The CSRK  contains all CAHPS products as well as detailed advice on their 
use.  Portions of the kit are available through the AHRQ website in addition 
to instructions for obtaining a complete kit, if desired.  

Finally, the name of this project has been updated to reflect its expanded 
focus.  In the final years of CAHPS I, the team began development of tools 
through which consumers and patients can assess care given to them by 
providers (as members of group practices) and care they receive in nursing 
homes.  These efforts will be built upon in CAHPS II.  Since the focus of our 
product development has expanded beyond health plans, the name “Consumer 
Assessment of Health Plans” no longer accurately describes the team’s work.  
However, CAHPS has become a recognized product name in the industry.  In view 
of this, we are keeping the project name “CAHPS” but will no longer use the 
original name for which this used to be an abbreviation. 


1.  Project Goals and Objectives
A brief summary of the development of products during CAHPS I and some 
information about CAHPS users can be found at  We encourage applicants to consult them.  
Our main reason for referencing these reports is to inform potential applicants 
who were not CAHPS I participants about team experiences and decisions made 
during that time frame.  In addition, one of the objectives of this RFA is the 
further development of products of the three CAHPS I projects: nursing home 
CAHPS (NH CAHPS), group practice level CAHPS (GCAHPS) and CAHPS for people with 
mobility impairments (CAHPS PWMI).  Applicants who were not CAHPS I 
participants may need this additional background information and information 
about the status of these three efforts to prepare an application.

Though the background documents represent the current “best thinking” 
regarding development and testing of CAHPS products, applicants should not 
feel constrained by their conclusions or recommendations.  We encourage 
applicants to “think beyond” these materials and present different strategies 
and new approaches to the research challenges stated in this RFA.  In this 
section we describe the goals for CAHPS II. 

This RFA includes five categories of work to be performed across two phases.  
The categories are: 1) refinement of current CAHPS products; 2) continued 
development of NH CAHPS, G-CAHPS and CAHPS PWMI; 3) development of new 
products; 4) quality improvement studies; and, 5) evaluation studies of the 
use of CAHPS products.  Each category is discussed below. Work in these areas 
will occur across two time spans or phases: phase 1 includes years one and 
two of the cooperative agreement; phase 2 includes years three, four and 
five.  The phase or phases within which a product is to be completed are 
listed below. 

With the new emphasis on quality improvement and evaluation research in CAHPS 
II, it is unlikely that any single organization wishing to respond to this 
RFA will contain in itself people with expertise necessary to develop and 
test questionnaires, develop and test reporting templates, modify existing 
CAHPS products, plan and conduct QI studies and perform evaluations.  We 
encourage applicants to team with individuals or organizations who add 
expertise in important areas.  Applications that respond only to individual 
elements of this RFA rather than the entire range of activities will be 
returned as nonresponsive.

Refinement of Current CAHPS Products:
Ongoing through Phases 1 and 2

For CAHPS surveys, refinement involves periodic updating of all survey-
related information in the CAHPS Survey and Reporting Kit and includes work 
on such elements as item wording, administration procedures, sampling 
guidelines, data preparation, analytic strategies, guidance on interpretation 
of results, as well as any other information that would facilitate use of the 
products on the kit.  Grantees are required to maintain existing as well as 
questionnaires under development.

For CAHPS reporting templates, refinement involves periodic updating of all 
print and computer reports-related information contained in the CSRK.  This 
includes such elements as development and testing of explanatory material in 
the templates; development and testing of different data displays; 
reconfiguration (if necessary) of reporting composites; development and 
testing of graphic designs and navigational aids.

Grantees will also be required to update all other CAHPS products as the 
consortium determines necessary.  These products include but are not limited 
to the booklet “Frequently Asked Questions about CAHPS” and the Survey 
Submitter’s Kit.  The current versions of these can be found on the AHRQ 

For all CAHPS products, revisions may also be based on feedback from current 
and potential users, changes in technology, changing requirements of 
accreditation organizations or other sources of information identified by the 
Project Officers.

Further development of G-CAHPS, NH CAHPS and CAHPS PWMI
G-CAHPS: Phase 1
NH CAHPS: Phase 1

In addition to product refinement, grantees will build upon the prior 
development and testing of three CAHPS products (G-CAHPS, NH CAHPS and CAHPS 
PWMI).  The former grantees and SUN contractor will deliver to the CAHPS II 
grantees all relevant materials.

Group Practice Level CAHPS (GCAHPS): The CAHPS team in collaboration with the 
Pacific Business Group on Health and funded jointly by the California Health 
Care Foundation and AHRQ developed and tested a GCAHPS instrument in 
California, Eastern Massachusetts, Denver, Knoxville, and St. Louis. The 
resulting instrument is in a beta test stage. Several other sites around the 
country are administering the survey. Additional questions remain about 
sampling and data collection. The CAHPS II goal is to continue development 
and address these issues so that the GCAHPS survey can be added to the set of 
CAHPS products. The final report on the developmental efforts to this point 
and suggested next steps are included on the AHRQ website.

Nursing Home CAHPS (NHCAHPS): This survey is jointly supported by CMS and 
AHRQ. A draft questionnaire is being cognitively tested with nursing home 
residents with varying levels of cognitive capacity to assess how well they 
can respond to alternate sets of response options. Also a preliminary 
assessment of sampling strategies for use in nursing homes for short stay and 
long stay residents has been prepared. A description of the work completed 
and planned for the current effort is included on the AHRQ website. The goal 
for work under this new CAHPS project will be to prepare a fully field tested 
instrument; sampling strategy and data collection protocol. Applicants are 
urged to offer approaches to achieving this goal by building on the current 

CAHPS for people with mobility impairments (CAHPS-PWMI):  This project is 
still at the conceptual stage of development.  The only activity that has 
been performed thus far is the convening of an expert group to look at the 
issues and challenges related to development of a survey for PWMI.  A summary 
of the April 2001 meeting of this panel can be obtained through the AHRQ 
website.  In the development and testing of CAHPS-PWMI products, grantees 
will collaborate with staff from the National Rehabilitation Hospital Center 
for Health and Disability Research (NRHCHDR), staff from the National 
Institute for Disability and Rehabilitation Research (NIDRR) and members of 
an expert panel convened by AHRQ, NRHCHDR and NIDRR.  

Development of new CAHPS products

Development and testing of PPO supplemental items: Phase 1
Development and testing of individual provider level CAHPS: Phase 1
Translation of questionnaires and reporting templates into other languages: 
Phase 1
Research on cultural comparability: Ongoing throughout phases 1 and 2 
New data collection technologies: Ongoing throughout phases 1 and 2
Development of OMB clearance packages: Ongoing throughout phases 1 and 2

Potential new CAHPS products include (1) development and testing of 
questionnaires, screeners, and supplemental questions; (2) development and 
testing of data collection technologies; and (3) development and testing of 
reporting templates.  Below we describe the types of products that may be 
developed though the budget supporting CAHPS II will not permit development 
of all of them.  Products will be selected after awards are made based on 
recommendations from the grantees, AHRQ assessment of need for the product, 
feedback from the CAHPS Steering Committee, expertise within grantee teams, 
and other sources of information.  Products described below are not listed in 
priority order.

o   Questionnaire development and testing:

--Perform complex literature searches and reviews (along with ability to 
track down relevant unpublished or in-progress studies);

--Conduct feasibility studies to determine whether prospective questionnaires 
are desired by consumers in addition to identify the challenges of collecting 
the data in a cost effective manner;

For example, consumers tell us that they are most interested in CAHPS related 
quality of care information for individual providers. Challenges exist in 
being able to collect these data in a cost effective manner. Applicants 
should propose studies which will respond to these challenges.  The study 
should also address data collection issues to ensure that survey results can 
be examined at the individual provider, group practice, and health plan 
levels.  If the results of the feasibility support the development of the 
questionnaire then the actual instrument may be developed and tested.

--Develop questionnaires for consumer assessment of health plans, services 
and providers.  This includes strategies for specification of content areas; 
preparation of psychometrically equivalent self- and telephone-administered 
survey versions;

During CAHPS II, CAHPS supplement items for PPOs will be developed.  CAHPS is 
designed to be applicable across different types of health care plans.  
However, it is desirable to pursue the development of supplemental items to 
address the specific requirements for assessing PPOs.
--Develop and test screener questionnaires to identify special populations 
such as mobility-impaired individuals;

--Develop and test sampling plans for testing purposes;

--Develop analytic plans for testing data;

--Perform psychometric testing of questionnaires; and,

--Translate questionnaire into languages other than Spanish;

Applicants may also need to re-evaluate the current Spanish language version 
to assess what changes, if any, need to be made in the translation to 
accommodate differences in Spanish dialects and make those changes.

--Evaluate the cultural comparability of questionnaires; and,

--Develop OMB clearance packages.

o   Data collection technologies:

--Explore the feasibility of collecting data via alternate methods such as by 
the internet, email, and other technologies. There are a number of challenges 
to collecting data in a way that will be cost effective and keep respondent 
burden at a reasonable level.  Applicants should offer studies to investigate 
ways in which technologies such as the internet might be used to meet the 

o   Report template development and testing:

Develop and conduct five year program of research for CAHPS reports.  This 
would require grantees to:

–Identify and review the relevant literature, which may include health 
education, consumer choice, social psychology, decision-making research, 
marketing research, research on the design of electronic reports and other 
areas.  The goal for this activity is to identify what we know and need to 
know about the characteristics of decision-makers, format and other features 
of print and non-print materials, decision situations and other areas that 
affect the way people make health care choices.
–Identify and obtain information about projects that are still in progress or 
would not be in the published literature. Locate consumer choice projects for 
which results may not have been published or which may still be in progress.  
Review these projects for relevant information and continue to check on those 
that are incomplete.  Web sites such as may be 
helpful for this activity.
–Formulate research questions and development research plan. Based on this 
information, identify variables that affect consumer choice and develop 
research questions related to these variables.  The reports team should use 
the development of new reporting templates (listed below) as opportunities to 
test research questions.  Since some research questions may be more 
appropriate for study in a laboratory setting, the team will propose and 
implement lab studies as well.
–Plan how report development projects can fit into this framework
–Keep in mind that some questions may best be explored through lab studies
–Prepare final report in year five which details which questions were tested, 
what the results were, what we now know about consumer use of this 
information, what we have left to learn.  

Develop and test reporting templates based on:
–Group practice level CAHPS, (one report for providers, one for consumers)
–Individual provider level CAHPS, (one report for providers, one for 
consumers; if the feasibility study supports this)
–Supplemental PPO items
–CAHPS for children with special health care needs
–Spanish language CAHPS 
–CAHPS in languages in which questionnaires are translated (to be determined 
by the Steering Committee)

This task includes preparation of material for the CSRK, including the 
development of complete instructions for analyzing data, developing 
composites and inserting the data into reporting templates.  All reporting 
templates will need to be developed in electronic and print formats.

Develop and test reporting templates in different languages.

Develop and test reporting templates for purchasers from small or medium-
sized organizations or purchasing coalitions. In many cases, the target 
audience is the benefits manager or the broker contracted to evaluate health 
plans for employee benefits purposes. About 50% of Americans with health 
insurance can choose among two or more health plans.  For others, choice 
among plans occurs at the level of the purchasing coalition, benefits manager 
or other entity that makes the purchasing decision.  People who make these 
decisions, especially those from small or medium sized organizations, may not 
fully understand health plan quality information and so may not be able to 
use this information to inform their selection decision. Additionally, value-
based purchasing requires both cost and quality data.  Cost data are 
difficult to integrate into consumer reporting templates since this 
information is specific to the purchaser.  However, purchasers would 
significantly benefit from having both cost and quality information 
integrated in a report.  Therefore, cost information could be integrated into 
this reporting template.

Develop templates for reporting CAHPS data for various audiences such as 
consumers, patients, health care benefits purchasers, different types of 
health care providers, health care quality improvement specialists, 
accreditation specialists, and researchers;

Reporting development and testing includes:

--Perform cognitive testing of questionnaires and reports for users;

--Recruit people who have poor cognitive skills (rather than relying 
exclusively on convenience samples) for cognitive testing purposes,  

--Conduct usability testing on products such as reports, booklets of 
frequently asked questions and future editions of the CAHPS Survey and 
Reporting Kit;

--Use graphic design and information design principles in the development of 
CAHPS products to “let the people [who use them], working in their own 
physical, social and cultural environments, accomplish their goals and tasks 
effectively and efficiently” (Hackos and Redish, 1998);

Prepare reports based on discussions that recommend different approaches for 
the development of reporting tools and strategies for CAHPS templates.  For 
example, applicants may consider gathering examples of existing documents, 
reports in other than paper media and decision tools the purpose of which is 
to provide health quality information to consumers, patients, and providers 
and developing a list of experts from which to convene a group (or groups) to 
review, discuss and critique a thoughtfully-chosen subset of these documents.  
These experts should represent diverse disciplines and points of view, 
including information design, graphic arts, web page design, health 
communications, decision theory, marketing, social marketing, health 
education and campaign development, “plain language” experts and people who 
prepare materials for those with poor communication or reading skills.  It 
may help to include people who have developed tools in areas other than 
health quality; for example, those who have developed systems to explain 
retirement options and related tax consequences to employees; those who have 
designed information that explains the pro’s and con’s of clinical trials 
participation to candidates; those who have designed successful user manuals 
and/or on-line support for the operation of mechanical or technical 

The second part of the reporting task is to develop templates in the context 
of a carefully-considered program of research that will advance our knowledge 
about effective ways of preparing and disseminating usable quality reports 
for patients, purchasers (consumers, benefits managers and others), health 
care providers and other audiences.  Though we continue to see the 
publication of individual studies which are well-planned and executed, 
research on consumer choice as a whole is not organized or systematic and 
thus has not proceeded as efficiently or effectively as it could.  A planned 
program of research which identifies a series of critical research questions 
and builds on their results over a five year time span could improve this 
situation. This research program will be implemented over the five year span 
of the project.  The reports team will revise it annually to incorporate 
findings from the previous year and feedback from the Steering Committee and 
to include development of different templates as we learn of the need for 
them.  At the end of CAHPS II, this program of research should result in 
systematic advances in what we know about purchaser and provider use of 
health quality information and should make significant contributions to that 

Quality improvement studies.

CAHPS began with a primary goal of providing high-quality information which 
consumers and purchasers would use to support their health plan selection 
decisions. As CAHPS has evolved, there has been  growing interest in 
exploring the usefulness of CAHPS results for designing strategies to improve 
the quality of health care from the consumer’s or patient’s perspective.  
Grantees will design and conduct studies to investigate how CAHPS can be used 
for quality improvement purposes.  

As an example, consider the CAHPS group practice level questionnaire.  As 
discussed above, this questionnaire will require the development of two 
reporting products: one for providers (to summarize survey results and 
identify areas for improvement) and one for consumers (to summarize survey 
results so that consumers can identify group practices that will best meet 
their needs).  Even if providers had these data, they may not understand how 
to make the leap from identification of problem areas (from their patients’ 
point of view) to implementation of improvements.  A CAHPS QI study could 
work with these providers to:
--Understand how providers would like for these data to be reported to them,
--Translate study findings into recommendations for improvement or changes in 
practice behaviors,
--Develop strategies and procedures designed to help change the identified 
--Help implement these change strategies,
--Provide mid-course corrections to the strategies if necessary,
--Develop a process evaluation system to confirm that change strategies were 
correctly implemented,
--Develop an outcome evaluation to determine if the changed behaviors had the 
expected impact, and
--Determine how and when to communicate to patients that concerns they 
conveyed through the survey were recognized and that changes were made.

As this sample task list implies, grantees may need to develop different and 
new CAHPS products to develop and implement a QI study.  All CAHPS 
questionnaires, including the one in development to assess care given at the 
group-practice level, were constructed to enable comparisons across different 
plans in different settings.  An intended and positive consequence of this is 
that the surveys produce data that allow comparisons between a PPO in 
California and a fee-for-service plan in Virginia.  A negative consequence 
(at least for the purpose of quality improvement) is that these data reflect 
views of the plan (or health care services obtained through the plan) from 
“50,000 feet” rather than at “ground-level,” i.e., that the results are not 
clearly linked with reproducible steps for improvement.  But the ability to 
develop an effective QI plan depends on specific, detailed ground-level 
information about the quality of services delivered.  This suggests that, 
though items from CAHPS surveys can serve as “red flags” for areas that need 
to be improved, it is likely that the developer of a QI program would need to 
collect additional qualitative or quantitative data to pinpoint specific 
behaviors that need to be changed. 

For example, if consumer ratings on the CAHPS group-provider level instrument 
identify a group practice with low scores in the Communication/Interpersonal 
Care domain, a researcher would need to know what specific provider behaviors 
led to this assessment.  Did the physician not explain the purpose of ordered 
tests or test results well enough?  Was the physician not sufficiently aware 
of the health issues that most concern the patient?  How did this affect  
treatment?  For responses to these and similar questions, the researcher may 
need to develop a qualitative data collection protocol such as focus groups 
or individual interviews.  Once the researcher has a good understanding of 
behaviors that prompted low ratings in this domain, he can then present the 
information to the providers in that group practice and work with them to 
develop strategies for change.

Like all CAHPS products, all interview protocols, focus group discussion 
guides and other such materials developed for QI purposes will be available 
to users free of charge.

In CAHPS II, grantees will design and implement quality improvement (QI) 
studies based on CAHPS data.  Each applicant to this RFA should propose a QI 
study which includes at least these elements:
–Use of an existing CAHPS instrument to identify an area which requires 
–Development and use of additional qualitative or quantitative data 
collection tools, if necessary;
–Design and implementation of an intervention; and
–Evaluation of the effectiveness of the intervention.  The evaluation should 
include a readministration of the CAHPS instrument used for problem 
identification to see if the instrument is sensitive enough to detect changes 
in plan/provider behavior.

GCAHPS should be completed in time for use in proposed QI studies.  
Therefore, applicants are encouraged to consider either the health plan or 
group level for proposed studies. Additionally, applicants may also want to 
consider collaborations with Peer Review Organizations to conduct these 

--Identify and establish partnerships with appropriate sites for QI studies;

--Develop strategies for using CAHPS data for quality improvement purposes.  
This may involve development and testing of additional qualitative or 
quantitative instruments;

--Develop strategies for translating these data into recommendations for 
improvement tailored to the specific target audiences;

--Develop programs for assisting plans and care providers in using CAHPS data 
to improve the quality of their programs and services;

Evaluation studies

The development of CAHPS products will require both process and outcome 
evaluation studies.  Grantees will perform two types of process evaluation 
studies: The first (sometimes called “formative evaluation”) requires 
thorough testing of materials with the target audience while materials are 
still in draft status.  “Materials” include report segments, reports in their 
entirety, questionnaire items and draft questionnaires, elements of the 
Survey and Reporting Kit, quality improvement plans and strategies and other 
materials.  The second type of process evaluation refers to obtaining user 
feedback about how final versions of CAHPS products and procedures work in 
their intended settings.  We are particularly interested in process 
evaluations of new products and procedures but need to collect additional 
data for existing products (like the Survey and Reporting Kit) so that we can 
continue to make data-based improvements to them. 

In general, outcome evaluation seeks to answer the question, “Did the 
intervention achieve the expected change?”  In CAHPS II, applicants will need 
to design and implement evaluations that demonstrate the effectiveness of: a) 
QI interventions based on areas of concern identified by patients through a 
CAHPS instrument and b) the effectiveness of CAHPS report in supporting 
consumer choice of health plans or providers appropriate for their needs and 

--Specify appropriate evaluation questions, identify appropriate outcome 
measures and appropriate respondents for each question, and specify times at 
which these questions should be asked.

--Develop an appropriate evaluation design.  The design should:  a) eliminate 
as many threats to validity as possible; and b) result in a clear 
demonstration of usefulness of the data-based reports in assisting consumer 
decision-making in an open enrollment situation, as well as identification of 
factors related to product effectiveness or ineffectiveness.

--Identify and use appropriate qualitative and quantitative data collection 

--Analyze data appropriately.  Applicants should pay particular attention to 
strategies for analysis of qualitative data, as well as the issue of 
conveying information from quantitative evaluation analyses in a manner that 
is understandable by and useful to personnel at sites implementing the survey 

--Develop recommendations based on evaluation data that can be used to modify 
CAHPS projects or procedures.  One component of the CAHPS evaluation plan 
should be the development of clearly written, user-friendly recommendations 
designed to promote data-based changes to products and procedures. 

-Identify and develop partnerships with sites appropriate for evaluation 

Examples of the kinds of questions that may be explored through this research 
include the following: 

We know that people need more than statistics to support their selection of a 
health plan or provider.  To make proper use of comparison data, people need 
to understand critical differences among health care delivery systems, the 
importance of a quality-based decision, and how to organize and prioritize 
diverse pieces of information for each of what may be several alternatives.  
How do we produce a report that covers all these bases but is brief enough to 
appeal to users who will spend only minutes reviewing the data and making 
their decision?

A related issue concerns people’s diverse needs for information.  Is it 
possible to design a single report that “layers” information to accommodate 
people who want only a listing of the top three plans available to them as 
well as people who want individual item scores for each variable? 

About 50% of Americans who have health insurance have the choice of two or 
more plans.  Benefits managers, other institutional purchasers, and brokers 
contracted by employers also often need to evaluate and choose among several 
plans.  A recent study by Iyengar and Lepper (2000) shows that increasing the 
number of alternatives among which one can choose can easily overwhelm the 
chooser and produce the opposite of the desired behavior.  How many health 
plans can consumers or purchasers evaluate before they feel overwhelmed?  How 
can we structure materials and the decision-making task to avoid overwhelming 

How do we design non-print modes of reporting (video, interactive video, web-
based reports) that are low cost and relatively easy to install and maintain?  
How can these alternative materials be disseminated or made accessible to 

Consumers want information about health care quality and how plans and 
providers compare to one another but are often intimidated by displays of 
statistical data.  How can we present quantitative data in a manner that 
allows reliable and valid comparisons but is also as easy to understand and 
user-friendly as narrative reports? 

People with extremely poor cognitive skills or those who have mobility, 
cognitive or sensory impairments may be unable to complete paper-and-pencil 
questionnaires or to read or understand data in typical reports.  How do we 
produce reports that are useful to these people?

Are high users of health care services (e.g. children with special health 
care needs, people with mobility impairments) likely to use quality reports 
in different ways than people who are not high users?  If so, how should we 
design report templates for these audiences?

What are the best modes of dissemination for each of our target audiences?

Recent research has shown that quality reports that emphasize the opportunity 
to protect oneself from risk are more compelling to purchasers than those 
that emphasize potential gain or benefit (Hibbard, 2000).  Is this finding 
solid enough that we should incorporate risk-based messages into the design 
of CAHPS reporting templates?  How can we develop risk-based messages that 
motivate consumers but do not alienate the plans whose data we are reporting?  
Are there other ways of reframing key messages in quality reports that will 
increase the likelihood that purchasers will pay attention to and use the 

Development of experiments that respond to these research questions as well 
as development of quality reports that consumers will read and find useful 
are tasks that require multiple types of expertise.  For example, these tasks 
require researchers who have: the methodological expertise to structure 
rigorous experiments both in laboratory and real world settings; knowledge of 
decision theory especially as it relates to consumer decisions; experience 
applying social marketing principles to the design of materials for 
consumers, patients and health care providers; experience with graphic 
design; knowledge of research which shows how existing quality reports have 
succeeded and failed; experience with design of electronic reports; expertise 
in document design and design of materials in plain language; experience with 
the design and evaluation of health messages and health education campaigns.  
In their applications, applicants should identify the expertise required to 
support their approach to report development and testing and include people 
experienced in these areas as part of their project team. 

3.  Study Organization

Each grantee must propose a Principal Investigator as well as a co-Principal 
Investigator who will be responsible for all aspects of project management 
when the lead person is unavailable.  These individuals should coordinate 
their schedules so that one or the other is always available.

The Principal Investigator, or the person she designates, serves as the 
official voice of that grantee team.  Communication from the grantee team to 
the AHRQ Project Officer should come from the Principal Investigator rather 
than from multiple members of that team.

Applicants should propose staff with a range of experience and skill levels.  
Our goal is to avoid the situation where extremely qualified or senior level 
staff are performing tasks that a person with less experience could perform 
just as well.

Grantees should also propose a lead staff person (and alternate) for each of 
four “cross-grantee teams” (instrumentation, reports, quality improvement 
studies and evaluation studies).  During the course of the project, other 
cross-grantee teams may be created. “Cross-grantee teams” are composed of 
members of each grantee team organized to perform defined tasks.  During 
CAHPS II, we will require that one staff person from the SUN contract 
participate in each cross-grantee team to prepare and publish call notes and 
sometimes to participate in the content of the work.  In CAHPS I, an 
instrumentation team was organized, as well as teams for reports, evaluation, 
survey and reporting kit development, user group meeting agenda development 
and other purposes.  The length of time cross-grantee teams are needed is 
dependent upon the work being performed by the team.  For example, the 
instrumentation and reports teams of CAHPS I were formed early in the project 
and continued to meet (by conference call and in-person) throughout its life.  
Other groups are formed for shorter term goals (e.g., development of an 
agenda for a user group meeting) and disband when those goals have been met.  
Sometimes certain tasks can be performed most efficiently through the 
development of sub-groups. For example, a sub-group of the survey and report 
kit team was developed to proof-read and edit the final version. The 
Objectives and Methods sections contains information about the substance of 
work for four of the cross-grantee teams.

Some of the long-term cross grantee teams (such as instrumentation and 
reports) have “conveners.”  The convener moderates conference calls and 
meetings; works with the designated SUN staff person to arrange the call, 
publish a call agenda and meeting materials on the CAHPS team web site; helps 
the team develop a long-term agenda including a task list and time line; 
reminds the team of impending due dates; and meets with the Project Officers 
on a regular basis to discuss progress toward goals, barriers and how to 
resolve them.

For critical periods during CAHPS I, the Principal Investigators met weekly 
by phone and sometimes in person with the AHRQ project officers, 
representatives from the SUN contractor, team conveners and other people as 
required.  There will be two major components to the organizational structure 
of the CAHPS II project–the CAHPS Steering Committee and the CAHPS Grantee 
Advisory Committee.  Members of the CAHPS Steering Committee will be: the 
AHRQ Project Officers, at least one representative from each of the 
organizations which partner with AHRQ to fund the CAHPS grants, the CAHPS 
Prinicpal Investigators and their alternates.  The project director of the 
Survey User Network contract will attend these meetings to provide 
information about CAHPS user needs but will not have an official voice in 
determining direction of the work of the cooperative agreements.  At this 
time, CMS is expected to be a CAHPS funding partner.  Other funding partners 
may be identified in the future. Additional members will be added as 
necessary as decided upon by the existing Committee members. 

The Steering Committee will set the overall direction of the CAHPS project 
and decide on project priorities from year to year. The Committee will 
establish a five year plan with broad goals at the beginning of the project. 
Each year it will review progress towards these goals, review emerging needs 
for new or different CAHPS products and adjust project direction if 
necessary.  More detailed information regarding grantees’ roles on the 
Steering Committee and in setting the project direction can be found in the 
“Process for Setting Project Direction subsection under the Special 
Requirements section below.

The CAHPS Grantee Advisory Committee will be comprised of representatives of 
the key stakeholders affected by the implementation of CAHPS products.  These 
might include representatives of consumers/patients; health care providers, 
purchasers; health plans; accrediting organizations and others.  The role of 
the Grantee Advisory Committee will be to advise the CAHPS grantees about the 
needs and concerns of the organizations who use CAHPS products.  The grantees 
will select the members of this group, set meeting agendas, make all meeting 
arrangements, and convene the group as often as necessary.  One 
representative from each organization on the Steering Committee are ex 
officio members of the Grantee Advisory Committee.

Interactions among the Steering Committee, the Grantee Advisory Committee and 
other CAHPS teams are discussed in the Special Requirements section.

4.  Budget and Related Issues

As mentioned earlier in this RFA, the CAHPS Steering Committee will determine 
the research and product development agenda on a year-by-year basis.  This 
approach constitutes the best strategy for meeting user needs as they evolve.  
But since applicants need to prepare five-year budgets which must include 
these as yet unspecified activities, we offer the following assumptions. 

-For budgeting purposes, applicants need to develop budgets devoting 
approximately 1/3 effort for each of the three areas listed below.

o  Continued development of GCAHPS and Nursing Home CAHPS (for completion of 
both GCAHPS and NH CAHPS over years 1 and 2)
o  PWMI CAHPS (over years 1 and 2)

o  Individual provider level CAHPS (over years 1 and 2)

-Applicants can find project summaries and a listing of remaining tasks for 
completion of Nursing Home CAHPS and for GCAHPS at  For the other two surveys, the CAHPS 
Submitter’s Kit found at the same website is a good source of information 
concerning critical characteristics of questionnaires that bear the CAHPS 
trademark.  As mentioned in the “Objectives,” subsection above applicants 
should view this document as a background resource and are not required to 
abide by its recommendations or suggestions.  

-The three grantees will be working collaboratively on GCAHPS, Nursing Home 
CAHPS, PWMI CAHPS, and individual provider level CAHPS so each applicant 
should describe their ability to perform all tasks necessary to construct a 
valid survey from item generation through development of complete and easy-
to-understand administration procedures. 

-Applicants should assume that the activities listed in the Timetable Section 
will occur during the five year time-span of this project.  

-Each grantee should budget $10,000 for each of years one and two for 
consultation with one staff person from NRHCHDR for CAHPS PWMI.

-Each reporting product must be developed and tested in print and electronic 

-Reports and questionnaires must be tested with users who have poor cognitive 
skills as well as with those who have average cognitive skills.

-One purpose of this RFA is to develop and test tools (e.g., questionnaires, 
reporting templates, analytic procedures) through which users can obtain 
assessments of health care quality from the consumer or patient point of view 
and report them to the appropriate audiences.  An additional purpose is to 
stimulate research that advances knowledge in the areas of:  l) consumer and 
patient-based health care quality assessment, 2) reporting quality 
information to various users, 3) use of assessment data for quality 
improvement purposes and 4) evaluation of quality improvement programs.  The 
“Objectives” section of this RFA contains many examples of research questions 
that could be addressed or studies that could be performed which would 
advance the knowledge in one of these areas.  Though applicants need not use 
these particular examples of research questions or studies, they should 
devote some part of the budget to the design and execution of research that 
will advance the field in one or more of these four areas.   

o   Meetings

For each year of the contract, the appropriate grantee representative(s) will 
be required to attend the following meetings:
- weekly conference calls with Principal Investigators, their back-ups and 
Project Officers
- cross-grantee team meetings as necessary (in-person as necessary; assume 
ten person trips)
- one Expert Panel meeting (in person)
- two Steering Committee meetings (in-person)
- two Grantee Advisory Committee meetings (in-person)
- one CAHPS User Group annual meeting (in-person)
- one NCBD Advisory Committee meeting (in-person)
- one NCBD Technical Advisory Panel (TAP) meeting
- one NCBD Executive Research Committee meeting
- five online workshops for CAHPS users

–For in-person meetings (e.g., of the Grantee Advisory Committee, cross-
grantee teams, entire consortium meetings, etc.), grantees may use free of 
charge the AHRQ conference facilities currently at 6010 Executive Boulevard, 
Rockville, Maryland.  Use of these facilities are subject to availability. 

–The grantee teams will share the cost of transportation, lodging and per 
diem for Advisory Committee member attendance at each meeting.  All other 
costs for these meetings (obtaining space, materials preparation, etc.) will 
be borne through the SUN contract.

–The grantee teams will share the cost of transportation, lodging and per 
diem for Expert Panel member attendance at each meeting.  All other costs for 
these meetings (obtaining space service, materials preparation, etc.) will be 
borne through the SUN contract.

–The CAHPS consortium holds one meeting of CAHPS user per year.  Though the 
SUN contractor takes the lead responsibility for this meeting, each of the 
grantees will contribute to planning the content of and making presentations 
at the meeting. Each grantee will be allocated two registrations free of 
charge.  Additional grantee team members that attend the meeting must submit 
registration fees.

o   Interaction with the SUN contractor

A member of the SUN contract team will be present for all meetings.  The SUN 
Contractor will work with grantees to schedule meetings and to prepare 
materials for those meetings. Additionally, the SUN contractor will summarize 
all meetings. Grantees will be required to work with the SUN contractor for 
the following:
- developing, testing, and revising the CSRK
- updating other CAHPS products as necessary
- answering technical questions that arise from the technical assistance to 
CAHPS users, if necessary
- providing all meeting materials, reports, and other items to the SUN 
contractor in a timely manner
- serving on the NCBD Advisory Committee, TAP, and Executive Research 
- participating in five online user workshops

o   Reports

The following reports will be required during CAHPS II:
- Summary report for each new survey instrument developed describing any 
difficulties encountered in the course of planning, implementing and 
evaluating the surveys
- Five year research plan for developing and testing reporting templates
- Summary report for each new reporting template developed describing any 
difficulties encountered in the course of planning, implementing and 
evaluating the templates
- Final report of quality improvement study
- Final report for all evaluation studies conducted
- Non-competitive renewal proposals at the end of each grant year
- Final report at the end of year five on the advances made to reporting 
quality data through these cooperative agreements
- Two summary reports per grant year describing progress
- Timeline and table of activities for each cross-grantee project (updated as 
necessary as decided by the Principal Investigators and the Project
- Notes and action items from annual AHRQ site visits
- Any other interim or final reports agreed upon during CAHPS II

5.  Timetable

Phase one includes years one and two of the project; Phase two includes years 
three, four, and five. The activities listed in this section describe AHRQ 
and CMS expectations for projects to be conducted and may change based on 
input from members of the Steering Committee.  Also, the “Objectives” section 
of this RFA contains topics that have been identified as important by CAHPS 
stakeholders but are not included on this list.  The Steering Committee may 
determine that studies based on these (or other) topics are worth pursuing.   

Phase I Activities
-GCAHPS survey
-NHCAHPS survey
-CAHPS-PWMI survey
-PPO supplemental items
-Individual provider level CAHPS (ongoing)
-New data collection technologies
-Translations of CAHPS into other languages (ongoing)
-Development of five year research plan on reporting quality information to 
various audiences 
-Research on the cultural comparability of CAHPS (ongoing)
-Spanish language CAHPS reporting template
-Translation of CAHPS reporting template into one other language
-OMB clearance packages (ongoing)

Phase II Activities
-Individual provider level CAHPS
-New data collection technologies
-Translations of CAHPS into other languages
-Research on the cultural comparability of CAHPS
-GCAHPS reporting template
-CAHPS-PWMI reporting template
-Translation of CAHPS reporting template into two languages
-Quality improvement studies
-Evaluation studies
-OMB clearance packages

Annual Meetings and Reports
-Progress reports (Months one and seven of grant year)
-Grantee Advisory Committee meeting (Months two and eight of grant year)
-Steering Committee meetings (Months three and nine of grant year)
-Non-competitive renewal proposal (Month eleven of grant year)
-CAHPS User Group meeting (currently anticipated for Spring 2002; may be 
scheduled in the fall for subsequent grant years)

1.  Reporting requirements

As soon as possible after the award date, the Principal Investigators and 
will meet in person with the AHRQ project staff and SUN contractor to review 
proposed project tasks for years one and two, make any agreed upon or 
necessary changes to the list, identify the subgroups required to perform 
various tasks, and determine which grantee will take the lead for which 
product (e.g., Spanish language version of the CAHPS reporting template) or 
project (e.g., feasibility study for an individual provider level 
instrument).  We will also determine who the government contact person will 
be for each product and project as well as whether a consortium-wide, in-
person meeting is required at that point or at some later point.  Remaining 
decisions will be discussed during weekly conference calls. 

In order to accomplish the Agency’s mission of translating research into 
practice and policy, researchers must inform the Office of Health Care 
Information when articles from the study are accepted for publication in 
journals.  Researchers should also discuss ideas about other dissemination 
and marketing efforts with the Office of Health Care Information at AHRQ so 
that efforts to disseminate and translate the research may be coordinated 
with the Agency in order to maximize the knowledge and application of the 
research by potential users, including clinicians, patients, health care 
systems and purchasers and policymakers.

The work of this RFA requires close coordination with the SUN contract 
activity.  One major jointly developed product is the CSRK, CAHPS Survey and 
Reporting Kit.  The grantees’ have principle responsibility for development 
of the first draft text for the CSRK.  The SUN contractor and the AHRQ Office 
of Health Communications Information (OHCI) are responsible for final 
formatting, editing, dissemination and actual physical production of the kit.  
Because so many team members from different parts of various organizations 
are involved in the CSRK and because CAHPS users have time-sensitive needs 
for information in the CSRK, failure by any single part of the team to meet 
agreed-upon deadlines can have disastrous consequences.  So it will be 
critically important for designated writers on grantee teams to meet the 
deadlines to which they agree. 

The CAHPS project officers will perform annual site visits with each of the 
grantees.  During the site visits (which usually last a day and a half), the 
grantees will present evidence of progress on each of the tasks for which 
they have responsibility; discuss problems, questions or other issues with 
the project officers; and work with the project officers to lay out 
activities for the coming year. 

Process for setting project direction

The process for setting the project direction is as follows:

--Drawing upon grantee work plans, the Steering Committee establishes the 
overall direction of the project with a five year plan at the beginning of 
the project. The Committee will meet twice during each grant year during the 
life of the project to establish or refine priorities for the coming year. 
The timeframe for these meetings can be found in the Timetable section.

--All Steering Committee members will participate in discussions of project 
direction and the distribution of work among the grantees.  Based on these 
discussions, the grantee representatives on the Steering Committee will be 
responsible for jointly proposing projects along with the distribution of 
work among the grantees.  

--Prior to the completion of each year of the grants the CAHPS grantee teams 
and the SUN contractor will consult with the CAHPS Grantee Advisory Committee 
to review project progress and secure their input into priorities for the 
upcoming year.

--CAHPS grantee teams will develop proposals for the upcoming year reflecting 
the Steering Committee’s priorities and input from the Grantee Advisory 
Committee. These proposals may suggest modifications to the these priorities.

--The Steering Committee will review proposals and decide which activities 
and products will be pursued.

--The grantee teams will modify proposals as necessary to address Steering 
Committee’s priorities. 

See the section below entitled “AHRQ Staff Responsibilities” for 
circumstances under which AHRQ may terminate or curtail the cooperative 
agreement activities.

Terms and Conditions of Award
These special Terms of Award are in addition to and not in lieu of otherwise 
applicable OMB administrative guidelines, HHS grant administration 
regulations at 45 CFR Parts 74 and 92, 42 CFR Part 67 Subpart A, and other 
HHS, PHS grants administration policy statements.  Applicants should be 
familiar with the Agency’s grant regulations, 42 CFR Part 67 Subpart A, and 
particularly sections 67.18-67.22.
Consistent with the U18 concept defined under Mechanism of Support, the 
dominant role and prime responsibility for the activity resides with the 
awardee(s) for the project as a whole, although specific tasks and activities 
in carrying out the studies will be shared among the awardees and the AHRQ 
Program Official(s).

1.  Awardee Rights and Responsibilities

o   General

Awardees will have primary and lead responsibilities for the project as a 
whole including:

–Survey instrument development and appropriate testing;

–Feasibility studies as necessary to support the development of new CAHPS 

_The grantee will write up notes and follow-up actions discussed at this 
meeting for submission to the Project Officers;   

–Report template development and testing;

–Translation of instruments and report templates into other languages;

–Evaluation of the cultural comparability of questionnaires;

–Design, implementation, and analysis of quality improvements studies, 
including evaluation of the impact of intervention(s);

–Design, implementation, and analysis of evaluation studies;

–Development of relationships with testing sites, QI study sites, and 
evaluation sites;

–Formation of the Grantee Advisory Committee, including selection of members, 
convening meetings, planning meeting agendas and documenting activities;

–Provision of AHRQ access to all data generated under this award as it is 
collected; and

–Cooperation with other key parties in these cooperative agreements, 
including but not limited to the SUN contractor; other CAHPS grantees; and 
AHRQ Project Officers and other AHRQ staff.  

o     Meetings

For each year of the contract, the appropriate grantee representative(s) will 
be required to attend the following meetings:
- weekly conference calls with Principal Investigators, their back-ups and 
Project Officers
- cross-grantee team meetings as necessary (in-person as necessary; assume 
ten person trips)
- one Expert Panel meeting (in person)
- two Steering Committee meetings (in-person)
- two Grantee Advisory Committee meetings (in-person)
- one CAHPS User Group annual meeting (in-person)
- one NCBD Advisory Committee meeting (in-person)
- one NCBD Technical Advisory Panel (TAP) meeting
- one NCBD Executive Research Committee meeting
- five online workshops for CAHPS users

–Grantees will need to cover all meeting costs (tape-recording, etc.) for in-
person meetings with the exception of the CAHPS User Meeting and the Advisory 
Committee meetings.  These costs include facility costs if AHRQ conference 
facilities are not used.    

–The grantee teams will share the cost of transportation, lodging and per 
diem for Advisory Committee member attendance at each meeting.  All other 
costs for these meetings (obtaining space, materials preparation, etc.) will 
be borne through the SUN contract.

–The grantee teams will share the cost of transportation, lodging and per 
diem for Expert Panel member attendance at each meeting.  All other costs for 
these meetings (obtaining space, materials preparation, etc.) will be borne 
through the SUN contract.

–The CAHPS consortium usually holds one meetings of CAHPS users per year.  
Though the SUN contractor takes the lead responsibility for this meeting, 
each of the grantees will contribute to planning the content of and making 
presentations at the meeting. Each grantee will be allocated two 
registrations free of charge.  Additional grantee team members that attend 
the meeting must submit registration fees.

o   Interaction with the SUN contractor

Grantees will be required to work with the SUN contractor for the following:
- developing, testing, and revising the CSRK
- updating other CAHPS products as necessary
- answering technical questions that arise from the technical assistance to 
CAHPS users, if necessary
- providing all meeting materials, reports, and other items to the SUN 
contractor in a timely manner
- serving on the NCBD Advisory Committee, TAP, and Executive Research 
- participating in five online user workshops

o   Reports

The following reports will be required during CAHPS II:
- Summary report for each new survey instrument developed describing any 
difficulties encountered in the course of planning, implementing and 
evaluating the surveys
- Five year research plan for developing and testing reporting templates
- Summary report for each new reporting template developed describing any 
difficulties encountered in the course of planning, implementing and 
evaluating the templates
- Final report of quality improvement study
- Final report for all evaluation studies conducted
- Non-competitive renewal proposals in Month eleven of each grant year
- Final report at the end of year five on the advances made to reporting 
quality data through these cooperative agreements
- Two summary reports per grant year describing progress (Months one and 
seven of grant year)
- Timeline and table of activities for each cross-grantee project (updated as 
necessary as decided by the Principal Investigators and the Project
- Notes and action items from annual AHRQ site visits
- Any other interim or final reports agreed upon during CAHPS II

2.  AHRQ Staff Responsibilities

The AHRQ Project Officer and other AHRQ staff will have substantial 
scientific and programmatic involvement during the conduct of this activity 
through technical assistance, advice, and coordination beyond the normal 
program stewardship for grants.  Collaboration on the development of the 
testing plan, evaluation design and survey protocol will occur after the 
award is made.  Specifically, AHRQ's role in the cooperative agreement is to 
provide technical assistance, advice, and support to the PI and to ensure 
that all related AHRQ-supported projects complement one another.  AHRQ will:

--Participate in all key project decisions, including but not limited to:  
identification of focal consumer groups, assessment of consumer information 
needs, selection of items/question modules to be tested, identification and 
resolution of methodological issues; 

--Work with the grantee to identify and negotiate with sites for quality 
improvement and evaluation studies, with the AHRQ project officer retaining 
right of final approval;

--Establish collaboration with and obtain project-related information from 
other Federal agencies and programs;

--Disseminate tested question modules and other research findings as they 
become available; and

--Participate in or coordinate strategy sessions with the grantees on an as-
needed basis.

--AHRQ will require prior written approval for the addition or deletion of a 
participating or collaborating institution, site, or other organizational 
These special Terms of Award are in addition to and not in lieu of otherwise 
applicable PHS grants policies and Federal regulations.

Any disagreement that may arise with respect to scientific or programmatic 
matters (within the scope of the award), between award recipients and AHRQ 
may be brought to arbitration.  An arbitration panel will be composed of 
three members – one selected by the Steering Committee (with the AHRQ member 
not voting) or by the individual awardee in the event of an individual 
disagreement, a second member selected by AHRQ, and the third member selected 
by the two prior selected members.  This special arbitration procedure in no 
way affects the awardee’s right to appeal an adverse action that is otherwise 
appealable in accordance with the PHS regulations at 42 CFR Part 50, Subpart 
D and HHS regulation at 45 CFR Part 16.
Data Privacy

Pursuant to section 924(c) of the Public Health Service Act (42 USC 299c-
3(c)), information obtained in the course of any AHRQ-supported study about 
an identifiable individual or entity must be treated as confidential in 
accordance with any promises made or implied regarding the possible uses and 
purposes of the data collection.  In the Human Subjects section of the 
application, applicants must describe procedures for ensuring the 
confidentiality of such identifying information.  The description of the 
procedures should include a discussion of who will be permitted access to the 
information, both raw data and machine readable files, and how personal 
identifiers and other identifying or identifiable data will be restricted and 
safeguarded.  Plans for compliance with the Privacy Rule as 45 CFR Part 164, 
if applicable, should be set forth.

The grantee should ensure that computer systems containing confidential data 
have a level and scope of security that equals or exceeds those established 
by the Office of Management and Budget (OMB) in OMB Circular No. A-130, 
Appendix III - Security of Federal Automated Information Systems.  The 
National Institute of Standards and Technology (NIST) has published several 
implementation guides for this circular.  They are: An Introduction to 
Computer Security: The NIST Handbook; Generally Accepted Principals and 
Practices for Securing Information Technology Systems; and Guide for 
Developing Security Plans for Information Technology Systems.  The circular 
and guides are available on the web at  The application of these 
confidentiality and security standards to subcontractors and vendors, if any, 
should be addressed.

These and other requirements applicable to AHRQ grantees as well as important 
legal rights are set out or referenced in the Agency’s grants regulations at 
42 CFR 67, Subpart A.  (Available in libraries and from the GPO’s website

Rights in Data

AHRQ grantees may copyright or seek patents, as appropriate, for final and 
interim products and materials, material they developed individually: 
including, but not limited to, methodological tools, measures, software with 
documentation, literature searches, and analyses, which are developed in 
whole or in part with AHRQ funds.  Such copyrights and patents will be 
subject to a worldwide irrevocable Federal government license to use and 
permit others to use these products and materials for government purposes.  
In accordance with its legislative dissemination mandate, and subject to 
statutory confidentiality protections, AHRQ purposes may include, subject to 
statutory confidentiality protections, making research materials, data bases, 
results, and algorithms available for verification or replication by other 
researchers; and subject to AHRQ budget constraints, final products may be 
made available to the health care community and the public by AHRQ or its 
agents, if such distribution would significantly increase access to a product 
and thereby produce public health benefits.  Ordinarily, to accomplish 
distribution, AHRQ publicizes research findings through the media and other 
appropriate channels in coordination with AHRQ’s Office of Health Care 
Information but relies on grantees to publish research results in peer-
reviewed journals and to market grant-supported products.


It is the policy of AHRQ that women and members of minority groups be 
included in all AHRQ-supported research projects involving human subjects, 
unless a clear and compelling rationale and justification are provided that 
inclusion is inappropriate with respect to the health of the subjects or the 
purpose of the research.  

All investigators proposing research involving human subjects should read the 
UPDATED "NIH Guidelines on the Inclusion of Women and Minorities as Subjects 
in Clinical Research," published in the NIH Guide for Grants and Contracts on 
August 2, 2000 
A complete copy of the updated Guidelines is available at  To the 
extent possible, AHRQ requires adherence to these NIH Guidelines.

Investigators may obtain copies from the above sources or from the AHRQ 
Publications Clearinghouse, listed under INQUIRIES, or from the NIH Guide Web 

AHRQ also encourages investigators to consider including children in study 
populations, as appropriate.

AHRQ Program staff may also provide additional information concerning these 
policies (see INQUIRIES).


Prospective applicants are asked to submit, by September 24, 2001, a letter 
of intent that includes a descriptive title of the proposed research, name, 
address, and telephone number of the Principal Investigator, the identities 
of other key personnel and participating institutions, and the number and 
title of the RFA in response to which the application may be submitted.
Although a letter of intent is not required, is not binding, and does not 
enter into consideration of any subsequent applications, the information 
allows AHRQ staff to estimate the potential review workload and to avoid 
conflict of interest in the review.  AHRQ will not provide responses to 
letters of intent.
The Letter of Intent is to be sent to Maria Wey  (see address under 


The PHS 398 research grant application instructions and forms (rev. 5/2001) 
available at are to 
be used in applying for these grants. This version of the PHS 398 is 
available in an interactive, searchable PDF format. Although applicants are 
strongly encouraged to begin using the 5/2001 revision of the PHS 398 as soon 
as possible, the NIH will continue to accept applications prepared using the 
4/1998 revision until January 9, 2002. Beginning January 10, 2002, however, 
the NIH will return applications that are not submitted on the 5/2001 
version. For further assistance contact GrantsInfo, Telephone 301/710-0267, 

State and local government applicants may use PHS 5161-1, Application for 
Federal Assistance (rev. 5/96), and follow those requirements for copy 

AHRQ is not using the Modular Grant Application and Award process.  
Applicants for funding from AHRQ should ignore application instructions 
concerning the Modular Grant Application and Award process, and prepare 
applications according to instructions provided in form PHS 398.  
Applications submitted in the modular format will be returned without review.

Beginning with applications submitted for the February 1, 2001 receipt date, 
Institutional Review Board (IRB) approval of human subjects is not required 
prior to peer review of an application.  The “AHRQ Revised Policy for IRB 
Review of Human Subjects Protocols in Grant Applications” was published in 
the NIH Guide on September 27, 2000.

Applications kits are available at most institutional offices of sponsored 
research.  They may also be obtained from the Division of Extramural Outreach 
and Information Resources, National Institutes of Health, 6701 Rockledge 
Drive, MSC 7910, Bethesda, MD 20892-7910, telephone (301) 710-0267, E-

AHRQ applicants are encouraged to obtain application materials from the AHRQ 
Publications Clearinghouse  (see INQUIRIES).

The RFA label available in the PHS 398 (rev. 5/2001) application form must be 
affixed to the bottom of the face page of the application.  Type the RFA 
number on the label.  Failure to use this label could result in delayed 
processing of the application such that it may not reach the review committee 
in time for review.  In addition, the RFA title and number must be typed on 
line 2 of the face page of the application form and the YES box must be 
marked. The RFA label is also available at:

A complete, and signed, typewritten original of the application, including 
the Checklist and three signed photocopies, must be submitted in one package 

Center for Scientific Review
National Institutes of Health
6701 Rockledge Drive, Room 1040 - MSC 7710
Bethesda, MD  20892-7710
(20817 for express/courier service)
At the time of submission, two additional copies of the application, labeled 
“Advanced Copy (s)” must also be sent to:

Maria Wey
Center for Quality Improvement and Patient Safety
Agency for Healthcare Research and Quality
6011 Executive Blvd., Suite 200
Rockville, MD  20852
Telephone:  (301) 594-6626 
FAX:  (301) 594-2155
E-mail Address:

Applications must be received by November 14, 2001.  An application received 
after the deadline may be acceptable if it carries a legible proof-of-mailing 
date, assigned by the carrier, and the proof-of-mailing is not later than 1 
week prior to the deadline date. If an application is received after that 
date, it will be returned to the applicant without review.  

Application Preparation (for Using CMS Data)

For applications that propose to use Medicare or Medicaid data that are 
individually identifiable, applicants should state explicitly in the 
“Research Design and Methods” section of the Research Plan (form 398) the 
specific files, time periods, and cohorts proposed for the research.  In 
consultation with Center for Medicare and Medicaid Services (CMS), previously 
called Health Care Financing Administration (HCFA), AHRQ will use this 
information to develop a cost estimate for obtaining the data.  This estimate 
will be included in the estimated total cost of the grant at the time funding 
decisions are made. 

Applicants should be aware that for individually identifiable Medicare and 
Medicaid data, Principal Investigators and their grantee institutions will be 
required to enter into a Data Use Agreement (DUA) with CMS to protect the 
confidentiality of data in accordance with AHRQ’s confidentiality statue, 42 
USC 299c3(c), the Privacy rules at 45 CFR Part 164, if applicable, and 
standards set out in OMB Circular A-130, Appendix III–Security of Federal 
Automated Information Systems.  The use of the data will be restricted to the 
purposes and time period specified in the DUA.  At the end of this time 
period, the grantee will be required to return the data to CMS or certify 
that the data have been destroyed.  

Unless AHRQ is able to negotiate exceptional arrangements, for the sole 
purpose of assuring that data confidentiality is maintained, included in the 
DUA is the requirement that the User agrees to submit to CMS, a copy of all 
findings within 30 days of making such findings.  The user further agrees not 
to submit these findings to any third party (including but not limited to any 
manuscript to be submitted for publication) until receiving CMS’s approval to 
do so.

In developing research plans, applicants should allow time for refining, 
approving, and processing any CMS data requests.  Requests may take 6 months 
from the time they are submitted to complete.  Applications proposing to 
contact beneficiaries or their providers require the approval of the CMS 
Director and may require meeting(s) with CMS staff.

CMS data are provided on IBM mainframe tapes using the record and data 
formats commonly employed on these computers.  Applicants should either have 
the capability to process these tapes and formats or plan to make 
arrangements to securely convert them to other media and formats.

Questions regarding CMS data should be directed to the AHRQ program official 
listed under INQUIRIES.

In carrying out its stewardship of research programs, the AHRQ, at some point 
in the future, may begin requesting information essential to an assessment of 
the effectiveness of Agency research programs.  Accordingly, grant recipients 
are hereby notified that they may be contacted after the completion of awards 
for periodic updates on publications resulting from AHRQ grant awards, and 
other information helpful in evaluating the impact of AHRQ-sponsored 

AHRQ expects grant recipients to keep the Agency informed of publications as 
well as the known uses and impact of their Agency-sponsored research.  
Applicants are to agree to notify AHRQ immediately when a manuscript based on 
research supported by the grant is accepted for publication, and to provide 
the expected date of publication as soon as it is known, regardless of 
whether or not the grant award is still active. 

To receive an award, applicants must agree to submit an original and 2 copies 
of an abstract, executive summary, and full report of the research results in 
the format prescribed by AHRQ no later than 90 days after the end of the 
project period.  The executive summary should be sent at the same time on a 
computer disk which specifies on the label the format used (WP5.1 or WP6.0 is 

AHRQ plans to present a webchat to respond to questions from prospective 
applicants on September 20, 2001 at 4 p.m. EST.  Participation not a 
prerequisite to applying.  Applicants who are interested in participating 
should send a written request to Maria Wey (contact information can be found 
in the INQUIRIES section). For further information, contact (301) 594-6626.

Upon receipt, applications will be reviewed for completeness by CSR and 
responsiveness by AHRQ.  Incomplete applications will be returned to the 
applicant without further consideration.  Applications that are complete but 
are not responsive to the RFA will be returned to the applicant without 
further consideration.  Applications that are complete and responsive to the 
RFA will be evaluated for scientific and technical merit by an appropriate 
peer review group convened by the AHRQ in accordance with the review criteria 
stated below.  

As part of the merit review, all applications will receive a written 
critique, and also may undergo a process in which only those applications 
deemed to have the highest scientific merit will be discussed and assigned a 
priority score. 

All applications will be judged on the basis of the scientific merit of the 
proposed project and the documented ability of the investigators to meet the 
research objectives of the RFA.  Although the technical merit of the proposed 
project is important, it will not be the sole criterion for evaluation of a 
study.  Other considerations, such as importance and timeliness of the 
proposed study and access to patients/data will be part of the evaluation 

General Review Criteria

The reviewers will comment on the following aspects of the application in 
their written critiques in order to judge the likelihood that the proposed 
research will have a substantial impact on the pursuit of these goals.  Each 
of these criteria will be addressed and considered by the reviewers in 
assigning the overall score weighting them as appropriate for each 
application.  Note that the application does not need to be strong in all 
categories to be judged likely to have a major scientific impact and thus 
deserve a high priority score.  For example, an investigator may propose to 
carry out important work that by its nature is not innovative but is 
essential to move a field forward.
1.  Significance.  If the aims of the application are achieved, how will 
scientific knowledge reporting quality information be advanced?  What will be 
the effect of these studies on the concepts or methods that drive this field?
2.  Approach.  Are the conceptual framework, design, methods, and analyses 
adequately developed, well-integrated, and appropriate to the aims of the 
project?  Are the proposed data sources appropriate and adequate?  Does the 
applicant acknowledge potential problem areas and consider alternative 
3.  Innovation.  Does the project employ novel concepts, approaches or 
method?  Are the aims original and innovative? Does the project challenge 
existing paradigms or develop new methodologies or technologies?
4.  Investigator.  Is the investigator appropriately trained and well suited 
to carry out this work?  Is the work proposed appropriate to the experience 
level of the Principal Investigator and other researchers (if any)?   Is the 
project (or workplace) well organized?
5.  Environment.  Does the scientific environment in which the work will be 
done contribute to the probability of success?  Do the proposed experiments 
take advantage of unique features of the scientific environment or employ 
useful collaborative arrangements? Is there evidence of institutional 

The initial review group will also examine: the appropriateness of proposed 
project budget and duration; the adequacy of plans to include both genders, 
children, and minorities and their subgroups as appropriate for the 
scientific goals of the research and plans for the recruitment and retention 
of subjects; the provisions for the protection of human and animal subjects; 
and the safety of the research environment.

Special Review Criteria

The special review criteria include experience and skills in consumer 
assessment and reporting of health plans and other providers.  It is not 
necessary that this experience be based on previous CAHPS experience.

1.  Experience with similar activities outlined in the Objectives section.

2.  Experience with the measurement of consumer and patient assessments of 
health care plans, services and providers.

3.  Experience with the needs of various audiences who may use health care 
information (e.g., consumers, patients, purchasers, plans, providers, payers, 
State and local-level health care organizations, etc.) and the ways in which 
they may use this information (selection of plans and providers; comparison 
of plan/provider performance; identification of local or multi-site quality 
problems; etc.).

4. Experience with all aspects of fielding a large-scale, multi-site survey, 
including but not limited to:  formulation of a methodologically sound 
design; performance of validity and reliability studies on large-scale survey 
instruments; identification and resolution of methodological issues 
associated with sample selection and sample size, and with administration of 
surveys to the subpopulations of interest and within a variety of health care 
delivery settings; identification and resolution of methodological issues 
associated with mode of administration.   

5.  Experience with producing detailed survey operations manuals suitable for 
use by groups with minimal as well as extensive experience in fielding large-
scale, multi-site surveys.   

6.  Experience with identifying and recruiting appropriate sites for survey 
instrument testing, report template testing, QI studies, and evaluation 

7.  Experience with collaborating with, provide technical assistance or 
training to, and/or monitor the progress of test sites, community-based 
organizations, data supplying organizations and site representatives.

8.  Experience with collaborating with government agencies in the performance 
of research studies and demonstrations.

9.  Experience with developing and testing understandable and useful print 
and electronic reports which contain health quality data for use by a wide 
range of audiences (patients, consumers, providers, health plans, national, 
state and local policy makers, government agencies, etc.).

10.  Experience in designing questionnaires and reports for people with poor 
cognitive skills and/or poor reading skills.

11.  Experience in designing, implementing and evaluating quality improvement 
programs in health care settings.

12.  Experience in designing and implementing process and outcome 
evaluations, including:  specification of evaluation questions and outcome 
measures; use of quantitative and qualitative data collection methods; data 
analysis; development of data-based recommendations and use of the 
recommendations to improve products and procedures.

13.  Experience in participating as a productive member of a multi-
organization team over a lengthy performance period in the development of 
high-quality products and procedures for their use.

Applications will compete for available funds with all other applications 
under this RFA.  The following will be considered in making funding 
decisions: 1) quality of the proposed project as determined by peer review; 
2) availability of funds; 3) program balance with respect to types insurance 
structure and organizational delivery system features, enrollee versus 
community studies, populations studied, geography including rural/urban mix, 
regional representation, and targeting understudied areas with unmet 
evaluation needs; 4) compatibility with other awardees with respect to 
cooperative activities; and 5) policy relevance. 


Copies of the RFA and copies of the grant application form PHS 398 (rev. 
5/2001) are available from:

AHRQ Publications Clearinghouse
P.O. Box 8547
Silver Spring, MD  20907-8547
Telephone:  1-800-358-9295 
TDD Service:  888-586-6340

The RFA is also available on AHRQ’s Web site,, and 
through AHRQ InstantFAX at (301) 594-2800.  To use InstantFAX, you must call 
from a facsimile (FAX) machine with a telephone handset.  Follow the voice 
prompt to obtain a copy of the table of contents, which has the document 
order number (not the same as the RFA number).  The RFA will be sent at the 
end of the ordering process.  AHRQ InstantFAX operates 24 hours a day, 7 days 
a week.  For comments or problems concerning AHRQ InstantFax, please call 
(301) 594-6344.

AHRQ welcomes the opportunity to clarify any issues or questions from 
potential applicants who have read the RFA.  Written and telephone inquiries 
concerning this RFA are encouraged.   Direct inquiries regarding programmatic 
issues, including information on the inclusion of women, minorities, and 
children in study populations to:
Maria Wey
Center for Quality Improvement and Patient Safety
Agency for Healthcare Research and Quality
6011 Executive Blvd., Suite 200
Rockville, MD  20852
Telephone:  (301) 594-6626 
FAX:  (301) 594-2155
E-mail Address:

Direct inquiries regarding fiscal and eligibility matters to:
Al Deal

Grants Management Specialist
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 601
Rockville, MD  20852
Telephone: (301) 594-1843
FAX: (301) 594-3210
This program is described in the Catalog of Federal Domestic Assistance, 
Number 93.226.  Awards are made under Title IX of the Public Health Service 
Act (42 USC 299-299c-7) as amended by P.L. 106-129 (1999).  Awards are 
administered under the PHS Grants Policy Statement and Federal Regulations 42 
CFR 67, Subpart A, and 45 CFR Parts 74 or 92.  This program is not subject to 
the intergovernmental review requirements of Executive Order 12372 or Health 
Systems Agency review.

The PHS strongly encourages all grant and contract recipients to provide a 
smoke-free workplace and promote the non-use of all tobacco products.  In 
addition, Public Law 103-227, the Pro-Children Act of 1994, prohibits smoking 
in certain facilities (or in some cases, any portion of a facility) in which 
regular or routine education, library, day care, health care or early 
childhood development services are provided to children.  This is consistent 
with the PHS mission to protect and advance the physical and mental health of 
the American people.


Website references:  SUN Contract -, 
NCBD -, Talking Quality -  

CAHPS Documents (available on the AHRQ website):  CAHPS Survey and Reporting 
Kit, GCAHPS Final Report, CAHPS-PWMI Expert Panel Meeting Summary, NH-CAHPS 
Summary, Frequently Asked Questions about CAHPS booklet, CAHPS Submitter’s 
Kit, Report of Input from Stakeholders on Design of CAHPS II, SUN contract 
scope of work.

Hackos JT, Redish JC. User and Task Analysis for Interface Design. 1998; John 
Wiley & Sons, Inc.:New York, NY.

Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garkinkel S. Increasing 
the impact of health plan report cards by addressing consumers' concerns. 
Health Aff. 2000 Sep-Oct;19(5):138-43.

Iyengar SS, Lepper MR. When choice is demotivating: can one desire too much 
of a good thing?  J Pers Soc Psychol. 2000 Dec;79(6):995-1006.

Shaller DV, Sharpe PS, Rubin RD. A national action plan to meet health care 
quality information needs in the age of managed care. JAMA. 

Return to Volume Index

Return to NIH Guide Main Index

Office of Extramural Research (OER) - Home Page Office of Extramural
Research (OER)
  National Institutes of Health (NIH) - Home Page National Institutes of Health (NIH)
9000 Rockville Pike
Bethesda, Maryland 20892
  Department of Health and Human Services (HHS) - Home Page Department of Health
and Human Services (HHS) - Government Made Easy

Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, Audio or Video files, see Help Downloading Files.