Skip Navigation
U.S. Department of Health & Human Services Logo
OER Logo   OER Public Websites Archive   Archive  
This website contains archived materials provided for historical reference purposes only.
The content and links are no longer maintained and may be outdated.
Archive Home
About Grants
Grants Process
Electronic Grants
Funding Opportunities
NIH-Wide Initiatives
Forms & Deadlines
Grants Policy
News & Events
About OER

Related Archives         
ARCHIVED - Peer Review Oversight Group (PROG) on the Rating of
Grant Applications (RGA): Deliberations and Decisions

At the November 20-21, 1996, meeting of the Peer Review Oversight Group (PROG), a major topic was the Rating of Grant Applications (RGA). At the previous meeting (July, 1996), PROG members had discussed the recommendations in the RGA report, and considered information which had been obtained from the scientific community through a variety of channels. At that time, the PROG decided to table several of the ten recommendations in the RGA report and to focus primarily on whether to use explicit criteria to structure the review, if so what these criteria should be, whether to score/rate those individual criteria, and whether to retain reviewer assignment of a global score or derive an overall score from criteria subscores. Pilot experiments were recommended, and the preliminary results of pilots by both the Division of Research Grants and the National Institute of Allergy and Infectious Diseases were presented to the members at the November meeting.

Based on the information available, the following decision was made after the November meeting: There will be no changes in the basic numerical scoring system at this time. There was enthusiasm for having reviewers continue to assign a global score to each application and that practice will continue. There was generally low enthusiasm for the idea of assigning scores, whether numeric or alphabetic, to the individual criteria or deriving an overall score using such subscores, and those practices will not be adopted, but will be discussed further at the Feburary PROG meeting. The issue of the rating scale itself has been tabled for future discussion; for the present time, the 1.0-5.0 rating scale, with 1 as the best possible score, will be retained.

In addition, there was much discussion of the information on the use of criteria, and indications are leading in the general direction of using criteria to structure the written critique and the review discussion. A final decision on this issue will be made in January or February 1997 based on additional information currently being obtained from program staff. The criteria to be used would include at least Impact, Feasibility, and Investigator/Environment; whether Innovation/Creativity would be best dealt with as a separate criterion is under continuing consideration. There was general agreement on the importance of fostering innovation and creativity in research, and considerable discussion on how best to ensure that such applications were submitted, receive appropriate review and are supported by the Institutes and Centers. Regardless of whether a separate review criterion is adopted, the consideration of Innovation/ Creativity in review and in research in general will continue to be of the utmost importance for the PROG and for the NIH.

The criteria under consideration are stated below, and will be accompanied by some guidelines or suggestions/examples of what they might encompass:

IMPACT: The extent to which, if successfully carried out, the project will make an original and important contribution.

FEASIBILITY: The extent to which the conceptual framework, design, methods, and analyses are adequately developed, well integrated, and appropriate to the aims of the project.

INVESTIGATOR/ ENVIRONMENT: The extent to which the investigators, available resources, institutional commitment, and any other unique features will contribute to the success of the proposed research.

INNOVATION/CREATIVITY (working definition): The extent to which the project employs novel concepts, approaches or methodology.

It was emphasized at the meeting that it is important for the scientific community to understand that the use of these criteria does not represent a change in the PHS 398 instructions, nor does it add criteria to the areas that reviewers are instructed to use in considering the scientific merit of applications during the review process. Rather, this effort represents a reformatting or reorganization of the presentation of the information, in the hope that it will result in more informative and perhaps more balanced critiques. This should be an advantage to those who rely on this information, both program staff in the Institutes and Centers and research investigators.

A working group was formed to gather information on the views of NIH program staff regarding the use of a specific criteria for creativity will be chaired by Dr. Claude Lenfant, Director of the National Heart Lung and Blood Institute and will include representatives from the biological, clinical and behavioral sciences.

Note: The above summarizes only the issue on the Rating of Grant Applications discussed by the PROG; for more detail on these deliberations and a summary of the other issues discussed, see the Summary Minutes of the PROG for November, 1996, which will be posted on the World Wide Web at on the Grants page under Peer Review Issues. At the next PROG meeting (Feb. 13-14, 1997) members will discuss the issue of how to evoke creative, innovative projects, the possible use of a separate criterion for creativity, and the wording of any further explanation of the criteria to be used, along with other agenda items.

Archive web This web page is archived and provided for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the Archive Home Page for more details about archived files.