REINVENTION ACTIVITIES RELATED TO NIH EXTRAMURAL PROGRAMS STATUSREPORT: DECEMBER 1994 NIH GUIDE, Volume 23, Number 44, December 16, 1994 P.T. 34 Keywords: Grants Administration/Policy+ National Institutes of Health Vice President Gore's National Performance Review (NPR) outlined a plan to reinvent the Federal government to work better and cost less. In response to this directive and the accompanying mandate to reduce the size of the Federal workforce, the NIH extramural programs have been designated one of several Executive Branch "Reinvention Laboratories," to lead the process of change for grants administration in the Public Health Service (PHS). The imprimatur of this status has facilitated the efforts of the Office of Extramural Research (OER) to reevaluate NIH policies and procedures. As a consequence, both the momentum to make changes and the opportunity to effect them have increased. A large number of initiatives are currently underway at NIH. Some are new and directly responsive to the mandate of reinvention; others are a continuation of ongoing efforts to improve the way NIH does business. Some address issues specific to extramural activities, while others address overarching issues at the NIH, PHS, or HHS level that potentially have impact on extramural programs. They emanate from ideas contributed by individuals and groups, both within and outside of NIH. When feasible, pilot studies are being conducted. The activities are coordinated by the Extramural Reinvention Committee (ERC), which operates under the auspices of the OER and ensures sharing of knowledge and prevents duplication of effort. An essential feature of this coordination is that it does not serve a "gatekeeping" function, i.e., it does not serve as a bureaucratic impediment to the free flow of ideas, activities of working groups, or development of pilot experiments. The reinvention activities outlined below are organized into three sections: those which directly impact on the submission of competing applications and the peer review process; those which deal with post- award issues; and those which aim to improve the efficiency and cost- effectiveness of internal NIH processes. The reinvention activities discussed below range from those that are fully developed to those that are still in the conceptual phase. They represent the present state of an evolving operation, which will continue to rely on the valued input of all interested parties. This report is one of a series in which the state of reinvention activities will be updated. PEER REVIEW AND PROCESSING OF APPLICATIONS: Triage - (full implementation beginning February 1995) - Beginning with the February 1995 round of review (i.e., review of applications submitted October/November 1994), all DRG study sections will employ the triage process routinely, and Initial Review Groups (IRGs) within Institutes and Centers (ICs) may use it at their option. This represents an extension across NIH of a triage process that had been used successfully for a number of years by the ICs for the review of applications received in response to Requests for Applications (RFAs). In triage, peer reviewers are asked to identify the pending applications (approximately half) that are "unrealistic candidates for funding in the present budgetary environment" (not to be equated with "disapproval" or "not recommended for further consideration"). Those applications are not discussed at the study section meeting, and are not given priority scores. If even one member of the study section believes that the application has some chance of being funded, the application is fully discussed and scored. A pilot study has been conducted. The first phase of the pilot study, performed during the February 1994 review round, included four study sections within the DRG; participating review groups designated 30 to 50 percent of the applications as not to be discussed, and reviewers consistently indicated that there was more time for discussion of the applications that were scored, and that meetings ended earlier than usual. Expanded pilot studies were conducted in the June and October 1994 rounds of review. While triage can now be considered to be fully implemented, the process will nonetheless continue to evolve, with adjustments and modifications if necessary. Modified Summary Statement - (full implementation beginning February 1995) - The first-phase triage study incorporated a streamlined format for the expedited production of summary statements. Applicants whose projects were unscored received a critique that was essentially unabridged comments from the reviewers, with the intent that this would allow for a quicker return of the critiques and provide the applicant more time to amend and resubmit the application, if desired. Feedback from applicants involved in the first-phase study (and others) indicated that, in general, receiving essentially unedited reviewers' comments is preferable to receiving a synthesis of the critiques prepared by NIH staff. In the expanded triage study, the use of essentially unabridged reviewers' critiques was extended to all applications. For the scored applications, a "Resume and Summary of Discussion" was included to convey the highlights of the discussion at the study section meeting leading to the final rating, plus a paragraph detailing budgetary recommendations as appropriate. The modified summary statement format will be used for all applications reviewed by DRG study sections beginning with the February 1995 round of review (i.e., those submitted in October or November 1994). Expedited Release of Summary Statements to Applicants - (concept under discussion) - In current practice, summary statements prepared by Scientific Review Administrators (SRAs) are routed to Program staff at the ICs, who review and then release them to the applicants. This introduces a time lag, sometimes substantial, between preparation of the document and receipt by the applicant. Now that full implementation of triage and the use of modified summary statements are in place, it has been suggested that summary statements for unscored applications, which consist of the essentially unedited reviewers' comments, might be released to applicants directly from DRG. This would provide considerably faster feedback to investigators whose applications were unscored. Retrospective versus Prospective Review - (concept under discussion) - The usual NIH peer review process is primarily prospective in that it typically focuses on a detailed proposal of the specific research studies for which the applicant is seeking support. However, some contend that a retrospective review, focussing primarily on the investigator's recent accomplishments, would be advantageous. Researchers would be relieved of the burden of writing detailed plans that are likely to change in the course of conducting their investigations, and could instead spend the saved time on research; evaluation of past scientific record could afford the precision of hindsight and is believed by some to be the best predictor of continued creativity and accomplishment; and success could be less dependent on grantsmanship skills. On the other hand, there are concerns that investigators just beginning their careers or at career turning-points may not be well served by retrospective review. Ongoing discussion is focused on the question of under what circumstances a retrospective-based review might be appropriate. Historical data on funding patterns and information derived from examination of precedents for retrospective peer review at NIH and other Federal agencies are being considered in these discussions. Further deliberations, involving both NIH staff and extramural researchers, are clearly needed. Restructuring DRG Review Groups: - (concept under discussion) - Ideas are being considered about how the new Initial Review Group-based organization of DRG can be exploited to broaden the range of expertise represented on individual study sections without sacrificing focus. In addition, discussions are under way to identify mechanisms for periodic review of the organization and function of IRGs and their constituent study sections, to ensure that they evolve in pace with scientific advances. "Just in Time" - (pilot experiments currently under way) - "Just-in- Time" postpones the collection of a fairly substantial amount of information that currently must be provided for all competitive applications. Data on other support and complete budget detail would not be requested at the time of application, and the biographical sketch would instruct applicants to provide only information related to research background and experience, including, at the option of the applicant, the sponsored support relevant to the proposed research. This approach would simplify and reduce the administrative burden associated with the NIH grant application without compromising the initial review group determination of scientific merit or the reasonableness of the proposed budget. Detailed information relevant to the award of the project would be exchanged "just in time" prior to award. This delayed request and receipt of information should relieve administrative and paperwork burdens for the approximately 70-75 percent of applicants who will not receive an award. In order for "Just-in-Time" to be effective, it is essential that applicants and their institutions be prepared to provide the detailed information quickly, if a decision were made to fund an application. "Just-in-Time" pilot tests are being conducted through a number of RFAs and Program Announcements from several ICs. This provides an opportunity to explain the requirements to a defined set of applicants and to evaluate the results. Identification of High Risk/High Impact (HR/HI) Research Applications - (implementation in place since June 1994) - Since June/July, 1994, reviewers serving on DRG study sections have been asked to identify those applications that both involve high risk research and have the potential for high scientific impact. This identification occurs after the merit review and assignment of priority scores, and will require agreement of at least two members of the IRG. The goal is to determine whether IRGs can better identify HR/HI applications when specifically asked to, and if the information they provide will aid program staff and the National Advisory Councils to better identify special research opportunities for consideration. Amended Applications - (concept under discussion) - Currently, there is no limit to the number of times an unsuccessful application can be resubmitted in an amended version. This impacts on review burden, and can result in inefficiency, particularly in cases where the original application is either fairly weak overall or for the most part quite strong. Applicants with intrinsically weak proposals have been known to incorporate reviewers' specific suggestions into amended versions, cycle after cycle after cycle, with the expectation that eventually this will suffice to merit a fundable score. On the other hand, applications that are very strong overall, but for which reviewers have raised a few relatively minor questions or criticisms (the sort that could easily be dealt with by a one or two page response from the applicant), are frequently voted scores that put them beyond the nominal payline for funding. Those applicants must then resubmit an amended version, which is then subjected to the lengthy review process all over again. This introduces delays in the initiation of worthwhile research activities. Such delays can be deleterious to the careers of beginning investigators, and can result in serious inefficiencies of both cost and time when the delay results in a funding discontinuity for a renewal project. Discussions are focusing on a proposal to limit the maximum number of amended versions permitted to two (i.e., no more than three submissions of the same project), and on ways in which time-consuming resubmission of promising applications might be avoided (e.g., by increased use of deferral and request for additional information by study sections and/or by Program staff for presentation to National Advisory Councils). Cost Management: budgeting for total costs rather than direct costs - (concept under discussion) - The usual current practice is for applicants to request, and for reviewers and program staff to recommend, grant budgets in terms of direct costs. Indirect costs usually increase total costs by approximately 30 percent. However, indirect cost amounts may vary unpredictably over the duration of a multi-year grant, as institutional indirect cost rates change as a result of negotiations or as direct-cost spending patterns vary (e.g., if funds originally budgeted for equipment are instead used for supplies, or vice-versa). Thus, it is difficult to estimate true out-year funding commitments for budget projection purposes. An alternative approach, which is standard practice in several other Federal agencies, is to consider the requested and recommended budgets at all stages of the process in terms of total costs. In addition to accurately reflecting the total cost of the research and stabilizing the out-year budget commitments, this provides cost management incentives to the grantee institution and investigator. Modular Grants - (pilot experiments under way) - The concept of the modular grant is based on a return to the underlying principle of a grant as an assistance mechanism that does not necessarily provide full-cost reimbursement for the research project. In the modular grant model, applications would be submitted and/or awards would be made with direct (or possibly total) costs in modules of a given amount, e.g., $50,000, with work proposed within these incremental categories. Alternatively, a limited series of capped award levels (e.g., $100,000, $200,000) might be used. The modular grant is envisioned as eliminating the need for submission or review of detailed budgets. Fiscal accountability for individual projects could largely be shifted to the recipient organizations, with NIH retaining responsibility for oversight of the organizations' monitoring systems and remaining accountable for the scientific progress of the projects it supports. A feasibility test of the modular grant mechanism is being undertaken by the National Heart, Blood, and Lung Institute (NHLBI) through a Request for Applications (RFA) for R01 applications that was issued in the Fall of 1994, with applications due in January 1995. A second pilot is planned, also at NHLBI, in conjunction with an RFA for multi-project applications. This concept could be extended to address total costs. Post-Council Notification - (limited implementation in place) - Several ICDs have eliminated routine use of post-council letters. Instead, all applicants are informed of the status of their pending applications in the letters that accompany the summary statements and that they will be notified if the National Advisory Council makes any recommendation that differs from or alters the recommendation of the IRG. The use of post-council letters is thus reserved for only the small group of applicants for whom the National Advisory Council takes a separate action, such as addressing a rebuttal or recommending deferral, or to inform the applicant of funding or possible funding (NHLBI only). Full implementation of this reinvention would eliminate the mailing of more than 25,000 letters from the NIH each year. Electronic Research Administration - (in planning stage; some pilot experiments under way) - Electronic research administration initiatives are based on the concept that electronic files would be created to serve as the repository for all information generated during the life cycle of each grant. This data base would be accessible to authorized institutional and NIH staff, who could each review and add information as required. When fully implemented, this should streamline the administration of research grants and cooperative agreements. One major initiative involves the electronic submission of grant applications. A feasibility experiment in which R01 applications were submitted electronically in their entirety, using software provided by NIH (Automated Grant Application System, AGAS), was carried out for five review cycles (February 1993 through July 1994) in cooperation with seven major grantee institutions. Analysis of the results of the experiment have not yet been completed. However, the experiment was limited by the fact that internal NIH procedures for the post-receipt handling of applications are not yet electronic and therefore require the use of paper documents. These trials illuminated the diversity in the way applicant organizations use automation to create applications and to capture relevant data for use within their inhouse systems. Therefore, NIH has adjusted its plan to emphasize the publication of transmission stream specifications. This will open up opportunities for independent software vendors to develop grant application creation software, which can create data streams compatible with NIH processing systems. Another experiment (Limited Electronic Submission System, LESS) was begun in October 1994 in cooperation with two major grantee institutions, in which the face page, second (Abstract) page, certain budget items, and personal data for the Principal Investigator for R01 applications are submitted electronically, with the remainder of the application submitted as paper copies. The aim is to feed the electronically-submitted information directly into NIH's internal data base (IMPAC), without the need for intervening paper copies or manual re-keying of the data. In the near future, this process will be further simplified through an initiative to provide each institution with a unique, assigned number that would be used to link the application with administrative information already contained in IMPAC, e.g., institutional assurances and certifications, names of institutional officials. By using these institutional profiles, the need to provide data repeatedly on a project-by-project basis would be eliminated. Increased electronic availability of information about extramural research programs - (planned enhancement of current implementation) - Access to the NIH Guide and telephone directory has been provided on both the NIH Gopher and NIH Grantline; the PHS Grants Policy Statement will soon be available on the NIH Gopher; and the CRISP database, which lists all NIH grant and contract awards, is available on the NIH Gopher. The full text of the NIH Guide and the Table of Contents is available through separate LISTSERV subscription lists (NIH Guide, Vol. 23, No. 20, May 27, 1994). Efforts are underway to add several items, such as a subscription list for information from the Office for Protection from Research Risks (OPRR); rosters of advisory groups including initial review groups; the minutes of the public portions of the National Advisory Council meetings; the NIH Extramural Programs; and descriptions of ongoing programs such as the First Independent Research Support and Transition (FIRST) Awards, the National Research Service Act (NRSA) Awards, and the Minority Supplement Program. POST-AWARD MANAGEMENT OF GRANTS: Notice of Grant Awards (NGA) - (full implementation announced in NIH GUIDE, Vol. 23, No. 42, December 2, 1994) - The awarding Institutes and Centers (ICs) have recently implemented a new procedure whereby only a single copy of the NGA is mailed to the grantee institution's administrative official, with the instruction that the institution distribute copies to the Principal Investigator and other interested parties. This eliminates the mailing of copies to several offices from NIH, and will serve as a phase-in to the longer-term goal of electronic transmission of NGAs. Streamlining the Non-Competing Award Process - (implementation begun October 1994) - The goal is to create a process that is more efficient and less burdensome for all parties involved. One guiding principle is that, to the extent possible, the financial and administrative information required for the responsible award and administration of a grant should be obtained at the time of the competing award; the noncompeting applications could then focus on the scientific progress of the research. Toward this end, beginning October 1, 1994, financial reporting for Type 5 applications has been streamlined. All out-year budgets are provided to NIH program staff at the time of initial award. Therefore, instead of submitting four different financial documents each year (year "x" budget, estimate of year "x-1" expenses, Financial Status Report, and quarterly Federal Cash Transaction Reports), only the Financial Status Report and the Federal Cash Transaction Reports are required. Annual reports are still required, but these should focus on scientific progress, changes in scientific direction, changes in other support, and other matters relevant to the programmatic management of the grant. This streamlined approach had been tested for two years under the Federal Demonstration Project. Further streamlining is under consideration, notably the possibility of eliminating the annual Financial Status Report. NIH INTERNAL STREAMLINING: A number of experiments and other activities are underway to improve the efficiency and cost-effectiveness of internal NIH processes. For example, the NIH Extramural Research Reinvention Laboratory has requested redelegation of authorities related to advisory committee administration and grants administration policy and regulation, with the goal of reducing paperwork, effort, and delay related to obtaining clearances at several levels within the DHHS. Other internal reinvention activities at various stages of development include: streamlining the DRG process for nominating study section members; expedited assignment of applications in DRG for renewals and amended applications; evaluation of new staffing patterns to identify ways to accomplish more work with fewer staff; reinventing aspects of National Advisory Council activities; streamlining internal clearance and approval of cooperative agreement RFAs; streamlining internal review for non-competing awards; reinvention of contracts management procedures; and simplification of other grants management-related processes. (Suggestions may be submitted to Dr. Wendy Baldwin, Deputy Director for Extramural Research, National Institutes of Health, Building 1, Room 152, Bethesda, MD 20892, or via e-mail: DDER@NIHOD1.BITNET or DDER%NIHOD1.BITNET@cu.nih.gov) .
Return to NIH Guide Main Index
Office of Extramural Research (OER) |
National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892 |
Department of Health and Human Services (HHS) |
||||||||
Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, Audio or Video files, see Help Downloading Files. |