Announcer: From the National Institutes of Health in Bethesda, Maryland, this is All About Grants.
Megan Columbus: Welcome again to another edition of All About Grants. I am Megan Columbus from NIH’s Office of Extramural Research. Have you ever wondered who was reviewing your grant application and how they were selected? Today we have with us Dr. Cathie Cooper who has been a scientific review officer with NIH’s Center for Scientific Review for over 10 years here to provide us with some insight into how study sections are formed. Can you start us off by talking about the purpose of a peer review group?
Cathie Cooper: So the initial peer review group, their charge is to provide a fair, independent, expert review of an application’s scientific and technical merit, which we define as the likelihood that that work will exert a sustained and powerful influence on the fields involved.
Megan: And what about the role of the scientific review officer in recruiting the members?
Cathie: The SRO is a designated federal official who has the overall legal responsibility for the review process. So the SRO is ultimately responsible for insuring that each application receives a thorough and unbiased and high quality peer review, and that all applicable laws, regulations, and policies are followed while doing so.
Megan: And so when we think about putting together, or the composition of the review group, you just mentioned the laws and regulations. What are those exactly?
Cathie: So peer review panels are grounded in the Federal Advisory Committee Act, which was enacted by Congress in 1972, and it governs the composition and function of committees that provide advice to the federal government. Each agency is responsible for establishing uniform guidelines for how these committees operate, so our Office of Federal Advisory Committee Policy is responsible for interpreting the laws and establishing the guidelines for how NIH peer review panels function.
Megan: Does the law get down to the kinds of things like the composition of the panels and that kind of thing?
Cathie: Generally, there’s provisions in the law for diversity and inclusion, but the nitty gritty is determined by the both the Office of Federal Advisory Committee Policy and individual institutes’ and centers’ best practices.
Megan: So the scientific review officer has to work within a framework of these laws. Then what’s the process for how that scientific review officer goes about actually finding reviewers?
Cathie: So we use a fairly wide variety of resources to find reviewers, both regular and temporary. We use diverse resources to ensure that our panels don’t become bias or skewed towards one opinion or one school of thought. So successful applicants are often times a very good resource, mainly because their science is very fresh in our mind, and we’ve heard the committee’s assessment of the individual’s expertise and commitment to the process. We use NIH RePORTER and PubMed. We also frequently get recommendations from major scientific societies, from leaders in the fields, other NIH staff such as other SROs or program directors, as well as current and former study section members. Another source is that we attend scientific meetings, and we meet people or see people that have spoken at scientific meetings. Occasionally, we get volunteers who call us up or send us their CV, and ask us if they can review on our study section panels.
Megan: What makes for a good reviewer? You say you get recommendations. Is it their accomplishments in the field, is it how they interact?
Cathie: It’s all of those things. For temporary reviewers we’re often times trying out someone that we haven’t used personally. We ask the question, “Are they an active and productive scientist?” We can find that out by looking and seeing whether they have grants, whether they’re publishing well, and if they’re publishing in high impact journals. Are they likely to be considered an authority in the field? That’s also very important. Again that’s high impact publications and if they’re invited speakers at conferences. I like to see reviewers that have had some review experience, and I frequently will ask other scientific review officers if they have used someone and if they are a good reviewer. I always ask somebody to serve first as a temporary reviewer because I like to check out the depth and breadth of their scientific expertise, are they objective, do they have good communication skills, including can they articulate their opinions very clearly to a group during discussion, and also their ability to work collegially with other members on the panel. And then I always am looking for diversity, so I’m trying to make sure that amongst those reviewers I have women, minorities and people who are from diverse parts of the country.
Megan: You’ve mentioned a couple times that you’re looking for people who already have NIH funding. Is that a requirement or is that just something that’s desirable?
Cathie: It’s desirable. It’s certainly not a requirement for a temporary reviewer. Our regular members are generally expected to have ongoing NIH funding; however, depending on the field, they might not and that can be okay.
Megan: You talked a couple times about trying people out on temporary panels. Could you tell me the distinction between what you’re referring to as a temporary panel and what’s a chartered study section?
Cathie: A chartered study section has a core membership that’s called the roster, the membership roster. It can range anywhere from 12 to 28 regular members that have been nominated by the SRO and approved by the NIH Office of the Director. And these are people who are recognized as being leaders in the field, authorities in their area, and who have been tried out as a temporary reviewer and determined to be a good reviewer. They are supplemented on these regular panels by temporary members that we bring on at a meeting-per-meeting basis, basically to cover additional expertise or for additional people when we have more applications than we anticipated. Special emphasis panels are generally recruited fresh each time, although there are some exceptions such as the small business panels, which have a core of recurring members; however, there’s no roster and there’s no approval process. Most special emphasis panels are recruited from scratch each time you need to run one.
Megan: And that’s based on the science of the request for application or…
Cathie: So we can get a group of applications, example are the continuous submission member applications that we get. We can get a special program announcement that we need to review or a program project or any other special review responsibility, and so we have a group of applications we look at that, we identify the science, then we go out and determine what reviewers we need to bring in to cover the science and to be able to review the number of applications. Then we go out and start making phone calls using the same list of sources that I mentioned earlier.
Megan: NIH’s Center of Scientific review is responsible for reviewing about 70% of NIH applications, is that right?
Megan: To do that I believe they use over 15,000 reviewers a year, and so we’re casting a wide net. But it can never really be wide enough, can it?
Cathie: No and we really want to avoid using just a focused group of reviewers or a small group of reviewers. We want to be as inclusive as possible, and the reason for that is we want to make sure that everyone has input into the review process, not just a small group or clique of reviewers. That’s the reason that I encourage people to volunteer to serve on peer review panels. You can either do that by looking for a study section that you would enjoy serving on, calling the SRO or sending them an email, or when an SRO or somebody contacts you and asks you to serve, say yes.
Megan: It’s part of the whole research system here in the United States. So I think it’s paying back what you’re getting in kind of thing.
Cathie: Right well, I always think of it as feeding the organism that feeds you.
Megan: That’s right. You know the other thing I’d add to that is, I think it’s a really wonderful learning opportunity.
Cathie: It is.
Megan: It allows you to see not only see what other people are doing, but allows you to understand at a very different level what makes for a strong application and what comes across well when people are in that kind of setting.
Cathie: That’s true. And even though we generally use more senior and experienced reviewers on our panel because they have the depth and breadth of expertise that allows them to give a more knowledgeable assessment of the applications, we’re very careful to include junior scientists and younger investigators on the panel, as well, and part of the reason is that when I use a newer investigator, generally not a brand new investigator, but a newer investigator, they always tell me after the review meeting they’ve learned so much about how to write a grant. So I really look forward to including them on the panels. In addition, CSR is piloting a new program that we call the early career reviewer, where we will take complete novice reviewers, people who have not reviewed for NIH before, very early in their career, probably new investigators.
Megan: And you know, I think providing the fresh perspective is a good thing for the review process, as well.
Cathie: It is. Also, another point I’d like to make is that in some fields the truly knowledgeable reviewers are not the senior reviewers because the field is developing too rapidly.
Megan: One more question. Is there any requirement for people who have already received NIH funding to serve as reviewers?
Cathie: A Guide notice was published encouraging people who have received NIH funding to participate in the peer review process, and I encourage everyone to do that.
Megan: Thank you so much for joining us today. It’s been a pleasure. For NIH and OER this is Megan Columbus.
Announcer: Want to learn more about the grant review process? Check out our podcasts from August 27 to November 8, 2010. And coming up in our next episode, understanding the new appeals process.