Announcer: From the National Institutes of Health in Bethesda, Maryland, this is All About Grants.
Megan Columbus: Welcome to another edition of All About Grants. I’m Megan Columbus from the NIH Office of Extramural Research. NIH recently issued a new policy on appeals of the initial peer review, that’s the part of the review process that’s focused on evaluating the scientific and technical merit of the grant. Today we’re speaking with NIH’s Review Policy Officer, Dr. Sally Amero. Sally, it’s not unusual for an applicant who did not do well in the review process to disagree with their score. They’re putting together their best application, and they feel that it’s a strong application. Usually the score is justified and there’s no question. And usually the process happens the way it should, but there may be times when an appeal may be appropriate. When can or even should somebody appeal the review?
Sally Amero: Well that’s a really good question. The policy is written to cover those instances where something does go wrong in the peer review process. So it really is intended to address a flawed review, not a difference of scientific opinion, but truly when the process did not work.
Megan: Could you give an example of what a flawed review, something that may not have gone right?
Sally: Well actually there are four areas that the policy is intended to cover. One is evidence of bias on the part of one or more of the reviewers, note the emphasis on there of evidence of bias; conflict of interest as specified in regulations that apply to peer review; lack of appropriate expertise on the review panel; or factual errors made by one or more of the reviewers that could have affected the outcome substantially.
Megan: “That could have affected the outcome substantially” is probably the key part there.
Sally: Right. On that one that’s key.
Megan: I can imagine that the question about the expertise of the reviewers may be one that’s questioned.
Sally: That’s a frequent one that we hear a lot, and defining expertise is tricky. Sometimes we deliberately recruit reviewers who have a very broad perspective on the field at large. Other times we define expertise in terms of working in the exact same area, so that whole spectrum falls under the definition of expertise. It really is the responsibility of the applicant or the applicant organization to make the case that the expertise was not appropriate.
Megan: Could you provide maybe some examples of when it’s not appropriate to appeal?
Sally: Well, certainly differences of scientific opinion would be an example of a situation that is not appealable. So just disagreeing with the points that the reviewer makes is not an appropriate appeal. The appeal has to address a flaw in the process, not the opinion of the reviewer.
Megan: Well, and the application is being scored by many reviewers, so one opinion, unless they made an extremely cogent argument, shouldn’t be what makes or breaks your application, anyway.
Sally: Well that’s true. So, we typically assign three reviewers to read each application in detail and provide written comments, criterion scores, and so forth. But it is everyone on the panel who does not have a conflict of interest with that application who offers a final score.
Megan: So I’ve determined that I have a concern and it meets one of the four criteria that you’ve outlined for us. What should I do and who should I contact first about the fact that I’m considering an appeal?
Sally: Well, I think the way to visualize this is to think that at the end of the initial peer review process, the scientific review officer hands the baton for the application over to the program official in a funding institute or center. So after the summary statement is released, your point of contact should be your program official. That person’s name and contact information will appear in the top left hand corner on the first page of your summary statement. That’s the person you should contact.
Megan: And so should I be giving them a telephone call? Should it be sending them an official letter?
Sally: I would start out with a phone call just to talk about your concerns and see if the program official thinks that they fall into one of the appealable categories. The program official will also talk with the scientific review officer to gain a little bit more information about how the review was conducted.
Megan: And often times that program official was actually at the review and heard the review. So it’s often times the PI at this point who’s interfacing with the NIH on the issues related to their application, but technically the application comes from the institution. When an investigator wants to file an appeal, how does he or she deal with the institution?
Sally: Well, so let’s say that the investigator has had a conversation with the program official, they want to file a formal appeal letter, which has to be in writing – that could be email but it has to be in writing – and it has to show active concurrence from the authorized organization representative, the AOR. So the investigator would need to go to the AOR at their institution and get a signature or an email confirming that the AOR is aware that this is going forward.
Megan: Okay, so this concurrence would be more than just copy them on a message to NIH?
Sally: Right, there has to be active involvement of the AOR, or the AOR could submit the appeal letter on behalf of the PI.
Megan: What’s the right timing for an appeal to be filed? Is there a point in time when it’s just too late?
Sally: So let us just talk about what happens to an appeal once it comes in with the active concurrence of the AOR. At that point it has crossed the threshold into a formal appeal, and it must be made available to the funding council or the council in the funding IC where that application could be funded. So, it behooves the investigator to submit the appeal well in advance of council, but it does happen on occasion when that is not possible or the review might have happened very late in the round, the summary statement wasn’t issued much before council. So the official policy states that the investigator has up until thirty days after the relevant council has met.
Megan: And so wouldn’t it just go to the following council?
Sally: That would depend on the issues at hand and some of the funding considerations, but it could be handled electronically with that particular council.
Megan: But the real target is you really want to try and get to the council round for which you submitted.
Sally: Exactly. And so you want to get your appeal in very soon, not hastily – one wants to think about it carefully – but there is a lot of organization and preparation for a council meeting, so if one could get that in say, six weeks before council that would be ideal.
Megan: And if the council agrees with the appeal then what happens?
Sally: So the council really has two options: they could support the appeal, and in that case, the outcome would be a re-review of the same application that was reviewed the first time – no amendments, no updates, no changes whatsoever. Keep in mind, though, that the clock is ticking this whole time, so that re-review can’t really happen until the next review round, in most cases. So the outcome might be a more favorable score, but not necessarily, but it will definitely mean a delay in consideration of the application.
Megan: And if the council doesn’t agree with the appeal then..?
Sally: Then the review that is being contested stands, and that score and summary statement are permanent.
Megan: So is the council decision appealable?
Sally: No, that’s the end of the process. There are no further processes to appeal the outcome.
Megan: Is there anything else you want to add on this topic, Sally?
Sally: I encourage everyone to read the policy carefully and to think carefully about whether they want to appeal their review.
Megan: Thank you so much for joining us today. For NIH and OER, this is Megan Columbus.