Research Misconduct and Detrimental Research Practices

Recorded: October 14, 2022

Elyse Sullivan: Thank you for joining this virtual event focused on research misconduct and detrimental research practices. My name is Elyse Sullivan, and I am moderating today's event. I'm very pleased to introduce our experts for today. We have Dr. Patricia Valdez, who's the NIH Chief Extramural Research Integrity Officer, and we have Dr. Ranjini Ambalavanar, who's a scientist investigator within HHS' Office of Research Integrity. Our speakers will be sharing valuable policies and processes and guidances related to research misconduct followed by some good time for Q and A. We'll then take about a 5-minute break, and then we'll move on to the case-study portion of the event. We're going to be presenting some interesting scenarios and case studies to sort of test your knowledge and allow for some discussion. So with that, Dr. Valdez, I will turn this over to you.

Dr. Patricia Valdez: Great. Thank you, Elyse. So I want to start out by just reminding everyone .. Whoops, go back to the slide .. that the integrity and stewardship and protection of research and research subjects is really everyone's responsibility, so we all have a responsibility to ensure that research is done with integrity. That includes the NIH, the HHS Office of Research Integrity, universities and research institutions as well as journals. So I wanted to also let you know who our partners are. So again, as Elyse said, I'm in the NIH Office of Research Integrity. Now we work with .. . So in blue, these are all the offices at the NIH, so we work with NIH OPERA, who's our grants compliance office, NIH OLAW for animal research and then NIH OMA for any type of grant-fraud issues. And then we also have our HHS partners, so that includes the Office of Research Integrity, who's with us today, Ranjini, the Office of Civil Rights in HHS and OHRP for human research protections and OIG, Office of the Inspector General. And in addition, we also, of course, work with recipient institutions, and again, we consider this a partnership, so we're both in this together. So want to briefly go over with you the overview of our allegation review process at the NIH. Now my team handles many different types of allegations. Today we're going to be focusing on research misconduct, but I also wanted to let you know that our office handles other types of allegations that could come in. Those include harassment or discrimination, sexual harassment, bullying, et cetera, also grant fraud. These are .. . This type of allegation will be referred to a different office. We also handle foreign interference and peer review integrity. Now so these allegations come into our system. We have a confidential system that we use to collect the allegations. Our team will do an assessment. Many times these are preliminary assessments if we're going to refer it out, and then we'll take certain different types of actions depending on what the outcome of that assessment was, so one thing that we might do is contact the institution to ask more questions. We might remove an individual from surveying and peer review, and we might refer the allegation to an agency or office with oversight responsibility. For instance, if we're assessing a research-misconduct allegation and it looks like it's specific, credible, involves NIH funding, it's research misconduct, then we'll make a referral over to Ranjini's office so in the Office of Research Integrity. And then we can also take administrative actions. Again, we very much want to make sure that the grant and the research is conducting .. . is being conducted correctly and with integrity, and so we may .. . and I'll talk about this more. We may end up taking some administrative actions if there are issues regarding compliance involving grants. And then in worst-case scenarios, there are regulatory actions that we can take, and those are mainly against institutions, so that doesn't happen very often. Okay, and so I'm going to turn it over to my colleague at ORI. Ranjini, take it over.

Dr. Ranjini Ambalavanar: Thank you, Patricia. So I am an investigator at ORI like Patricia mentioned, and I'm one of the seven different scientist investigators in our office handling research-misconduct allegations and doing oversight review on investigations that carried out in our institutions who receive Public Health Service funding. So what is research misconduct? According to our regulations, PHS Policy on Research Misconduct, as shown in my right side of the screen, it's defined as fabrication, falsification or plagiarism in proposing, performing or reviewing research or in reporting research results. So what is fabrication? Is making up data or results and recording or reporting them. So any research record is included. Whether it's reported or not, falsification or fabrication in research record is part of the problem. Okay. Falsification is manipulating research materials. Sorry about this. It's going slow. so let me talk about falsification as manipulating research materials, achievement or processes or changing or omitting data or results such that the research is not accurately represented in the research record. Plagiarism, as you all know, is the appropriation of another person's ideas, processes, results or words without giving appropriate credit. So is research integrity generally high? The answer is yes. So many .. . Most scientists do research with a high degree of integrity, and that's how they are contributing to advancements in science. Say for .. In a example lab one, here's a lot of research that happens, and this white area indicates it's a high degree of integrity happens, but there are some labs that'll have also some questionable research practices, and maybe there is some actual data falsification, fabrication that may fall into research misconduct. If this is kept some practices were not watched and some integrity issues are not addressed, you can go into the lab two situation where the shaded areas grow, and you end up being a lab with a lot of research misconduct, data manipulations that happen. So it's a slippery slope. If you don't pay attention, you can actually .. A lab that was really functioning with high integrity can go into this research misconduct area and be questioned, and so avoid little, small lapses in judgment that could lead you into questioned research practices and even to research misconduct allegations, so taking shortcuts and not conducting appropriate controls and composite images with reused images from a different experiment or not paying attention to outliers, just dropping them without any justification and things like that, if nobody's questioned, you are encouraged to do more and more and watch that this can actually lead to research misconduct questions to your lab. Research misconduct findings, also for a finding to be made, Public Health Service definition under definition of PHS Regulation, the allegation should be proven by a preponderance of the evidence, that fabrication, falsification and plagiarism should be proven, and the misconduct should be proven that the respondent acted intentionally, knowingly or recklessly, so that's the intent that should be proven, and to prove intent, it should be an individual who intended to do, so we have to prove who did it, so when it's intentional, honest error or differences in opinion are not misconduct actually, and we also have to prove the actual significant departure from accepted research practices in the relevant research community. So because of this, we get a lot of queries to our office or allegations, and a lot of them, as you see in this graph, these green bars, these are the different years that we received so many allegations that are in the green bars. Even .. . Whether they are misconduct allegations or not, we get a lot of queries, but the red bars indicate the number of actual misconduct cases that went through investigation, so you can imagine the research integrity is much more than the outright misconduct, and some of the questions that come to office, Patricia touched upon some of the issues like harassment and so on, even financial frauds and things. Those queries also sometimes come to our office, and there are other ones like authorship disputes, duplicate publications, misuse of animals and human and financial mismanagement. All these come, and we either forward it to .. We usually forward it to the appropriate agency for handling those issues. So some respondents have told us, or during the investigations and so on, they said, "It's .. . Grant applications only need preliminary results, so it's okay to not have a full .. . actual full data. You can reuse. You can do some .. . It's an example," so here as you can see, there's .. . But falsification, fabrication or plagiarism in grant applications can be misconduct, so be careful in what you put in a grant application. For example, use valid data. Don't use data that you are not sure about, and you manipulate, and verify data that someone gives you for accuracy by taking the raw data, and don't plagiarize. If you're using some text from somebody, cite it, and that way it's not plagiarism. You give credit to the author, and avoid placeholder images. We have heard in different cases that it seems like a practice to use placeholder images and forget to replace them. Not sure if that's the real practice in the lab or whether that's an excuse that came after being questioned, but, however, if you have a practice of using placeholder images, that's not a good practice. That can actually put you in trouble later on. So here's a question for you, so say you submit a grant, NIH grant application, not aware that the data and/or text that's included by others were falsified or plagiarized. Are you liable for research misconduct? Please type in your answer. I guess you cannot go to the chat. So anyway, in the meantime, think about the answer yourself, and we'll come to it. So while you are thinking, there were at least two recent cases in 2017 and 2018, ORI cases that handled by administrative law judge established that the PI and/or corresponding author can be liable for research misconduct even if he or she was completely unaware of the falsification or plagiarism, so these are just two examples of the cases that happened, but it may or may not be the same for some other cases because these two cases had other additional FSP in them. So, however, the answer to your question is, why would you risk being even questioned? So the answer to the question is, yes, you can be liable for research misconduct. Okay. I'll quickly go through the steps in research-misconduct proceedings, so somebody makes an allegation to .. . And those are colleagues, publication readers on the online or paper readers and peer reviewers, coauthors and journal editors. Who do they make allegations to is the to the institutional officials, funding agency, ORI or journals or anyone, so whoever receives the allegation conduct an assessment to see if the allegation is actually credible and specific, and if it comes to us, ORI investigators will do the initial assessment and forward to the institution, so forward it, and institutional research integrity officer will organize the remaining process that involves inquiry, and if the inquiry determines further look is +important, then investigation follows. These two have time limits in our regulation, 60 days and 120 days. If this process don't end in this within this time, the institution will write to ORI and request an official extension of those deadlines, and generally we understand, and extension is given. So at the end of the investigation, the institution submit the report to ORI, and the ORI reviews the report and evidence that was used to make a finding of misconduct or not, and then we agree with the institution and either represent a separate finding of a public service finding, or we sometimes do not make a separate finding but agree with the institution and decline to pursue a separate finding. And when we make a finding, we recommend administrative action that includes fixing research record, special certifications, suspend or terminate funding, supervise offenders, and some respondents are debarred from receiving PHS funds for a certain amount of time or number of years. And all ORI findings are published with the name of the individual and the science and the .. what literature was effected, and we also publish an NIH/ORI website as well. So what is our role in prevent research misconduct? Every one of us have a role in it, and the science starts with the researchers funded by funding agencies, and the institutions have to foster integrity in their institutions, and journal, peer reviewers, government regulatory agencies and whistleblowers, we all have a role in it. We .. You know what our roles are, but because the research data is collected by researchers, let's talk a little more about what happens. So the scientists are under big pressure, constant pressure, different kind of pressures, also pressure, so during that pressure, they kind of justify to do some data manipulation, and they rationalize, yeah, it's okay to do it. And so this pressure and rationalization, what is causing them to act actually is the opportunity to do so, and example is nobody's looking at me. Nobody's supervising. I can just do the summary data and provide to whoever is my supervisor, and if the opportunity calls, they do act on falsifying data or manipulating data. So the next slide shows some of the examples where the statements were made by respondents in the interviews, actual interviews. So here he says, "I felt it was necessary to get a paper in a high-profile journey in order to get a faculty position." That's the pressure. And there are more that indicating pressure and lack of training, lack of mentorship, so here's, "I had been applying for a green card and felt pressured to make a good paper." That's a different kind of pressure. "Half of me wanted to make my PI proud, and the other half was terrified of failing, so I fabricated data." And the next one is poor supervision. "I was scared to go to my PI who .. . He used to scream and yell at me when things did not work as planned." And inadequate training, so these are some examples that if we know in the lab as a PI we could avoid that trouble that comes later by pushing the lab members to do different things. So here's an example I'm going to show, a figure that appeared in "Nature Medicine," Figure 3c, but the right side is the figure in NIH grant application. They are supposed to .. They are represented as different experimental result. However, there are problems in this. So the figure in left, the cells in the left figure have been reused and rearranged on the right to represent a different experiment. Three cells have been reorganized in place and labeled as separate ones. So in the second column on the right-side figure or third column actually, third row, these cells in the red square are exactly the same. The respondent copied and pasted to represent two different time after the LPS treatment. You cannot tell that they are same because the respondent has intentionally manipulated the copied cell to hide the similarity. Not only that, these other two cells on the ends left and the rightmost and the leftmost are also the same. So I'm going to show you in the next slide. You cannot believe these are the same at this view. However, when you do some intensity enhancement and size adjustment, you can actually see the similarities in this. There's no question, and this is something we had to do and demonstrate that they are the same. Here is another example. This is tissue sections, and this is pancreatic tumor. Published figure is in a journal, figure, but it came through pieces from different grant figures, and you can see here, and this is a trimmed portion and enlarged and another trimmed portion from a different grant, and it's called hepatic tumor. Here is a pancreatic tumor. So there's a lot of misrepresentation of data in addition to reusing from different places from a different experiment. And I'll show you one example quickly on a clinical study. This is a follow-up study after breast-cancer treatment of a patient called Patient 10, so I am .. . I ask you to focus on the dates and the comments on the follow-up visit forms, so here the patient visit was on January '88 and November '88. The patient is fine in these dates. Okay. So the next follow-up was .. . I'm sorry this is a little slow .. on '89 March and February 1990, so the 1990 is the last visit. However, the patient chart or patient actual clinical chart had a death certificate that was dated in 1987, which is 28 months prior to the reported follow-up, last reported follow-up and the first 4 months after the .. . prior to the first follow-up, so this is a clear falsification of follow-up visits. Patient did not even exist during those times. So usually clinical studies of several different documents can be falsified as listed on the right here, entry criteria, follow-up visits, consent forms and so on, and why this happen is because these individuals who collect this data are not well supervised or trained to do the job. They have excessive workload, and there is incentives for recruiting patients and so on. So .. And why this happens in clinical studies is, some of these .. As you can see, these are respondents in clinical studies, 68 cases. These are the professors, but compared to the number of professors and respondents, many others who collect actual data or interviews and anything are all research assistants and students and associates and so on. Some of them do not even have any interest in the outcome of the clinical study. They are focused on incentives and so on, and a big part of it is excessive workload and lack of supervision, so that's something that we can avoid having. And some .. Also how you can improve is to have policies and procedures to have data maintenance and record keeping and knowing that institution has a big responsibility because the institution is receiving the grant funds, and you have a bigger responsibility then overall responsibility. And with that, so what we can do as a senior official, set the tone for the institution and make integrity a priority, and as a administrator, keep up the policies and procedures, and as a PI, establish specific standards for the staff in recording, reporting and publishing data, and as a staff scientist in the lab or postdoc or grad student, commit to integrity and practice daily. And thank you and handing over to Patricia.

Dr. Patricia Valdez: Thanks so much, Ranjini. So I'm going to continue. So as you heard, the process and the proceedings for research misconducts, inquiries, investigations can take some time, and in addition, when an institution makes a finding, then they must report that to ORI. ORI has oversight ability at that point, and so they may then at that point be considering whether or not to pursue federal funding. Now in the interim because these processes do take some time, there are interim actions that we can take at the NIH. Of course, we want to protect the public. We want to protect any research participants, the research itself, the research process and public funds. So some of these interim actions include but are not limited to specific award conditions on grants, so these would be included in the notice of award. It may require additional supervision on an individual. It may require certification of data. For instance, if a report indicates that the person is not keeping the raw data or there's severe data-management issues, we could impose a special award condition that they must get the data certified by the NIH. In some cases, we could request a change of PI. We could also restrict funds or suspend or terminate awards and working with the institution to do that. And also, in many different types of integrity cases, we may refer the case to the HHS Office of the Inspector General, or OIG. Okay, so oftentimes institutions are wondering, "When do we contact NIH?" because we understand that the research-misconduct proceedings are confidential. You want to make sure that somebody ..no information leaks. This isn't still .. Maybe a lot of times you're still at the allegation stage, and we don't know necessarily what the outcome is going to be. But we do ask institutions notify us when there are developments that have a significant impact on the award-supported activities. For instance, if there are problems, delays or adverse conditions which materially impair the ability to meet the objective of the award, that could be things like the PI is on administrative leave, or there's .. You've identified really big problems, large problems with the data, and so the research can't continue and needs changes in scope. So that notification should include a statement of the action taken or contemplated and any assistance that is needed to resolve the situation. And now I want to mention this is on the NIH Grants Policy Statement, which I have a link to below. So also when to contact NIH, another time is when there is .. when you identify potential fraud, waste and abuse of NIH grants, so we ask that institutions and anyone really report false statements related to research misconduct to NIH or report that directly to HHS OIG. If there were administrative .. So if there was a case where grant funds were misspent, we could potentially administratively recover those funds. We could also take administrative, civil or criminal action, working with our partners in law enforcement under a variety of statutes relating to fraud and making false statements or claims. So I want to talk a little bit more about false claims because this can be related to research misconduct. So as Ranjini mentioned, research misconduct findings are made against individuals, so you can't have a finding unless you know who did it, you identify the person who did it. Now, false claims can be made against the individual or the university itself. And so these are the regulations here, the U.S. codes. So you can have civil false claims, so that's 31 U.S.C. 3729-3733 This is the False Claims Act, which ..So this refers to someone knowingly presenting or causing to be presented a false or fraudulent claim for payment or approval, for instance, grant funds. This person would have knowingly made use or cause it to be made used, a false record or statement material to a false or fraudulent claim. Now false claims can also be criminal. In some rare cases, they are pursued as criminal. Usually they're pursued as civil, and then I'd also like to mention here false statements as well. I'm not going to go into that much here, but there's also a criminal code 18 U.S.C. 1001, which is regarding false statements. So how does this apply to NIH applications? So false records or statements included in grant applications could be considered false claims or false statements. So some of these examples might include falsified or fabricated data, so there could be a research misconduct finding regarding this falsified or fabricated data. But then also at the same time that falsified or fabricated data could be a false claims, and so OIG may get involved in that case. Also, failure to disclose other support and/or grant overlap. For instance, if the just in time documents before a grant is made, we do ask the institution and the individuals to make sure that they list all of the other pending and active support that they have, and the reason that we ask that is so that we can make sure that we're not really duplicating funding. So we want to make sure there's not overlap of .. scientific overlap, budgetary overlap or commitment overlap. So if that's not disclosed, that could be a false statements or false claims. And also misrepresenting the level of effort of key personnel, so you want to make sure that the people who are working on the grants are actually doing the work, okay? And then for these cases, there also must be a determination of materiality. So it has to show that it did affect. For instance, it did affect the decision of the funding agency to make the award. So I wanted to show this one example from last year of a false claims settlement. So this involved a former scientist who used to work at Massachusetts General Hospital, so they identified .. They conducted the research misconduct investigation, identified problematic data, and then they came to us, and they did .. They paid back the funds for the grant. Now, in addition, the Department of Justice decided to pursue potential false claims case against the PI himself, so this is a Dr. Lee, and so this was the settlement. So this is a statement from this settlement, which was, again, released .. this statement was released by the Department of Justice. It says, "The NIH grant application process relies on scientific integrity, accuracy and honesty from individual principal investigators, but Dr. Lee supplied falsified results, inauthentic data and false statements instead." So why does this matter? So that same case, the announcement from DOJ went on to say ..so this said acting U.S. Attorney Nathaniel R. Mendell, "Defrauding the NIH wastes taxpayer money, limits the availability of funding for other research and undermines the central purpose of scientific inquiry. We commend MGH, Massachusetts General Hospital, for disclosing the alleged false statements, for repaying funds and for taking meaningful steps to prevent future recurrences." So something to keep in mind as .. And I'm sure all of you know that getting a grant is not easy. It's very competitive, and so when someone comes in and submits false information to the NIH, it causes us to make the inappropriate funding decisions, which really is terrible because then there's a lot of other really good research that's being left on the table because we just don't have the funds to support them, so we want to make sure that we're only funding the research that's conducted with integrity. So I'll go ahead and stop there. It's time for Q&A.

Elyse Sullivan: Great. Thank you so much to both of you. Let's turn over .. . So we've been collecting questions using the Q&A box, and we've got folks behind the scenes who are triaging these, so we will get going. Let's see. Okay. First question. So this is a question probably for Ranjini. If an institution has an allegation of misconduct and the institution has never handled an allegation of research misconduct before, can they contact ORI for assistance, and if so, when in the process?

Dr. Ranjini Ambalavanar: Yes, you can contact at any time, and the earlier the better if you need help or if you have questions, call us. We have investigators on call every day. If you need the phone number, I have a slide at the end, so you can write to us through an RIO form, and we have a website you can use.

Elyse Sullivan: Wonderful. Thanks, Ranjini. Let's see. Another question for you about the trends in the number of cases and queries. This individual said it has to be encouraging that the number of queries and cases has been going down since 2015. Are there any positive trends related to these lower numbers?

Dr. Ranjini Ambalavanar: Not really. It's not going down. Actually, it went up in 2012 and around that time because there's one anonymous complainant sent us a lot of allegations and then it went down because whatever reasons, but we still get anonymous allegations. There's nothing wrong with that, but at that time, it just spiked up, and we have a lot of allegations that were not Public Health Service funded, not relevant to our office, and they were also very old research work. So some of it actually ended being cases, but that was why you see that trend going up and down, but it's really almost staying at the same level apart from that spike.

Elyse Sullivan: And then, Patricia, can you .. do you have any data to speak to, any trends sort of in the NIH .. allegations that NIH has received?

Dr. Patricia Valdez: I will say that allegations have increased a little bit, and as I mentioned before, we also handle other types of non-research misconduct allegations, and so we definitely are seeing an increase, for instance, in harassment allegations. And some of these actually have a bit of both, so we have the harassment part as well as the research misconduct part where you see there might be a PI who is bullying or harassing individuals and then that could lead to potential research misconduct, and so it's been a, yeah, slightly increasing, I'd say, for research misconduct, but definitely increasing for harassment.

Dr. Ranjini Ambalavanar: Yeah, I could say it's increasing instead of going down. The last 2022 data is up to September, so we have 3 more months of calls.

Elyse Sullivan: Thank you. Let's see. I think this question is going back to the example that you presented, Ranjini, where you asked if there was an NIH grant submitted. It contained falsified or plagiarized information, who's on the line? So we said the answer was the PI or author. You're ultimately on the line even if they were not aware. The question is, can an administrator at an organization be held liable, or does it stop at the PI level?

Dr. Ranjini Ambalavanar: So the respondent, the named respondent is PI because he .. he or she is the one who put the data .. submitted the grant for his or her research, but the institution is kind of .. in a indirect way will be affected, but the administrator, no, not directly affected. I can Yeah.

Dr. Patricia Valdez: Yeah, that's an interesting question because I get this question sometimes in respect to false statements because, again, the research misconduct, they have to determine who actually committed the research misconduct, who was sitting there and manipulating the western blot or whatever it was. And then if that's included in a grant application and it's submitted to the NIH and then the administrator who is the authorized organizational representative checks the box that I certify that everything is correct and true in this application and this can subject me to USC 1001, so .. for false statements, that's a concern, definitely, but I'll just come back to say that I've never seen an AOR be directly held responsible for false statements. Usually it's the institution as a whole.

Elyse Sullivan: Thank you. Thanks, Patricia. Thanks, Ranjini. Let's see. Moving on. So when ORI does make a finding of misconduct, how are the different administrative actions determined? Are there certain factors that kind of weigh .. What are the differences consequences that are imposed?

Dr. Ranjini Ambalavanar: Yes. We look at different factors. The .. . Mainly the one is scope of misconduct, so say a person, a junior investigator in one paper, one figure or a few figures or a few papers was of a long-term, like 10 years, of misconduct and things like that, that's taken into consideration, and the next one is aggravating factors, whether they cooperate or not, that's given some credibility, so there are these factors that come in to play in making that determination about action.

Dr. Patricia Valdez: What about human subjects? It seems like if there's human subjects that may have potentially been harmed and those types of administrative actions are usually much harsher. Is that the case?

Dr. Ranjini Ambalavanar: Yes and no, but for example, the recent case on a technician .. That's not human research, actually, so lifetime of involvement, but technicians who are not supervised and trained, they can actually ruin a lot of a PI's research, so that's something I would watch, but that's not human research. I don't know of having that, but .. No. It comes to you, I guess.

Dr. Patricia Valdez: Yeah.

Dr. Ranjini Ambalavanar: It's more other type of consequences for human research, though.

Dr. Patricia Valdez: I'm thinking like Eric Pullman and his older thesis from way back .. .

Dr. Ranjini Ambalavanar: Misconduct, we mainly go by scope and .. .

Dr. Patricia Valdez: Mm-hmm. Mm-hmm.

Dr. Ranjini Ambalavanar: .. . aggravating factor, human research can be classified into that, yes.

Elyse Sullivan: And then back to sort of the allegation process, is there a time limit that you have to file an allegation within for it to be pursued?

Dr. Ranjini Ambalavanar: Yeah, there is a 6-year jurisdictional limitation. So if you remember earlier, I mentioned about the allegations that came in .. . a lot of allegations came in 2012, '13 time. They were allegations from very old papers, so if you make an allegation on a paper or a publication grant that was submitted 6 years before that date of allegation, that's outside jursidictional limits. However, there is a subsequent use exception. If you have used that research in promoting your future and current research and citing it to get grants and things like that, then that has an exception. So yes, the time limit is 6 years to answer to your question, yeah.

Elyse Sullivan: Thanks, Ranjini.

Dr. Ranjini Ambalavanar: Mm-hmm.

Elyse Sullivan: This next question is an interesting one. So the scenario here is, as scientific teams grow, PIs and coauthors are often incorporating data from a large research team that have different areas of expertise, team science. For example, in this one scenario, somebody may be collecting and analyzing MRI while others are electrophysiologists. If you work in such a different scope of research than some others on your team, reviewing the raw data for its validity can be easier said than done. How do we really address .. Are there any tips for addressing that responsibility of a PI in a team science type of a way when you may not actually have the expertise to understand if their data is truly accurate?

Dr. Ranjini Ambalavanar: Who are you asking that from? I'm going to pass it to ..

Dr. Patricia Valdez: That's the question there. But I mean, I can say .. Definitely collaborate with people that you trust. Make sure that there is transparency in how they're doing the work. I think as much as you can, learn about what they're doing would be very helpful, but, again, I think if the fields are very, very vastly different, again, I would make sure that you're collaborating with someone that's trust .. and someone who's very transparent in what they're doing. I don't know. Ranjini, do you have any other thoughts on that?

Dr. Ranjini Ambalavanar: Yeah, and the examples I told you about, and it was extra cautionary statement and just because in those cases administrative law that determined that, it doesn't mean that it's going to be in every case if it's just .. I don't know. We cannot predict, but this has happened. Also, I mentioned in those two cases examples that I gave, there were other issues, more issues in those cases, so it was no excuse to say that, "Okay, it was something that I didn't know" kind of thing because there was falsification in other data sets.

Elyse Sullivan: Thank you. So, Ranjini, again to you, can a retraction of publications be a consequence of an ORI investigation? Is that one of the actions that can be imposed?

Dr. Ranjini Ambalavanar: As an action?

Elyse Sullivan: A retraction, mm-hmm.

Dr. Ranjini Ambalavanar: Retraction as an administrative action?

Elyse Sullivan: Mm-hmm.

Dr. Ranjini Ambalavanar: We recommend retraction, and if we are settling a case with a respondent, we can put it as a requirement in our .. that the respondent signs that I will retract the paper, but we recommend usually, yes. Yeah, I think ...

Dr. Patricia Valdez: Retraction ..

Dr. Ranjini Ambalavanar: Correcting the literature is one of the .. So we are here to protect PHS funding and correct the literature and make sure the science is done with integrity.

Dr. Patricia Valdez: And I'm sure if anyone out there reads retraction watch, these other types of places, one issue is that there can be findings, but then it takes a very long time to retract the paper. And one thing that we have done in the past is to go to the institution and remind them that they did .. there was a finding involving research at your institution and, "Hey, we're still funding you, and why haven't retracted the paper?" So we can put a little bit of pressure on institutions when needed. But, again, if anyone out there is aware of such a situation, feel free to let us know.

Dr. Ranjini Ambalavanar: So retraction and correction happens throughout the process depending on the case. Sometimes the authors themselves submit corrections, or some others retract voluntarily. Some institutions take action depending on developed policies and those details, and journals contact the authors and request something. So.. And at the end of the procedure, we recommend retraction. So it happens all throughout the proceeding depending on the case and the institution.

Elyse Sullivan: Thank you. Okay. This is a question of Ranjini. Has ORI considered expanding the current definition of research misconduct to include things like bad or unethical library practices that lead to the production of unreliable or unproducable data? And I know there's sort of a request for information out on the street, right, about the current regulation. So do you want to speak to what they're considering?

Dr. Ranjini Ambalavanar: I cannot talk for that. We do get bothered by these other .. We don't condone any of those other research practices. We are aware of all that, and there will be some time that those will probably be included, but I cannot talk for that.

Elyse Sullivan: Okay. So sort of a stay tuned.

Dr. Ranjini Ambalavanar: Yeah, yeah, yeah.

Dr. Patricia Valdez: Yeah, but they ..

Dr. Patricia Valdez: Go to ORI's website and comment on their RFI.

Elyse Sullivan: Yeah, maybe ..

Dr. Ranjini Ambalavanar: Yes.

Elyse Sullivan: Yes.

Dr. Ranjini Ambalavanar: With the recent RFI and, yes.

Elyse Sullivan: Let's see. Who is responsible .. I think this would be for Patricia. Who is responsible for false statements related to effort reporting or other support? And AOR certifies, but are they on the line for that, or is it a PI?

Dr. Patricia Valdez: Right. So that's an interesting question, too, because ..so most of the time, these situations are .. I guess these types of cases are made against the institution. So as I mentioned before, I very rarely see an individual AOR being held responsible for a false statement in a grant application that they certified, so usually it's the institution as a whole. Now, there are some cases where there may be decision to pursue a false statement made against the individual as well like the PI or the researcher who put that information in the grant, but I think most of the time I see it, it's mainly with the institution as a whole.

Elyse Sullivan: Thank you. This question again is for Patricia. Can you elaborate on the demonstration of the materiality statement regarding did it actually affect outcome of the funding?

Dr. Patricia Valdez: Exactly. So what we do sometimes, so if there is a finding of research misconduct even if .. And it's in a grant .. Maybe it's in a grant application, and it's maybe one part of a figure, not the whole figure. So what we do in our office is we'll go to a subject-matter expert at the institute ..it usually extends to that under the grant .. . and ask them the question, "Do you consider this important?" And so .. "Is this piece of data important for the decision to fund the grant?" "If the data was not there, would you still have funded the grant?" in other words. And so sometimes they'll come back and say, "Oh, that's just a very small piece that didn't really play a part in our decision to fund the grant," so we can go, "There's no materiality." We usually say, "Okay, fine." But if it is material and they said, "Okay, this is a very important piece of the research and the story that they're trying to tell," then that is a problem, and so we will then likely go through and request a return of funds. And, again, and we're considering a false statement, that's another thing that they do consider as well when pursuing those types of cases is the materiality. They have to really show that that piece of data was very important for the decision to fund.

Elyse Sullivan: Oh, and then a related question. We're a couple minutes away from our break. So we have time for maybe one or two more questions. So if NIH is seeking the repayment of funds based on misconduct, how do they quantify how much was wasted and should be repaid, a certain subset of the work, what was impacted? Yeah, what's sort of the calculation there?

Dr. Patricia Valdez: Yeah, that's a really good question, and there's a lot of people involved in determining this, but some of the things that we do consider I'll just mention that is just looking at the output of the grant. If the grant produced 100 papers and one paper had to be retracted, then you can kind of say, "Okay, that .. Maybe not a lot of money needs to be returned because there was a lot of good stuff that came out of that grant." However, if the grant produced say five papers and four of them needed to be retracted, that will tell us that, yeah, this is not ..there wasn't much that was usable coming out this grant, and therefore, we would request the return of funds. So that's kind of one example. And, again, the materiality will come into play here as well, but that's kind of ..gives you a general sketch about how we do that, but, again, a lot of people are involved, a lot of discussions. We work with the funding institute as well as offer our grants compliance office to make those calculations.

Elyse Sullivan: Wonderful. Thanks. Thanks so much, and we are just about out of time for our Q&A session, so what we're going to do is we're going to go to a 5-minute break so everyone can refresh themselves. When we come back, we have an interactive case study discussion. So this is where we're going to turn on the chat. We're going to ask folks to raise hands. We're going to kind of talk about some interesting scenarios. So in 5 minutes, we'll see you back here. Thank you for joining us here again.

So like I said, we're going to get into some case studies, and we really want this to be interactive and for you to participate in this conversation. So for those of us who are here live, we've turned on the chat now, so the chat at the bottom of your screen is live. So if we're asking for some polling or some feedback, go ahead and use that chat. If you are interested in answering a question verbally, you can use the raise hand function, and we can call on some select folks. I do want to issue a reminder that if you have a specific case or specific allegation, please reach out to NIH or ORI through our specific channels, and this venue is not the time to be kind of disclosing any of those details, so just want to make sure that everyone is going to .. understands that we're here to be respectful and if there are any allegations, we do have many channels for you to let us know those. All right. So, Ranjini, I believe you have our first case study today. Do you want to take it away?

Dr. Ranjini Ambalavanar: Yeah. Thank you. Okay. So this is a scenario. You as the principal investigator, you are conducting a Phase II trial, clinical trial, on a new cancer drug that your university has patented. One of the studies in your trial ..study subjects dies in your trial, but you don't believe that the death was related to your trial. What do you do? So can you type up A, B or C .. C is coming up ..in the chat box. And then we'll discuss.

Elyse Sullivan: All right. Let's see.

Dr. Ranjini Ambalavanar: C is a not ..

Elyse Sullivan: I see a lot of C coming in.

Dr. Ranjini Ambalavanar: Where is the C? I don't see it.

Elyse Sullivan: Sorry, in the chat, if you open up your chat window.

Dr. Ranjini Ambalavanar: I know. I know. But slide ..

Elyse Sullivan: Oh, I'm sorry. You might need to kind of go back. It showed for a minute .. There it is. We have some A, some B, a lot of C ..

Dr. Ranjini Ambalavanar: I'm sorry ..

Elyse Sullivan: and a lot of good interaction here from the chat. Thank you all. All right. Somebody initially said one answer and then changed their mind to C. We've got some Bs. All right. Ranjini, you want to talk us through what the correct here is?

Dr. Ranjini Ambalavanar: Yeah. So most of you answered correctly. So you are reporting to IRB immediately. Okay? It's a serious adverse event. Whether you are sure or not about the cause of death, a death should be reported right away to the IRB. Anybody else have any other answers? So some said not .. A responses are there, but that's not recommended. Note the death in your notes, of course, but continue your research as planned. I would inform the IRB, and the second .. B is include this as a serious adverse event in your next scheduled communication to the IRB, no. Without waiting, C is the right answer. It has to be reported right away. Okay. Say you are the Institutional Official for human research protection, and the RIO was somehow informed about this, and he reports to you .. . He or she reports to you. You regularly review IRB minutes, and on this day, you noticed the report of a death in a Phase II trial in the same case. So IRB has determined that the death was the result of the trial and that it should be put on hold. Also, IRB minutes .. In the IRB minutes, the consent form in the deceased patient's file was not signed. The minutes state the IRB will investigate the matter further. Waiting for answers. What questions should you ask as the Institutional Official? A, B or C.

Elyse Sullivan: All right. So folks want put that in the chat at this point.

Dr. Ranjini Ambalavanar: In the chat box, please.

Elyse Sullivan: So the chat is .. Yep. Don't use the Q&A box. Use the chat this time. Let's see. Okay. I see some all of the above.

Dr. Ranjini Ambalavanar: Yes.

Elyse Sullivan: Let's see. C. Okay, we got a smattering. We've got As, Bs, Cs. We've got a lot of all of the above.

Dr. Ranjini Ambalavanar: Very good. All of the above is .. Hmm. Yep. Correct. I'd say it's all of the above. So how much is left in the project? Not to make any decision there. It has to be paused. Who was the funding agency? If .. Whether you have reached out to funding agencies, whether you have reached out to ORI, any adverse event should be reported to ORI as well as the funding agency, and in this case it would be likely NIH, and I'm including the regulation that talks about reporting to ORI in circumstances like that. So there are some questions on the chat box. Elyse, can you help with the question?

Elyse Sullivan: Yep, so I think some people are commenting some .. There's a question about what are the .. if we could quote the policy or the regulation surrounding this, so we can probably do that later at the end. Yep, you're good.

Dr. Ranjini Ambalavanar: Okay. Okay. Let's move on. Okay. The next day, the IRB chair calls you, the official, to report that there was also a discrepancy between the information in the deceased patient's clinical file and the information listed on the research intake form for that patient. The research intake form was signed by one of the research nurses for the drug trial. Who should you report this to at this time? Again, it's A, B, C, OHRP and PI.

Elyse Sullivan: All right. We're seeing some Bs, A, D, all. Yeah, we're kind of all over the board here. We've got a lot of ..a little bit of everything.

Dr. Ranjini Ambalavanar: All answers are correct actually, but one or the other is not correct. PI's department chair. Yeah, so more responses coming in. So RIO should be informed, one thing. And like we discussed in the previous slide, ORI should be informed and funding agency, OHRP. Department chair, yes, that's important. The chair should know, but it's not that .. You're not only the correct answer. All is correct, yes.

Dr. Patricia Valdez: So what about ORI? Is

Dr. Ranjini Ambalavanar: ORI because the patient chart and enrollment form are different, so there could be some falsification of research record. Whether there is a record .. . falsification or not, if there's an adverse event, you do report, so at least it's on the record, yeah.

Dr. Patricia Valdez: So if ..

Dr. Ranjini Ambalavanar: I sent in the chat box.

Dr. Patricia Valdez: And so if there's an assessment and they find this, that's the point .. Is that the point that they should contact ORI, or is it after an inquiry?

Dr. Ranjini Ambalavanar: Our regulation calls for reporting any adverse event to be on the safe side. Whether it's research misconduct or not comes after it. I put the ..

Dr. Patricia Valdez: To report ..

Dr. Ranjini Ambalavanar: It's 93318 in the regulations, so .. .

Dr. Patricia Valdez: And then about OHRP, report to them immediately, is that right?

Dr. Ranjini Ambalavanar: Right.

Dr. Patricia Valdez: I'm sorry. I don't see the question .. the answers anymore, but .. .

Dr. Ranjini Ambalavanar: Yeah. So the RIO and IRB chair coordinate review of all relevant clinical and research records. So this is kind of starting like a research misconduct proceeding, right? Because when you see the discrepancy between enrollment document and the clinical chart, you see there is some falsification going on, and also there is a person who signed the form that is the nurse that enrolled the patient. Anyway, after an hour .. Second bullet I'm reading. After an hour, they have found three other cases where the information regarding eligibility criteria on the research intake form does not appear to match that in the patient's clinical file. They also found several instances where records completed by the same nurse for the patients' follow-up visits to monitor health after conclusion of the therapy do not include subject's initials as required by the protocol, so consent forms and in the therapy were not signed by the patient. Should this matter proceed from assessment to inquiry? That's a no-brainer. Yes. Yes.

Elyse Sullivan: And we've got a lot of yeses in the chat.

Dr. Ranjini Ambalavanar: A lot of yeses. Yeah, so here we have enough evidence to move forward, and one thing we haven't asked is if this is PHS-funded research, that it's relevant to us. If the funding comes from non-NIH sources, ORI does not have jurisdiction over it. Okay. Thank you, and that's the end of this case, and I'll hand over to Patricia to really .. Let's discuss the case two.

Dr. Patricia Valdez: Yeah, I want to go back because I saw there were some questions about the .. whether they're supposed to report every adverse event to ORI. And so as I understand it, you should go to the IRB and to the IC and to OHRP, but is it only reportable to ORI if there's a research misconduct investigation ongoing? Can you clarify that?

Dr. Ranjini Ambalavanar: Yeah, so if a proceeding is going on as in this case, I put the slide later, but during the proceedings they .. even before the death happened, you found these records falsified, and during misconduct proceedings you come to know that there is a death, then you have to inform us.

Dr. Patricia Valdez: Right. So, yeah, once the proceedings have started. Okay.

Dr. Ranjini Ambalavanar: Once .. Yeah. We deal with research misconduct, so that's where we come in. Otherwise, your funding agency and OHRP definitely, yes.

Dr. Patricia Valdez: Only if PHS funds are involved.

Dr. Ranjini Ambalavanar: Yeah.

Dr. Patricia Valdez: Sorry, all the alphabet soup. I will try to spell it out. So I have a case study. This is actually a 10-part case study. So just hang in there with me, and we'll kind of follow through, and I'm going to have you kind of tell me what to do next. It's like Choose Your Own Adventure. Okay. So part one. So, again, RIO is Research Integrity Officer. ORI is the Office of Research Integrity, where Ranjini is at. So this is part one. So Rebecca RIO, so she's the institutional RIO, informs HHS ORI of a decision to move to investigation after an inquiry into allegations of falsified data in multiple NIH-supported publications belonging to the PI, Dr. Smith. The publications span several years, and Dr. Smith is corresponding author on all of them. Now the inquiry committee found that the data in question were also used in an NIH grant application that was recently awarded, so the investigation committee will consider this data as well. Okay. So the question here is, should Rebecca RIO also inform the NIH of the ongoing investigation? And I think here you could .. Do they have to have permission to raise their hands to answer questions?

Elyse Sullivan: They can. Yes. So when we were doing the A, B and C, it was .. Okay. Let's see. We've got some hands raised. All right.

Dr. Patricia Valdez: All right.

Elyse Sullivan: Let's see. I'm going to call on Michelle Wong. I'll allow you to talk and ask you to unmute. And you can go ahead.

Dr. Patricia Valdez: Hi, Michelle.

Michelle Wong: Oh, hi. I meant just to vote on the side of yes when I accidentally hit the raise hand. Yes.

Dr. Patricia Valdez: Well, you're right.

Michelle Wong: I'm glad.

Dr. Patricia Valdez: Okay. So I'll go ahead ..So yeah, so the answer is yes. Again, because the falsified data, the potentially falsified data is included in an NIH grant application that was recently awarded, that's something that we want to know. Again, if there's a research misconduct proceeding, understanding that things need to be kept confidential but if there are grant compliance concerns that arise, you definitely need to let NIH know about the ongoing investigation. And so we generally ask institutions to let us know when they decide to go to an investigation after an inquiry. Okay. All right. Part two. Let's see if I can do this. If anyone else .. We can have multiple people answering questions too. So part two, shortly after the investigation begins, Dr. Smith sends an e-mail to his lab asking that the person responsible for the data falsification come forward immediately to end the investigation. One of the lab members forwards the e-mail to Rebecca RIO.

Oh, I saw something in the chat. It says, "Is notifying ORI the same as notifying NIH?" No, it's not actually. So we do communicate on some cases, so when ORI finds that there is a problem like human subjects at risk, they will let us know, but we're distinct entities, so we're all part of HHS, but ORI is in a different place, different office than the NIH. So what you tell to ORI does not go to the NIH. Okay? So report to both.

Okay. So here we are. So we have .. . One of the lab members forwards this e-mail about Dr. Smith sending this e-mail asking for the person responsible for the data falsification to come forward. So what should Rebecca RIO do about Dr. Smith contacting lab members about the investigation? Okay. Does someone want to raise their hand and say something?

Elyse Sullivan: We have one hand raised. Mita, Meta, I'm going to call on you and ask you to unmute. Sorry if I butchered your name.

Mita: Yeah, hi. No, my question is slightly different. If someone's publication there was inquiry and investigated and was retracted and this all happened during the grant review process and the grant got funded, do the NIH needs to be informed about this? Without knowing whether there is data used or not in the grant, but say if something like this happens and it happened while the review is going on, but by the time funding comes, maybe write up that funding or something, the paper is retracted. What one is supposed to do?

Dr. Patricia Valdez: So if NIH supported the research that went into creating that paper, so if that paper relied on NIH funding and there was a finding of research misconduct involving that paper, then NIH should be informed. Okay? Okay. Does anyone else raise hand?

Elyse Sullivan: Nope. I don't see any raised hands.

Dr. Patricia Valdez: Okay. So they are telling me that .. Yeah, instruct Dr. Smith to cease and desist. Yes. So really she needs to go and tell Dr. Smith that he cannot interfere with research misconduct proceedings because this is not appropriate. Okay. So Rebecca RIO reprimands Dr. Smith for attempting to interfere in the research misconduct proceedings. All right. The following week, Dr. Smith asks a few lab members to meet with him privately to discuss the allegation. During this meeting, he pounds his fists on the desk and demands the lab members to tell him who is responsible for the figures in question. When they fail to give him a name, he screams, and he throws a lab notebook at the wall, narrowly missing their heads. Paula Postdoc calls Rebecca RIO to tell her about Dr. Smith's questions and his violent behavior. To whom should Rebecca RIO report Dr. Smith's violent activity? What do you guys .. Anyone raise hands? Anyone raise hands?

Elyse Sullivan: Yep. I see a hand, and just a reminder, this is to sort of to .. regarding the case studies not for additional questions. Robin Lewis, would you like to talk us through?

Robin Lewis: It should be reported to HR and dealt with through that perspective from the .. Well, as far as throwing a lab book and screaming at them, that's an HR issue.

Dr. Patricia Valdez: Yeah, absolutely. Thank you, Robin. So yeah, it's definitely an HR issue. It could potentially be .. I guess HR is probably the best place to go. Some institutions have different reporting mechanisms and offices to report, but in general, HR is the place to go. And again, this is a little bit separate than the research misconduct portion. Really this person .. Dr. Smith has started to engage in other harassing behaviors that are of concern. Okay.

Okay. So part four, Rebecca RIO files a report with HR. The HR investigation finds that Dr. Smith bullied his lab members and created a hostile work environment. As a result, university officials place Dr. Smith on administrative leave. Dr. Smith is not allowed on campus and is prohibited from communicating with members of his lab. Okay. So now here is a question. Who needs to be notified about Dr. Smith's change of status, the fact that he is now on administrative leave, not allowed on campus, prohibited from communicating with members of his lab? Who ..

Elyse Sullivan: All right. Let's see, and we have a few hands raised. Let's go to David Hudson. What do you think?

Dr. Patricia Valdez: Hi, David.

David Hudson: Patricia, I think we notify you of this.

Dr. Patricia Valdez: Hey. Yes. Hey, David. How are you?

David Hudson: Good. Thank you.

Dr. Patricia Valdez: Yeah, absolutely. So yes, so now .. So in case you're not aware, there was . With your last appropriations after 2022, there is a new law put in place where if there is a .. someone who is a PI in an award or key person, key personnel on an award, named in the nearness of an award, if that person has been disciplined for harassment, bullying, unsafe work environment, the institution is required to notify the NIH. Okay? So that's .. So I just wanted to let you know that. Okay. So yeah, so .. So there's a question about protected status. I just saw it over there, but yeah, hostile work environment is irrelevant of the protected status, so we want to make sure .. So there's a requirement on the .. . that all NIH work is being conducted in a safe work environment, so that's what this is about.

Elyse Sullivan: And, Patricia, there's a question. Wouldn't you also have to get permission for a change in effort related to ...

Dr. Patricia Valdez: Oh. Yeah. Yes.

Elyse Sullivan:. this ..

Dr. Patricia Valdez: Very good question. So that is part of the next piece.

Elyse Sullivan: Whoops.

Dr. Patricia Valdez: But no, but that's a really good question. So now the guy, Dr. Smith, is basically not in the lab and gone. So what happens now? There's an NIH award that has no PI working there. So here's what they did. So those university officials, they notified NIH of the disciplinary actions taken against Dr. Smith in response to the bullying and creating a hostile work environment, which is what I was talking about. So they decided, "Okay, let's .. We're going to have Paula Postdoc in charge of the NIH project since Dr. Smith is on administrative leave." They decided, "Paula Postdoc, she's the most senior in the lab. We thought that she could make a good PI." So the question is, can the university decide to make Paula Postdoc the PI of the grant and just let it be and happen? Okay. I see a lot of answers. Any raised hands?

Elyse Sullivan: Yep. We got a few raised hands. How about Sarah Hall? What do you think?

Sarah Hall: I think no, that they would need to get permission from the grants management specialist.

Dr. Patricia Valdez: Very nice. Yes, thank you, Sarah. So yeah, so the university official, usually it's the AOR, the .. I'm sorry, wait for the alphabet. It's the Authorized Official University .. . Authorized .. . University Authorized Organizational Representative, AOR, see I even get mixed up too. So that person will need to contact the NIH grants management specialist, and hopefully they have come up with somebody in mind, or if they come forward with Paula Postdoc, they can put Paula Postdoc forward, but they have to get prior approval from the NIH. So they'll go to the NIH grants management specialist and say, "Here's Paula Postdoc. We think she's great. We think she can do the work," and they give the bio sketch. And then program official, the grants management person, they will go ahead and talk about them, determine whether or not she is appropriate to serve as a PI. So, again, prior approval needs to happen. Prior approval needs to be obtained by the NIH. Okay. Okay. And yeah, likely ..depending .. We probably wouldn't put the postdoc as a PI, just saying, but you never know.

Okay. So part six, okay, Rebecca RIO works with the university's Authorized Organizational Representative, AOR, to obtain prior approval from NIH for a change in PI on the active award. And we don't know who that PI is. It's somebody not the postdoc. Okay. As the investigation proceeds, it becomes clear that the majority of the raw data for the figures in question are missing. Rebecca RIO and university officials decide to stop drawing down funds on the NIH project because they're uncertain about the authenticity of the data included in the application, and they're concerned that the subsequent research might be affected. Okay. Here's the next question. Who needs to be notified if the university decides to stop drawing down funds or stop spending on the NIH award?

Elyse Sullivan: Do we have any hands? Looks like Mary Ann Allen. Would you like to talk us through?

Mary Ann Allen: Sure. I'm not actually sure of the answer of this. What I was going to ask is, these are great answers, but what if you don't know the answer to them, and you hit one of these? Who do you go to at NIH if you don't know the answer to this?

Dr. Ranjini Ambalavanar: Good question.

Dr. Patricia Valdez: Yes. Yes. So okay, well you can come to our office and let us know, the Office of Extramural Research, to ask questions, but I will say, if you have a grant and you are .. Just take a look at the notice of award. The notice of award will give you the name of the program official and the grants management official or the grants management specialist. Those are the people that you need to contact regarding this type of a change. So in this case, the NIH needs to be notified, and, again, look on your notice of award and determine who is the grants management specialist for that particular award and contact them. Now, one thing I'll also mention here is that, again, this is a research misconduct proceeding. It's got to be confidential, so we don't want you to spill your heart out and tell the grants management staff that there's this research misconduct going on involving Dr. Smith and blah, blah, blah. We don't want you to go that far. So, again, keep it confidential. You can tell them what .. that you need to change the PI or you need to take some NIH extramural grant in this case, we're stopping drawing down funds on the project. Now, at the same time, you want to contact our office in OER just to let us know about the research misconduct investigation. So you say, "Okay, we have informed the NIH that there's a research misconduct investigation." Our office will keep the information very siloed. We only share that information on a need-to-know basis, and because we do tell our extramural staff, like the POs, the program officials, and the grants management specialists about the importance of confidentiality, sometimes somebody forgets they've heard that message, and they may tell someone. So to be careful, we say tell us about the issue, the research misconduct issue, and tell the grants management specialist about any type of actions you need to take on the grant, and we can help coordinate that with you too. Okay? Okay. Let's go to part seven. Okay. So several months later, the investigation continues, and Dr. Smith notifies university officials that he has a tentative job offer at a new university out of state. The job offer requires that he bring the NIH grant with him, so Dr. Smith asks his current university to transfer the active NIH grant to his new university. Okay. Quick question right here. So who's grant is it? Is it the university's grant, or is it Dr. Smith's grant? Does anybody have ..raise a hand? Yes, okay. I see university's. Okay. Yes, so it's the university's grant. It's not Dr. Smith's grant. He can't demand that the grant come with him. So good. So Rebecca RIO and other university officials, they have reservations about transferring the grant, especially since the data in the application may be unreliable. So Dr. Smith mentions that he is contacting a lawyer to make sure his interests are protected. And I know when lawyers get involved, institutions sometimes are .. Sometimes they may cave, but hopefully they don't. And so the question here is, what are the university's options regarding the grant? What can they do now that Dr. Smith says he's moving to a new institution and he wants to take the grant with him? Does anyone have any raised hands?

Elyse Sullivan: I don't see any raised hands. There's some .. Oh, I do .. I do see one.

Dr. Patricia Valdez: There's a couple of options here.

Elyse Sullivan: Yeah, okay. Mary? Let's .. Would you like to talk us through?

Mary: Sure. I was just going to say, since they've started a research misconduct investigation, whether or not the grant leaves, they should continue that investigation at their institution. I'm not exactly sure whether they can transfer a grant that's under an investigation. That would be a question that I have for the panelists.

Dr. Patricia Valdez: Yeah. Okay, yeah. No, you're absolutely right. They definitely need to continue the investigation. Does anyone else want to comment on what they might want to think about doing with the grant? What are their options regarding the grant? Should they just let it go to the new university? Is that their only option?

Elyse Sullivan: I see ..

Dr. Patricia Valdez: Any other hands? I see some answers.

Elyse Sullivan: Susan? I pick you, Susan.

Susan: Yeah, I think that NIH probably would have something to say about where the grant goes or not. So if I were the institution, I would contact NIH and tell them what's going on and seek assistance because I don't think it's good to transfer a grant that's under investigation.

Dr. Patricia Valdez: Yeah, that's so true, very true. And I do see some of the answers here. So ..Ooh, some good stuff. Okay. So basically there are some .. There's a couple of options. So the university can decide to relinquish the grant, and that means when they relinquish the grant, that means they're kind of giving it up, which gives the new institution the opportunity to come in and take the grant. So basically that's allowing the grant to transfer to Dr. Smith's new institution. So that's one option. The other option is the institution can decide to keep the grant, and they may decide to .. So if they do decide to keep the grant, they will have to identify a new PI and, again, get prior approval from the NIH for that new PI to work on the grant. Now, if they don't have anybody with the expertise that could serve as PI, then they could think about terminating the grant. This is what .. This would be what's called a bilateral termination, which means there's an agreement between the NIH and the institution that the grant will be terminated. So those are some of the questions. Is there anything else that I ..

Elyse Sullivan: So and, Patricia, we've got a couple questions about let's say the grant is allowed to be .. is relinquished and transfers to the new institution. Is institution A here required to tell institution B about the misconduct investigation or any of the goings-on, are they required?

Dr. Patricia Valdez: They are free to tell the new institution about it if their counsel allows them to. We don't have a requirement they do that. Now, the reason that we ask you to let us know about ongoing investigations is, one of the reasons is precisely this. If we find that you, the institution, relinquished a grant and now the new institution is trying to take it, what we can do is hold it until that investigation is complete, so we really want to know whether or not the data are problematic, whether it was material to the grant. So we can hold it at that point and then wait for the transfer to happen later. So that's something that we could do as well. Any other questions? Should I continue? Okay. Okay. We'll go on. We can come back to questions, too.

Okay. So part eight, the university decides to identify a suitable PI to the oven the grant for the remainder of the project. So this way the grant is going to stay at the university, and Dr. Smith's trainees are going to continue to be supported. This is something that the NIH really cares about is to .. We don't want to just take grants away from people. If the grant is able to continue, we want to make sure that it can. Okay. So Dr. Smith receives an official job offer from the new university, and he immediately resigns from his current university to start his new life, free of research misconduct allegations and investigations. Now, meanwhile, the investigation at his former university is nearing the end. So first question, should Rebecca RIO mention the ongoing investigation to the new institution? What do you guys think?

Elyse Sullivan: Let's see. So we've got one hand that David Hudson. I think you're a frequent flyer. Go ahead.

Dr. Patricia Valdez: Hey, David.

David Hudson: Yeah, thanks.

Dr. Patricia Valdez: What do you think?

David Hudson: Well, I think the answer to answer to the first question is fairly clear, that is we would not be under any circumstances thinking it was reasonable for us to contact the new institution, and I think our attorneys would back up .. They would insist even more strongly that we not do so.

Dr. Patricia Valdez: I think .. .

David Hudson: My understanding though is, if I can look at the second question or the second half of this, I understood that we could count on your office to make that contact and that we can use you as the intermediary to the new institution should you think that that's reasonable in this case.

Dr. Patricia Valdez: Yeah, and that's right. So when you think about it .. So the question, should you mention the ongoing investigation to the new institution, and here at this point since it's an ongoing investigation, there's no finding, and so depending on your institution, your counsel may say, "Well, we don't have a finding. We don't know if this guy did something bad or not, so we really shouldn't go to the new institution just yet." So that's a possibility. But as David mentioned, if we know there an ongoing investigation that involves a grant ..the data in the grant, then that's something you'd come to us, and we can help work that out. As I mentioned, we can hold it until you're done with your investigation and then we go from there. And if the grant ends up moving, what we can do is .. We would ask the first institution's permission to share an investigation report if possible because what we would want to do is to impose specific award conditions on the new grant, likely, like require additional supervision, things like that. Now, it's possible that the problem of the grant was just so bad that it had to be terminated anyway, so the new institution might end up having to terminate the grant, so it just depends on how it goes. Okay. So I think it would be nice if the first institution could notify the second institution, but clearly I understand the logistical problems ..the legal problems there that could arise. Okay. So the second part of the question is should the new university, so university two, ask Dr. Smith if he's cry under investigation or if his former institution made findings of research misconduct against him? Does anybody want to say more about this piece?

Elyse Sullivan: I don't see any hands yet.

Dr. Patricia Valdez: Okay.

Elyse Sullivan: But we've got a couple yeses in the chat.

Dr. Patricia Valdez: Yeah.

Elyse Sullivan: Okay. We've got a couple of hands. Let's see. Beth, would you like to talk us through?

Beth: I was just thinking that it could be a very legitimate question in your conflict of interest questions that you have with your institution and can be asked in that manner.

Dr. Patricia Valdez: Yeah, that's a really great idea. That's a really good idea. Now, I know there's at least one institution I can think of that is piloting putting this into new contract, particularly for tenured individuals, and the contract would say something like, "You must disclose to your new university whether or not you are subject of an investigation at your former institution or have findings against you at your former institution, and if you fail to disclose that to us, your contract is terminated." So this is something that people are thinking about more. Also, when it comes to harassment cases, and that's another situation where you don't want to pass the harasser, and so if the person is under investigation or they have a finding, the new institution would likely really want to know that information, and so I think the newest .. a lot of institutions are trying to find ways to incorporate that either through COIs or maybe these initial contracts to make sure that they are told this information. Okay. I see Lori had said, "But at the time they're asking, there are no findings, so the answer is no even with a pending investigation, need to ask if they're under investigation or have findings." Yeah, so if they're under investigation or if they have findings, yep, exactly. Again, if they're under investigation, again, there's no findings yet, so really you may not want to take actions at that point, so you might want to wait. Okay?

Okay. So let's see here. I'm going to part .. . our next piece. Okay. The investigation is complete a few months later. The investigation committee was unable to determine who was responsible for the falsified/fabricated data in seven NIH-supported publications and two grant applications, including the active award, so no raw data. Okay. They couldn't figure out who was responsible. The deciding official agrees with the committee that the publication should be retracted. So the report notes concerns about data management practices in Dr. Smith's lab, in particular the raw data for the figures could not be located. The report also notes that Dr. Smith's inclusion of falsified/fabricated data in an NIH grant application constitutes recklessness. Okay. So now .. So to whom should Rebecca RIO send the report?

Elyse Sullivan: Let's see. Any hands up? And I don't see any yet. In the chat, we've got some people saying NIH and ORI, NIH and ORI, just ORI.

Dr. Patricia Valdez: Okay. So according to the research misconduct regulations, institutions are required to notify ORI, so they must send a report to ORI. Now, at the same time, because NIH . This was an NIH-supported publications that are going to be retracted, and we'll have to think about what we're going to do with that, the institution should also notify the NIH. Now, again, we don't need the full report. We just need to know that there was a problem that impacted this NIH grant. So we need to know what grants were involved. We need to know publications or what the research .. what research was affected. So that way we can make a determination about for instance the active award, what should happen with that active award? So maybe now if the award is being held, it should be terminated. And then we can also, again, think about recoupment of funds depending on what the extent of the research misconduct was in the papers, so .. .

Elyse Sullivan: And, Patricia, we have a question about the use of the term recklessness.

Dr. Patricia Valdez: Oh, yeah.

Elyse Sullivan: Does that have a specifically kind of connotation here in the misconduct world?

Dr. Patricia Valdez: Ranjini can speak to that too. So Ranjini was mentioning the example where the PI was held ..who submitted the grant application was held responsible for the falsified data that someone else put .. had falsified, I guess, and that they may have been held accountable, and that could have fallen under recklessness, right, Ranjini?

Dr. Ranjini Ambalavanar: Yes. That was recklessness, but a caveat is you can't just call it reckless. Usually the recklessness come to as a reason is when the scientist is warned about some risk. So say some lab member is telling that this data are not reliable, someone .. other colleagues are making data, making up, and the PI still ignores it and does not address it and then puts data from that particular individual into the grant without checking the raw data. That's recklessness. You are reckless even after being notified, and so it's not just a PO .. You can't just call anything reckless, but you have to be in a tough situation then it counts as reckless compared to intentional misconduct.

Yeah.

Elyse Sullivan: And, Patricia, a couple of questions about notifying NIH. Who exactly at NIH would you notify in this scenario?

Dr. Patricia Valdez: Exactly, so that would be me, my office. Either .. I'm patricia.valdez@nih.gov, or we have a mailbox, nihresearchintegrity .. I think that's right .. @mail.nih.gov.

Elyse Sullivan: That's what it is.

Dr. Patricia Valdez: Put it in the chat.

Elyse Sullivan: Yeah, we'll find it and put it in the chat. I'm sure Christine can help us with that.

Dr. Patricia Valdez: It's actually .. It should be on the slides or somewhere, but I'll show you where to find it.

Elyse Sullivan: So it wouldn't go to the PO or the grant specialist. That would go to your office.

Dr. Patricia Valdez: No, because it's specifically about research misconduct findings, and you want to tell us because we'll make sure that it's held confidential and only shared with those on a need-to-know basis. Okay.

Okay. So part 10, this is the last piece. So ORI receives the investigation report and notifies NIH of the data retention concerns identified by the investigation committee. Rebecca RIO then e-mails the NIH RIO, that's me, about the investigation committee's findings and provides a list of affected publications and grant applications along with details of the findings. Okay. So the question is, should the university consider returning funds to NIH? Or should they just wait for NIH to come back and demand funds?

Elyse Sullivan: Let's see. Any hands? I don't see any hands yet.

Dr. Patricia Valdez: So I see, yeah, wait for NIH's determination or yes, so it's always helpful to have the institution come forward and when they're disclosing this to NIH say, "Hey, we've calculated this, and here's the part that's a problem, and we propose to return these amount of funds." That's kind of a .. It's a good effort. Then of course, we'll come back and look at it and see if we agree with it, but it's always, I think, helpful for universities to come back with a proposal .. to come first with a proposal. They don't need to, but just consider it. Okay. So, again, yes, you should consider it. Okay. so ..

Elyse Sullivan: We've got a question for .. Sorry. We've got a question, Patricia. What happens to people who are being paid off the grants that is being relinquished or terminated, particularly students and things like that?

Dr. Patricia Valdez: Yeah, and so .. And exactly that. That's one of the reasons why that's kind of the last ditch effort, is we really don't want to have a grant terminated because we do worry about those trainees and whoever is being paid on the grant. So when we talk to institutions, we really encourage them to try to identify another alternate PI who can do the work or serve as PI. Because unfortunately, if we do have to resort to that last ditch effort, then unfortunately, the money is gone at that point. Yeah, so ..

Okay. So what are some actions that NIH might take in response to Rebecca RIO's notification? And I think I mentioned to you. So about these before. Maybe someone can just reiterate if anyone wants to raise their hand and talk again. Okay. Well, I mentioned all these things before, so we basically would think about return of funds, and so now that we know that this person, Dr. Smith, has data retention issues in his lab, we .. In any grants going forward, maybe we would maybe at least for a year or two, we may incorporate specific award conditions telling which .. state that he has to provide us with the raw data or have an oversight committee review the raw data at the institution. So there's a couple things that we could think about doing here. But, again, this person at this point is not debarred, so he hasn't .. There's not an ORI finding. This is just an institutional finding, so the person, this Dr. Smith can still apply for NIH grants. So .. But, again, because he's doing that, we're going to make sure that there are these award conditions put in place to make sure that the data are good and we don't have this problem again, so .. Okay. And that's. Let me just.. Oh, yeah, so here are the policy references for my case study. If you have questions, you can go back and see all these, request for prior approval, change of budget and scope, and this is about harassment piece. Okay. This is our website, so I just want to share with you if you're NIH staff, you can access staff resources here. If you have any questions or comments or want to report, just click on this button, OER, research integrity, that will take you to the mailbox that I mentioned, and you can make reports to NIH there or submit allegations there as well. Okay? And this is ORI's website, and, Ranjini, you can say more here. If you've got questions, ask ORI. All right. I think that is the end.

Elyse Sullivan: All right. Thank you so much to our presenters and our participants. We hope that this has been very useful and you have some new information and some resources to kind of get you going. I do want to issue a couple of thanks and a couple of reminders. So special thanks to Dr. Christine Ring, who is one of the Research Integrity Officers at NIH, for assisting with questions behind the scenes, and I do want to remind folks about that office hours opportunity. You can reserve a 20-minute chat with these very in-demand experts and get some of your questions answered, and I also wanted to mention that this is one of several preconference events. So our .. We have a monthly series. The next month's topic is International Collaborations, and this is all gearing up towards our sort of larger grants conference that'll be February 1 and 2. Let's see. So in closing, thank you so much. The slides are already available, and a recording will be available in about 5 to 7 business days within the grants conference center. So thank you again to everyone who participated, and we hope this is very useful.

Dr. Patricia Valdez: Thank you.