Preventing Publication of Falsified and Fabricated Data: Roles of Scientists, Editors, Reviewers, and Readers

Abstract

INTRODUCTION In science, a research paper is often the final result of a process of hard work, perseverance, skills, and good ideas. Accurate communication of new research findings in the context of previous work stands at the core of the scientific process and progress, and scientific misconduct can have devastating effects on the scientific community and society. The October 2014 issue of the Journal of Cardiovascular Pharmacology published a paper by Liu et al on inward rectifier channel activation by zacopride and its impact on cardiac remodeling and function. The original source paper is as follows: Liu CF, Liu QH, Liu EL, Zhai XW, Zhang L, Luo TE, Zhang WF, Feng QL, Cui XL, Zhao ZQ, Cao JM, Wu BW. Activation of IK1 channel by zacopride attenuates left ventricular remodeling in rats with myocardial infarction. J Cardiovasc Pharmacol. 2014;64:345–356. The reason why we refer this source in the text and not in the reference list is because we believe retracted papers should not appear in citation statistics. The paper was subsequently retracted in the March 2015 issue of the same journal for reasons of “flawed data representations” and the authors apologized for “making such serious and obvious mistakes in this article, which has the potential to mislead individuals working in the field.”1 We take this case as a starting point for evaluating publication fraud and then suggest a number of actions that may improve its prevention. Analysis of the Liu et al paper demonstrated that parts of the panels of figures 1, 2, and 5 in the original publication were overlapping. Figure shows 2 identical echocardiographic recordings said to be obtained from 2 animals under different experimental conditions. One image is stretched and has altered contrast with respect to the other image (here Fig. 1A). Figure 2 displays 3 partly overlapping regions of the same immunohistochemical section as being derived from 3 separate animals under different experimental conditions (Fig. 1B). Finally, Figure 5 depicts one electron microscopic image twice, to represent 2 different experimental animals. The second image is mirrored, turned, and enlarged with respect to the first (Fig. 1C). Although data presentation was flawed in this case, fraud has not been proven formally. Some of the authors submitted a new version of a manuscript including new artwork but apparently reporting the same study as judged from the study design and averaged data, to another journal on December 20, 2015, and published it on August 2, 2016.2 Seven authors featuring the original Journal of Cardiovascular Pharmacology paper do not reappear on this version of the paper. Sadly, the authors not only potentially misled the Journal of Cardiovascular Pharmacology readers by their flawed data representations but also those of other journals. We found that 2 other published papers from authors on the Liu et al paper were retracted in 2015.3,4 The retraction note for the first one indicates that they “have used different fields of the same samples … to represent two distinct groups on two occasions in Figs. 4 and 5.” Furthermore, they stated that “some raw data for … the use of echocardiography for Figs. 8 and 9a were not available for further analysis to exclude that also the same samples were used for two distinct groups.”3 The retraction note for the second paper states that the paper was retracted by the authors “because of publication of questionable figures in the article” some of which showed overlap with a previously published paper of the same authors.4FIGURE 1.: Examples of “figure irregularities” from the retracted article of Liu et al (Activation of IK1 channel by zacopride attenuates left ventricular remodeling in rats with myocardial infarction. J Cardiovasc Pharmacol. 2014;64:345–356). A, Excerpt from Figure 1 showing 2 echocardiographic recordings presented as different experimental conditions, which are in fact identical recordings (as indicated by us with the white dotted squares). B, Excerpt from Figure 2 (a–d) showing 3 images (a, b, d, indicated by us by black dotted squares) of an identical histochemical section, presented as obtained from 3 different animals. C, Figure 5 excerpt of an identical electron microscopic photograph (indicated by us by the white dotted squares and striped outline of an identical cell structure) presented twice as being obtained from 2 different experimental animals. Adapted and published with permission of Wolters Kluwer.More recently, a similar case appeared in the Journal of Cardiovascular Pharmacology. Again, an identical echocardiography recording was presented as being derived from 2 different experimental animals in which one recording was somewhat stretched compared with the other. These authors retracted their paper upon being informed of the editorial concerns about the duplication.5 These types of errors are not unique to publications in any one journal, but seem widespread in the research field. In the well-known proven case of scientific fraud by the stem cell researcher Hwang, duplications of 4 microscopic photographs said to being derived from different cell lines were revealed. The authors' initial defense that these were the result of accidental mistakes did not hold after an independent investigation by the Seoul National University was conducted.6 Different organizations, local boards, and journals use slightly different definitions of scientific misconduct. But they all have in common that the fraud is a deliberate act to mislead, as is clearly indicated by the definition from the Office of Research Integrity.7 Obviously, research is often a complicated chain of demanding experimentation, technical challenges, and data interpretation and it is clear that honest mistakes can be made during this process. These obviously fall out of the definition of scientific misconduct. On the other hand, data falsification (ie, the act of altering existing data or manipulating research processes to obtain favorable data), data fabrication (ie, making up data), and plagiarism (eg, using text or ideas from others without proper citation) all fall within the definition of research misconduct.7 We would like to use these acts in the context of “publication fraud,” which narrows the scope and distinguishes them from other forms of scientific misconduct such as unethical behavior with respect to human and animal studies, or sabotage. We will not discuss “salami” or “slice-and-dice” publication in which large data sets are split up to prepare multiple publications, whereas one comprehensive publication would suffice. Although this practice is, in our view, unwarranted, it is within the gray area of scientific conduct. There are no good estimates of the scale of publication fraud in science. In the Netherlands, 2 large series of fraud have been detected in recent years, one in social psychology and one in cardiology.8–10 Other research areas and countries have also faced similar misfortune.11,12 The website “Retraction Watch” reports on an almost daily basis on new retractions that often result from publication fraud being detected.13 With the exception of high-impact publications, the majority of these flawed papers will go unnoticed by the scientific community as a whole, let alone the general public. THE DETRIMENTAL EFFECTS OF PUBLICATION FRAUD Fraudulent papers cause damage at several levels. First of all, when undetected, false data and their general implications may push a research field into the wrong direction and thus may lead to fruitless follow-up research efforts by colleagues causing losses of valuable research money and time and frustration on the part of the people involved. Second, because research funding and publication opportunities are limited, honest scientists may be “outcompeted” by fraudulent ones to a varying extent. In this way, sparse resources may be wasted. Whereas fraud seems to involve only a minor segment of the total research budgets available,14 it is likely that most fraud goes undetected. Third, fraudulent data in medical research fields may cause harm to patients, especially when medical guidelines are based on these, for instance, as seen in the Poldermans case (perioperational use of beta-blockers),9,10 the Study 329 case (paroxetine use in adolescents and suicide),15 or when public opinion is based on these as seen in the Wakefield case (Measles, Mumps and Rubella MMR vaccine and the perceived, but nonexisting, association with autism and bowel disease).16,17 Fourth, the general public may lose faith in science, which may affect the willingness to support, contribute financially to, or participate in research. HOW TO PREVENT PUBLICATION FRAUD The question arises whether anyone involved in the publication chain of the several examples mentioned above could have discovered the figure irregularities or manipulations. We believe that the vast majority of researchers are honest and make every effort to present their data in the best possible manner to represent reality. Most editors, reviewers, colleagues, and readers will share this opinion, and data falsification and fabrication is not the first thing that comes to mind when reviewing a manuscript or paper. Unfortunately, falsification and fabrication of data is a reality in the scientific community. Furthermore, some specialized commercial bureaus exist in some parts of the world whose business is to prepare scientific manuscripts and steer them through the peer review process, and the possibility of malice exists here as well.18–20 To counter these threats to our scientific community, we will discuss a number of implicit and explicit control mechanisms for scientists, editors, reviewers, and readers, which may decrease the occurrence of publication fraud. Scientists Students are our future scientists. Many universities have implemented practical courses on research integrity, including publication fraud. Nevertheless, improvements in such courses can be made, and comprehensive courses should be implemented in each curriculum worldwide. Not only will such types of education increase awareness, they will also encourage a mindset in which issues can be discussed earlier and easier. In our experience, lectures and e-modules that present and discuss clear examples of publication fraud in the most recent existing scientific literature have much more impact than “only” presenting “ethical guidelines” to students. In assignments, we challenge our students by stating that “figure irregularities” can be found within 1 hour in papers that were published the week before. Students are very well able to distinguish between “good” and “bad” science and often wonder why so many cases of publication fraud go undetected. One of our students put it as follows: “I did know that I should be critical of a text, but the figures I always took for granted. After the course, I now start by viewing the tables and figures before reading and being influenced by the authors description of what should be observed.” If consequences (Table 1)21–25 and incentives (Table 2)14,26–29 of fraudulent behavior are discussed with students, this hopefully will produce a next generation of scientists who value “good science” more than the sheer number of publications or impact factors they achieve.TABLE 1.: Potential Consequences* of Publication Fraud for Authors and Their InstitutesTABLE 2.: Factors Increasing the Risk of Publication FraudAt the work site where primary data are produced, senior scientists must take responsibility for promoting a culture of “good science.” They should be aware of the subjects their juniors are working on at an almost daily basis and also should have insight in the experimental methodology, and the accompanying challenges and pitfalls. In the digital era, data falsification and fabrication are easier to achieve as compared with times when microscopically images, northern and western blot results, to name a few, were analog and artwork for publications was often prepared with the help of an institutional photographer or colleague. Current software for figure preparation is easy to use: the author, alone, can handle the entire process from data generation through final figure preparation,30 eliminating the implicit control mechanism derived from the involvement of a second person. Therefore, raw data sharing should be performed on a weekly basis within a team to prevent subsequent bias and manipulation. Furthermore, seniors should keep open minds. Well-planned and performed experiments will provide good data, even when the outcome is not in line with their hypothesis. Junior scientists should be made aware of the serious consequences publication fraud can have (Table 1). Although apparently less frequent, data manipulation can also result from the action of the principal or senior investigator as seen in the Diederik Stapel and Dipak Das cases.24,31 In such cases, the consequences can be more far reaching.32 Juniors should be aware that these practices may have a direct influence on their own careers. Their doctoral work can become compromised without their being aware of it, their laboratory can be closed, and funding can be restricted or terminated. Therefore, juniors should take responsibility for checking their individual contributions to the final version of a manuscript and contemplate whether the statements and overall conclusions made correspond with their own findings. This latter point obviously requires a safe environment in which perceived irregularities and gray areas can be discussed with peers openly. In cases in which juniors feel no other choice than to bring the fraud of their supervisor into the open, they must do so very carefully and preferably should contact their institution's ethics officer. Institutions have the responsibility to create an optimal environment to perform good science, which includes active research ethics policies. Visibility of the ethical guidelines and offices can be promoted in many places. Institutions may install whistleblower tools, which will help them to identify individual investigators who pose a hostile/abusive environment for their juniors. In response to accusations, institutions must act in accordance with established institutional, local or national guidelines, or guidelines provided by the scientific community. But above all, a safe environment that promotes discussion of research ethics and gray areas seems crucial. Finally, governmental legislature and funding organizations, at local, national, and international levels, can further enforce good scientific practice by active prosecution of fraud and reclaiming ill-used research funding. Because many universities and research institutes are closely linked to and financed by governments, common sense would state that such actions could be implemented without too many difficulties. Finally, publication pressure and impact factor stress with respect to competition for tenure positions and grants put pressure upon scientists and may impact publication fraud. We will not enter into this discussion but refer the readers to a few interesting findings and opinions in this area.27,28,33 Editors and Reviewers Reviewers are not policemen but are primarily recruited for their scientific expertise to evaluate the soundness of manuscripts on a voluntary basis. We think that publishers and journals should do their best to avoid sending fraudulent manuscripts to reviewers. First, we think that journals would benefit by adapting a policy to obtain written statements from each author on the validity of the data they provided and the conclusions linked to their contribution to the manuscript. Second, many journals screen manuscripts for plagiarism upon submission using special software, although the need for manual inspection still seems to exist.34 Several journals communicate that they screen artwork for irregularities after acceptance.35,36 These actions most likely reduce the number of fraudulent manuscripts that enter their submission portals, even when the analysis is being performed on randomly picked papers only. Preferably, however, screening of artwork should be done before sending a manuscript to reviewers so they only have to concentrate on the soundness of the scientific content. Unfortunately, such triage is time consuming, even when performed by trained experts in the field, and prolongs total manuscript handling time. If this check could be performed using software at a reasonable price, the process could become common practice for all journals. Until then, we feel that editors and reviewers should be aware of the possibility they are evaluating fraudulent work, as stated earlier by others: “be watchful, but not distrustful.”37 From our experience, reviewers feel unequipped for detecting publication fraud and think of it as an additional time-consuming task in the evaluation process of a manuscript. In our opinion, reviewers should only evaluate those experimental sections on which they have sufficient knowledge and, preferably hands-on, experience. A to the which parts of the manuscripts fall within the expertise will suffice. editors should obtain insight for each manuscript whether are left in the evaluation process and may whether additional reviewers should be of manuscripts of likely would benefit From a point of view, it may be very to start manuscript evaluation by screening the primary data for and when these to the who should then on the data manipulation is clear any a manuscript should be out of the review process as as all reviewers from the need to evaluate a fraudulent manuscript. In the should the issue to the at which the authors are in an to prevent of fraudulent to another Obviously, it is much to data manipulation when experimental are presented in or tables only. Reviewers should be aware that for insight in the raw data is a and journals may these In fact some for implemented the policy to that and and including should be provided in the figures or the could help to their reviewers by clear to Figure 1, of fraudulent behavior in the research area in which they We think that reviewers would it also when journals state that sharing concerns on potential misconduct is Furthermore, journals must be in the they the who fraudulent behavior from subsequent or publication fraud is the paper should be retracted and the retraction note should clearly state the reason for Unfortunately, misconduct is the reason for retractions to but too in retraction is In our opinion, retraction should more clearly state whether data or plagiarism is or proven by using these when editors feel at provide a description of the irregularities to our in Figure published retraction provide clear of irregularities found in figures and are good examples of such this practice will more clearly distinguish acts of scientific misconduct from honest mistakes and errors in a are a in efforts to decrease publication fraud. We and others often publication fraud in of papers from the same research Therefore, when one publication fraud in a paper or it is to screen additional published papers from the same authors for With some the practice of publication fraud is used on a can go on for years, and can with an author from one to In our experience, many journals and editors, but not all, to such by readers and act would help journals that these contributions of readers are and they install on their or provide special for reporting such and journals should also make clear statements on they handle whistleblower and on potential implications for readers who to publication fraud. should be however, that put the on of fraud on the even when the efforts of an or lead to retraction of a the of to may be because of institutional or even national In that the same individuals to their Publication of false data to the existing control in the publication process. of these practices

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 11 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.