Ethical issues in preparing and publishing systematic reviews

Abstract

The medical evidence base, or ‘literature’, forms the basis for clinical and policy decisions, so those who contribute to it have a responsibility to ensure that it is as accurate and unbiased as possible. Since publications are also used to judge the productivity of individuals and departments, and to select candidates for academic positions, it is important that those who did the work receive fair credit. Preparing a systematic review is a form of research, and should therefore be undertaken in a responsible manner to ensure integrity and avoid misconduct. This paper sets out practical and ethical issues to be considered when preparing and publishing a systematic review. It is written primarily for authors involved in The Cochrane Collaboration, but has broader relevance. Cochrane Review Groups, which are responsible for the quality of reviews that they publish, have highlighted concerns about unclear or inappropriate authorship, copying of material from other sources, and dual or duplicate publication. Various journals and editors’ organizations have produced publication guidelines and instructions for potential authors. One of the best known is the Uniform Requirements for Manuscripts Submitted to Biomedical Journals, produced by the International Committee of Medical Journal Editors (1). Guidance is also available in the Cochrane Handbook (2). However, many of the requirements and conventions are scattered, so it may be hard for researchers to get a complete picture of best practices. Another problem facing those preparing systematic reviews is that general guidance, designed mainly for preparing primary reports of research, may not be applicable or may not cover the special issues that arise with systematic reviews. This paper therefore aims to help review authors avoid ethical problems by providing an introduction and overview of good practice in preparing and publishing systematic reviews. Guidance on authorship is available from several organizations (Table 1). Although the order of authors is not covered by most guidelines, it is common practice for the author who did the majority of the work to be named first and the others to be listed in order of decreasing contribution. All listed authors should have made a useful contribution to the process and should approve the final manuscript. One challenge for junior authors is that a senior colleague (often the head of department) may insist on being listed despite having contributed little – this is termed ‘guest authorship’. Another problem is ‘ghost authorship’, which is when someone who contributes substantially to a publication and meets authorship requirements is not listed. Not only are guest and ghost authorship unfair, but ‘inappropriate authorship practices by senior researchers… set a bad example and are likely to damage relationships between team members’ (3). Cochrane Review Groups may be able to help in such situations, but ideally, each team will discuss and agree how authorship will be allocated at the start of the project. It is now common practice and a requirement for Cochrane Reviews that the input of each author is recorded. This system of listing individuals’ contributions aims to increase transparency and reduce the incidence of guest and ghost authors (4). While determining authorship is relatively simple for original reviews, updates of previous Cochrane reviews may pose challenges. If little work is required for the update, then new authors may be added but the original lead author should remain. Where new authors make substantial updates, they usually become the lead, but previous authors may remain in the author list providing they approve the final publication. The Cochrane Collaboration also has mechanisms to arbitrate in difficult cases. Earlier authors usually drop off at the second or subsequent updates unless they have made a suitable contribution to the new document, but this may depend on the extent to which the updated review changes from the original. It is courteous to acknowledge previous authors. Authors of Cochrane reviews should consult the Cochrane guidance on co-publication (5). This guidance covers the publication of Cochrane reviews in print journals and lists the journals that will consider publishing such reviews after they have appeared in The Cochrane Library. While some journals may be prepared to publish versions of Cochrane reviews that have already been published in The Cochrane Library, as a general rule, journals do not want to publish work that has been, or will be, published elsewhere. Multiple publication of clinical trials (especially if not clearly disclosed) can skew the results of meta-analyses and is equivalent to ‘double counting’ patients in a study (6). Repeated publication of positive findings and suppression of negative findings may also have more subtle psychological effects on prescribers, leading to a misplaced confidence in a particular treatment or underplaying its adverse effects (7). Since journal space (and readers’ time) is limited, repeated publication of one person's work may prevent another person's work from being published. Therefore, the general rule for primary data is that it should be published, in full, only once, and overlapping or redundant publications should be avoided. When it is impractical or undesirable to avoid producing multiple papers from a single study, they should be cross-referenced and their relationship made clear to editors and readers (e.g., when presenting interim findings and several follow-ups, or a sub-group analysis from a large data set). Since systematic reviews are such an important part of the medical evidence base, and can contribute directly to clinical decision making, it might be argued that they should be made available as widely as possible, and this might require publication in a number of places. However, the need to disseminate reviews widely needs to be balanced against the costs of doing so, and also take account of publishing conventions. Although peer reviewers are virtually never paid by journals, the ‘cost’ of peer review is borne by the research/clinical community, since reviewers are usually busy people who must give up some other activity (such as research or teaching) to find time for reviewing. Getting submissions reviewed also creates real costs for publishers, who must employ staff to administer the review process and provide the infrastructure needed to do this (e.g., computers to enable reviewer databases, email, manuscript tracking systems, etc.). Persuading suitably qualified people to act as reviewers (usually for no fee) is sometimes difficult, and if editors have to approach several reviewers because a high proportion decline to review, this will also increase the costs to the journal. For these reasons, most journals will not consider work that has been published elsewhere or is under consideration at another journal, since they consider this a waste of reviewers’ time. Finally, submitting a review to more than one place at once may result in slightly different versions being published (due to differences in editing at different journals), which may cause confusion for readers. It is therefore unacceptable to submit any piece of work to more than one publication at the same time (8). The Cochrane Library is a peer-reviewed publication, just like a regular journal, so rules about simultaneous submission apply. Getting a protocol accepted by The Cochrane Collaboration carries both benefits and responsibilities for authors. Your protocol is reviewed by experts, so you receive suggestions on the best way to perform your review. Acceptance by a Cochrane review group also provides access to support from this group. Once a protocol title has been accepted, no other group within Cochrane will work on that topic. However, getting a protocol accepted also means that you are committed to following the Cochrane methodology, using the information management system and Review Manager application, and submitting your review for publication in The Cochrane Library. Therefore, if you are working on a review for which the protocol has been accepted and published in The Cochrane Library, the first full publication of your review should be on The Cochrane Library. After this, you may approach journals to see if they wish to publish alternative versions of your review, such as shortened versions for different audiences or translations. However, you should always cite The Cochrane Library as the primary source or reference, and you must check that subsequent publications do not break your copyright agreement with the publisher of The Cochrane Library. You should also make it clear in any secondary publications that the review has been published before, and it is good practice to mention this in your covering letter to the journal editor. You should also ensure you have the necessary permissions from within the Cochrane Collaboration (see the co-publication guidelines for details) (5). Care should also be taken if you re-use material from your own previous publications. In general, it is better to write each publication from scratch, as this usually produces the best writing style and flow. However, it may be acceptable to repeat some wording, especially when reporting the review methods. Many journals require authors to transfer the copyright to the publisher, so you may need permission to republish your own work. However, if you have published in an Open Access journal, or under a Creative Commons license or alternative license to publish, this will not be necessary and you will be free to re-use your own work, so long as the second journal permits this. Plagiarism means using somebody else's words, images, data, ideas, or other original creations without acknowledgement or permission and claiming them as your own original work. The most extreme form of plagiarism is to take a complete work and republish it as your own – this is clearly dishonest, and most researchers would never consider doing such a thing. However, because scientific publications, and especially systematic reviews, require citations of other people's work, the distinction between legitimate citation and plagiarism may become blurred. It is acceptable to describe other people's research in your own words, with appropriate citations, but it is not acceptable to use their words in your publications unless the origin of the words is clear, e.g., they are presented in quotation marks. Thus, earlier in this article, we summarised the findings of Tramèr et al by writing ‘Multiple publication of clinical trials (especially if not clearly disclosed) can skew the results of meta-analyses…(6)’. These are our own words, and the source of the evidence is clear from the reference. If we had wanted to use a sentence directly from the Tramèr et al. paper, it would have been necessary to indicate this by using quotation marks and a different sentence construction. For example: Tramèr et al. concluded that “Covert duplication of data has major implications for the assessment of drug efficacy and safety” (6). The problem arises in deciding how much text constitutes a direct quotation, and how much is simply common use of words. For example, anti-plagiarism software programs such as Turnitin™ search for strings of six words and indicate when any six-word string is found in another publication. Yet some six-word strings could easily arise without copying, for example, in papers about ‘chronic obstructive pulmonary disease in smokers.’ The reason why using such words, even though they have appeared in other papers, would not be plagiarism is that they do not represent an original thought or expression by the first writer who used them, but are simply a common description used by many writers. In short, if you wish to use original expressions from other publications in your systematic review, you should put them in quotation marks even if you have also included a citation. Just referencing another person's work does not allow you to copy any section (of writing, data, or illustration). Plagiarism is becoming increasingly easy to detect. A simple Google™ search can be used to find sentences that have been published elsewhere and are accessible on the Internet. There are also text-matching tools, such as CrossCheck, that are designed specifically for use by academic institutions and publishers, and can check longer sections of text and also match against content that is not freely available on the Web (9). However, while plagiarism is considered a serious form of misconduct, and deliberate misrepresentation of another person's work as one's own is unacceptable, there are some special considerations for Cochrane reviews. Firstly, an updated review is expected to repeat large parts of text from the earlier version. While it is important to acknowledge the work of earlier authors, this is not considered plagiarism or even redundant publication. Secondly, it is accepted that the methods section of a review often contains standard wording to describe the methodology recommended for Cochrane reviews. Thirdly, most reviews start with an introduction, which provides context on the question to be addressed and describes the condition or treatment studied. This section is important and should be clearly worded to make the review accessible to patients. If one review group has developed clear and relevant text, it may seem pointless to write it a different way, just for the sake of variety and to avoid plagiarism. If you want to use introductory material from another review, you should discuss this with your review group and agree how it will be acknowledged. As well as acknowledging those who worked on the review (in the author list and acknowledgements), it is important to provide information about funding and competing interests. The Cochrane Collaboration has strict rules about funding (2). All sources of support or funding for a systematic review must be declared. It is extremely important to state any reasons why you may not be considered as a neutral or impartial author on a review (or any other publication) you are preparing. This is often related to financial support, but competing interests can also be personal, political, or academic (10). Therefore, all review authors should declare relevant competing interests. The World Association of Medical Editors notes that “Conflict of interest (COI) exists when there is a divergence between an individual's private interests (competing interests) and his or her responsibilities to scientific and publishing activities such that a reasonable observer might wonder if the individual's behavior or judgment was motivated by considerations of his or her competing interests. … Everyone has COIs of some sort. Having a competing interest does not, in itself, imply wrongdoing. However, it constitutes a problem when competing interests could unduly influence (or be reasonably seen to do so) one's responsibilities in the publication process.” (10) Thus, even if you do not believe you have been influenced or biased, you should declare any interests that a ‘reasonable observer’ might wonder about. Readers of Cochrane reviews, who include both consumers and clinicians, expect that the data extraction has been done accurately and that the authors have not attempted to slant the results in any particular direction (i.e., that they are unbiased). Authors therefore have a responsibility to plan for accurate data extraction by ensuring that data is extracted independently by at least two authors, any discrepancies are resolved, and the decision on which data to include is agreed by all authors. Multiple publications of a single trial should be identified, as their inclusion in a meta-analysis can lead to an over-estimation of the effect (6). There are several methods to achieve this. Ideally, at least one of the authors should read all the included studies to spot duplicate publications. Secondly, the forest plot should be checked for numbers that are identical, and author names should be checked. The latter is not entirely foolproof, as new author names can sometimes appear on a re-publication, so that even experienced reviewers can be misled by deliberately confusing redundant publications (6). Systematic reviews sometimes reveal apparent plagiarism of whole articles (re-published by a completely different set of authors either in the original language or in translation). In such cases, it is good practice to alert the publishers of both articles to point out the similarity and suggest they should investigate this. A responsible publisher should follow the COPE flowchart on how to deal with suspected plagiarism (11). Very occasionally, reviewers will have to deal with published research that is subsequently shown to be fraudulent (for example, where the authors made up results). A recent example was that of Dr Scott Reuben, who published extensively on pain medicine. A team who looked at the impact of withdrawing Reuben's publications from published systematic reviews concluded that ‘exclusion of the Reuben reports never made any difference when the number of patients from these reports was less than 30% of the total’ (12). While 30% may not always be representative, it may prove to be a useful guide. However, conclusions always need to be drawn carefully when the work of any one author dominates a review, irrespective of any concerns about fraudulent publications. The Cochrane Collaboration has a method of annotating papers identified as suspect, and these should be identified as part of any search using The Cochrane Library. If the fraudulent papers are a major part of the evidence, or have very different results from those of other studies, removing them from a meta-analysis may change its conclusions. This happened in the case of one systematic review on the use of vitamin supplements in elderly patients, where large studies by a single author had to be excluded (13, 14). Journal editors may take action against authors who ignore publication conventions and commit misconduct. Many follow the COPE flowcharts and guidelines on retraction (15); however, there are no universally accepted codes on this. Serious plagiarism (of an entire article or large sections of text) may lead to a retraction. More minor plagiarism (such as one or two sentences used without proper attribution) may lead to a correction (which adds a reference and/or acknowledges the original work). Redundant (or duplicate) publications may also be retracted, or journals may issue a notice of redundant publication. PubMed also (independently) flags duplicate publications (16). Journal editors may also notify the authors’ institution in cases of proven or suspected misconduct. This may result in an investigation or formal disciplinary action. Journals may also ban authors from submitting further work to their journal for a certain period, although such action is controversial (and is not recommended in the COPE guidelines) (17). Authors of systematic reviews have certain important responsibilities. They should follow the general conventions on publication ethics and guidelines produced by various organizations, but they should also be aware of special issues that may arise. In particular, they should ensure that contributors are properly acknowledged, that potential conflicts of interest are declared, and that the review does not contain plagiarized material. The Cochrane handbook provides much useful guidance, and authors should also seek advice from their Review Groups. EW works as a self-employed consultant, providing training and consultancy on publication ethics to institutions, organizations, and companies. She is currently Chair of the Committee on Publication Ethics (an unpaid position) and developed the COPE flowcharts and retraction guidelines. She is a former Visiting Fellow of the UK Cochrane

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 120 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.