Persistent Citation of the Only Published Randomised Controlled Trial of Omega-3 Supplementation in Chronic Obstructive Pulmonary Disease Six Years after Its Retraction

Abstract

Scientific articles are retracted infrequently, yet have the potential to influence the scientific literature for years. The only randomised controlled trial to explore the effects of omega-3 polyunsaturated fatty acids in people with chronic obstructive pulmonary disease was retracted in 2008 due to falsified data. The objective of this research was to determine the frequency and nature of citations of this retracted paper. Web of Science and Google Scholar were used to determine the number of times the retracted article was cited. Citations were classified as either “retraction acknowledged” or “retraction not acknowledged”. The search was conducted on 6 August 2013 and updated on 25 March 2014. Results: The search resulted in 76 citations, of which 24 occurred prior to the retraction of the article. Of the 52 citations occurring after the retraction, only two acknowledged the retraction. Of the citations not acknowledging the retraction, 20 referred to specific data and 30 cited the reference in passing. This retracted article continues to be cited by authors, suggesting that information about the retraction was unsuccessfully communicated to the scientific community. Continual citation of retracted literature has the potential to bias a field of research and potentially misinform end-users.

1. Introduction

The retraction of scientific literature can occur for a number of reasons including honest errors by researchers or academic misconduct comprising plagiarism, falsification and fabrication of data [1]. While recent evidence suggests that the retraction of articles is increasing [2,3], it does not occur frequently with a mere 0.02% of articles archived in PubMed between 2000 and 2010 retracted [4].

In planning a current randomised controlled trial [5], the authors conducted a literature search for studies investigating the effects of omega-3 polyunsaturated fatty acids (PUFA) in people with chronic obstructive pulmonary disease (COPD). Only one randomised controlled trial was found, titled “Effects of Omega-3 Polyunsaturated Fatty Acids on Inflammatory Markers in COPD” by Matsuyama et al., published in the journal Chest in 2005 [6]. This study purported to supplement the diet of 64 COPD patients with an omega-3 PUFA or an omega-6 PUFA supplement for two years, and reported significant improvements in the omega-3 PUFA group in the 6 min walk test, perceived rate of exertion (Borg score) and arterial oxygen saturation (pulse oximetry (SpO₂)). The study also reported a decrease in sputum cytokines (inflammatory regulators) in the group receiving omega-3 PUFA with no change in the omega-6 PUFA group. The study concluded that “nutritional support with an omega-3 PUFA-rich diet is a safe and practical method for treating COPD” [6]. This paper was retracted by the journal in 2008 [7], following an institutional investigation that found data had been falsified by the lead author.

Retraction guidelines published by The Committee on Publication Ethics (COPE) recommend that a retracted article should not be removed from the scientific record but should be clearly reported as retracted [1]. “Notices of retraction should: (1) be linked to the retracted article; (2) include the title and authors in the retraction heading; (3) state who is retracting and the reason for retraction; (4) be freely accessible and appear in all electronic searches for the retracted publication” [1]. The International Committee of Medical Journal Editors support these recommendations and also suggest that the retracted work should be clearly indicated in all its forms including abstract, full text and PDF [8].

The aim of this study was to examine the frequency and nature of citations (pre- and post-retraction) of the above mentioned article by Matsuyama et al. [6], in order to exemplify the potential impact of retractions on the literature.

2. Methods

The Web of Science (Thompson Reuters, 2014) Cited Reference search was used to determine the number of times the retracted article was cited. This database includes the most important and influential journals, as determined by the Thomson Reuters journal selection process and is therefore not all-inclusive [9]. The Web of Science search was then supplemented using the Cited Reference function in Google Scholar. Google Scholar is a more comprehensive database and includes a wider selection of scientific and academic sources. The search for citations of the retracted article was conducted on 6 August 2013 and updated on 25 March 2014. No publication, time or language restrictions were imposed. Non English language articles were translated into English.

In order to determine whether or how well various databases adhered to the recommendations of COPE and The International Medical Journal Editor guidelines for reporting retractions, the methods described by Wright and McDaid [10] were used. Briefly the retracted paper was searched by title in the journal Chest and in each database most likely to include the journal Chest (i.e., Embase, Medline, CINAHL, Scopus, Web of Science, CENTRAL (Cochrane), Google Scholar, Pubmed, and Proquest).

Each article citing the retracted paper was reviewed in order to determine the context in which Matsuyama et al. [6] was referred. Articles citing Matsuyama et al. [6] were first classified as either “retraction acknowledged” or “retraction not acknowledged”. Those articles that did not acknowledge Matsuyama et al. [6] as having been retracted were then further classified into “specific citation of data” or “cited in passing”. A “specific citation of data” described details of the Matsuyama et al. [6] study such as design and outcomes (for example “improvement in oxygen saturation in the omega-3 group compared to the control” or “concentrations of inflammatory cytokines decreased by 40%”). A “citation in passing” was a citation that did not specifically mention the details or results of the study but cited Matsuyama et al. [6] in support of a concept (for example “Omega-3 polyunsaturated fatty acids are useful for treating inflammatory conditions”). All citations regardless of source and language were included in the analysis.

3. Results

The Web of Science Cited Reference search identified 44 citations, while Google Scholar identified 84 citations. From the 128 citations, 52 were excluded (47 were duplicate citations and five did not refer to the retracted paper). This left 76 publications in the final analysis. With the exception of two articles [5,11], all citations were positive (i.e., they did not refer to the article as an example of poor research). The citations occurred in a number of different publication sources (Table 1), with the majority in peer reviewed journal articles. The sources citing Matsuyama et al. [6] were published predominantly in English (n = 55) with the remaining 21 sources published in Chinese (n = 10), French (n = 5), Polish (n = 2), German (n = 1), Italian (n = 1), Spanish (n = 1) and Thai (n = 1).

Of the 76 citations, 24 citations were published between 2006 and 2008 prior to publication of the retraction notice. Of the remaining 52 citations, two acknowledged the retraction and 50 did not acknowledge the retraction. Of those not acknowledging the retraction, 18 were classified as containing specific data and 32 were classified as cited in passing. It is suspected that three of the articles that had not acknowledged the retraction were duplicate publications as, despite having different titles and first authors, they had very similar content [12,13,14]. Figure 1 presents the citations by year of article publication. The time between article submission and publication varies widely between journals; it is therefore possible that some of the citations may have been submitted for publication prior to the retraction, and not published until sometime after. To explore this possibility, the time lag between article submission and publication was investigated for peer reviewed articles with a complete submission history (i.e., dates recorded for submission, acceptance and publication) (See Figure 2). The time from submission to publication varied from months to years, with articles submitted from 2011 onwards having relatively shorter processing times. Three articles submitted prior to the retraction in 2008 were not published until 2009.

Table 1. Publication sources in which the citation of the retracted article occurred.

Source	Number of Citations
Peer reviewed article	50
Book	11
Website	6
Training module (Nutrition for people with respiratory disease)	3
Theses 1	2
Letter to the editor	1
Conference presentation	1
Patent	1
Trade Magazine article	1

¹ Theses publically available through university websites.

Table 1. Publication sources in which the citation of the retracted article occurred.

Source	Number of Citations
Peer reviewed article	50
Book	11
Website	6
Training module (Nutrition for people with respiratory disease)	3
Theses 1	2
Letter to the editor	1
Conference presentation	1
Patent	1
Trade Magazine article	1

¹ Theses publically available through university websites.

Figure 1. Articles citing the retracted article (without acknowledgement of the retraction), by year and citation classification. SC(r) indicates that a self-citation (which was also retracted) appears in that year; d indicates a suspected duplicate publication in this year.

Figure 1. Articles citing the retracted article (without acknowledgement of the retraction), by year and citation classification. SC(r) indicates that a self-citation (which was also retracted) appears in that year; d indicates a suspected duplicate publication in this year.

Figure 2. Submission, acceptance and publication dates of peer reviewed articles citing Matsuyama et al. [6] for which a complete submission history is available.

Figure 2. Submission, acceptance and publication dates of peer reviewed articles citing Matsuyama et al. [6] for which a complete submission history is available.

Table 2 shows common access pathways for Matsuyama et al. [6] and how clearly its retraction is denoted via each pathway. When each database was searched for Matsuyama et al. [6], Medline, Embase and Scopus databases specifically included the term “retraction” in both the article search record and the citation title, while PubMed included the notification in the search record only. Five databases (CINAHL, Web of Science, CENTRAL (Cochrane), Google Scholar, and Proquest) provided no indication that the article had been retracted. Proquest provided a direct link to the PDF version of the article which was not marked in a way that clearly identified the article as retracted. Three databases provided a direct link to the HTML version of the article via the “Chest” site.

Table 2. Notifications of retraction and access pathways for databases listing Matsuyama et al. [6].

Database	Noted in Article Search Record 1	Noted in Search Record Title 2	Noted in HTML 3	Link to “Chest” Article (PDF)	Link to “Chest” Retraction	Link to “Chest” Site 4
Medline5	√	√	N/A	X	X	X
Embase5	√	√	N/A	X	X	X
CINAHL	X	X	N/A	X	X	X
Scopus 5	√	√	N/A	X	X	X
Web of Science 5	X	X	N/A	X	X	√
Cochrane/CENTRAL 5	X	X	N/A	X	X	X
Google Scholar	X	X	N/A	X	X	√
Pubmed	√	X	N/A	X	X	√
Proquest	X	X	X	√	X	X

X: No, √: Yes, N/A: Not applicable; ¹ An indication anywhere within the database search record that the article has been retracted; ² Usually the word RETRACTION or RETRACTED in the title; ³ A website which displays the article text, this is not applicable where the article links to the Chest website; ⁴ Direct to article html, which includes retraction statement; ⁵ Retraction appears as a specific citation as a result of the article title search; Retraction notification criteria is based on the recommendations for article retraction by the Committee on Publication Ethics (COPE). Specifically whether all forms of the article (including electronic formats) indicate the retraction.

Table 2. Notifications of retraction and access pathways for databases listing Matsuyama et al. [6].

Database	Noted in Article Search Record 1	Noted in Search Record Title 2	Noted in HTML 3	Link to “Chest” Article (PDF)	Link to “Chest” Retraction	Link to “Chest” Site 4
Medline5	√	√	N/A	X	X	X
Embase5	√	√	N/A	X	X	X
CINAHL	X	X	N/A	X	X	X
Scopus 5	√	√	N/A	X	X	X
Web of Science 5	X	X	N/A	X	X	√
Cochrane/CENTRAL 5	X	X	N/A	X	X	X
Google Scholar	X	X	N/A	X	X	√
Pubmed	√	X	N/A	X	X	√
Proquest	X	X	X	√	X	X

X: No, √: Yes, N/A: Not applicable; ¹ An indication anywhere within the database search record that the article has been retracted; ² Usually the word RETRACTION or RETRACTED in the title; ³ A website which displays the article text, this is not applicable where the article links to the Chest website; ⁴ Direct to article html, which includes retraction statement; ⁵ Retraction appears as a specific citation as a result of the article title search; Retraction notification criteria is based on the recommendations for article retraction by the Committee on Publication Ethics (COPE). Specifically whether all forms of the article (including electronic formats) indicate the retraction.

When the “Chest” website was searched for Matsuyama et al. [6], the search results did not identify the article as being retracted and the site provided a direct link to the downloadable PDF version which does not indicate the article as being retracted. The HTML version of the article, however, clearly states an erratum and retraction and provides a link to the retraction notice. The retraction notice states the article title and authors, who is retracting the article and the reason for retraction.

4. Discussion

The results of this study indicate Matsuyama et al. [6] continues to be cited six years after the retraction notice was published. The continuing citation of Matsuyama et al. [6] is concerning, especially when “second generation” citation is considered; i.e., articles citing Matsuyama et al. [6] are themselves cited. The total number of second generation citations of the first generation of articles that have cited Matsuyama et al. [6] is currently 947 (481 citations for articles published before the retraction and 466 times for articles published after the retraction).

The findings of this study are similar to those of Sox and Rennie [15], Korpela [16], Neale et al. [17] and Grieneisen and Zhang [18] who reported that, even after a retraction notice has been issued, authors and peer reviewers still, presumably unknowingly, cite the article as though no retraction notice had been issued. Korpela [16] reported that citations of a retracted article persist for as long as 24 years after the retraction [16]. It is acknowledged that there is a time lag between manuscript submission and publication (see Figure 2); however in 2012, three years after the retraction, the Matsuyama et al. [6] article was cited 15 times. Grieneisen and Zhang [18] suggest the use of reference management software (such as Endnote™, New York, USA) [19]) be linked to a comprehensive retraction database. This is an interesting suggestion, however at present there is no comprehensive retraction database available. The RetractionWatch blog [20] provides up to date information on article retractions, often providing further information beyond the retraction notice. However, this blog was established in 2010 and, while it occasionally investigates historical retractions, it does not claim to be and is not a comprehensive retraction database. Pubchase™ (California, USA) is an online cloud based library and search tool which can be used online or with smart devices (e.g., smart phones, tablets etc.), which collaborates with RetractionWatch to notify users when an article in their library has been retracted [21]. PubMed includes the retraction reference in the title of the article, making it obvious that the article has been retracted, and also allows users to search for retracted publications by filtering results with the article type “retracted publication”. This is useful for searches conducted through PubMed but does not assist in notifying readers who access the PDF directly.

Therefore, the inclusion of the retraction notice as the first page of a PDF or watermarking the pages of the PDF as retracted would assist all readers, regardless of how they access the article. The guidelines published by the Committee on Publication Ethics provide other useful points on the requirements of a retraction notice [1].

As an example of the effectiveness of marking the PDF of the article as retracted, we considered the retraction of another article published by the same lead author. Matsuyama et al. [22] was published in the Internal Medicine Journal in 2007 (indicated as self-cited in Figure 1) and was retracted in 2008. The Internal Medicine Journal clearly states on the HTML site that the article has been retracted and provides a link to the PDF which has the retraction notice as the first page of the article. In comparison to Matsuyama et al. [6], Matsuyama et al. [22] has only been cited five times since it was retracted (one refers to the retraction). It is acknowledged that a number of factors influence citation counts [23,24]; however this suggests that these strategies may be effective in alerting authors to the retracted status of an article. Authors should be aware of the potential for article retraction and should be proactive. It is suggested that, before submitting a manuscript for publication, references should be checked with the PubMed database, which tags retracted articles in the publication type field as retracted [15].

End-users of the information are not limited to the scientific community. Improvements in technology and widespread access to the internet allows the wider community to access primary scientific literature as well as secondary interpretations of primary studies written for formal and informal medical websites. To illustrate this, a Google search using the phrase “fish oil and COPD” was conducted on the 31st of March 2014. Of the top 10 results returned, eight referred to Matsuyama et al. [6] without mention of the retraction; six of these were links to or a summary of Matsuyama et al. [6] with no indication that the information had been retracted. Google uses the users web history and geographic location to return relevant results to the user; to account for this the same search was conducted on the same day on a public computer, in the University of South Australia (City East campus) library, and in this instance three of the top 10 results returned referred to Matsuyama et al. [6] with no indication of the retraction. The top result in both cases was a summary of the Matsuyama et al. [6] article without indication of the retracted status. The majority of the results that referred to the retracted paper were published prior to the retraction on medical information websites such as WebMD. The information on these websites has not been corrected since the retraction of the article, nor is there any indication that the referenced material has been retracted.

A growing number of studies are investigating retraction prevalence, trends and impact on citation patterns in large databases [17,18,25]. Whilst our report on the impact of a single retracted paper may not have broad generalizability, our intent was to consider whether authors citing a retracted study indicated that the study had been retracted. There is considerable high level evidence to support a beneficial role of fish oil (containing omega-3 PUFA) in a number of inflammatory diseases, yet evidence of benefit in COPD is sparse and contradictory. Hence publication of a randomised controlled trial showing benefits of omega-3 PUFA in people with COPD would be noteworthy and likely to be highly cited by researchers and clinicians in the field and by the wider community using secondary medical information web sites. This may impact on future directions in this specific field of research.

The novel focus of our study was to see whether and how articles citing the retracted article acknowledged the retraction. A recent study by Chen et al. [26] clearly demonstrated that retracted articles can have a far reaching influence on citations patterns in scientific literature. The processes developed and used by Chen et al. [26] to synthesise citations, patterns and networks clearly advance the area in terms of direct and indirect citation of retracted articles. However, the volume of citations undertaken in Chen’s et al. [26] analysis did not permit them to identify the orientation of the citing sentence (where a positive orientation indicates agreement with the retracted article and negative indicates disagreement or identifying the retracted article as retracted). Our smaller sample of citations of a single retracted paper and the time frame since the retraction date permitted us to determine the citing sentence orientation and whether authors indicated that the paper had been retracted.

This study demonstrates a need for journals to report the retraction of articles in a uniform and explicit way to assist authors in recognising retracted literature and avoiding inappropriate citation. The continued citation of retracted articles by authors who are unaware of their status has the potential to bias the interpretation of the literature and adversely impact the direction and design of future studies. The retracted status of an article needs to be clearly and widely distributed so that authors and potential peer reviewers are aware that the article has been retracted.

5. Conclusions

In summary, while retraction rates are relatively low, a retracted article has the potential to impact the scientific literature long after it has been retracted. The way in which journals report retractions varies, with some providing clear retraction information, and others providing very little. A uniform approach by journals in the reporting of article retractions including the inclusion of “retracted” in the title and an indicator within the PDF document is important to alert authors. Authors and reviewers should also be aware of the potential for literature in their area to be retracted and should make attempts to determine whether any cited literature has been retracted prior to the submission of manuscripts.