Trust Us—We Are the (COVID-19 Misinformation) Experts: A Critical Scoping Review of Expert Meanings of “Misinformation” in the Covid Era

Round 1

Reviewer 1 Report

Conclusions require deep revision and should not include questions and extraneous quotations.

Conclusions require deep revision and should not include questions and extraneous quotations.

Author Response

Reviewer 1 (comments copied verbatim)

Major comments

Conclusions require deep revision and should not include questions and extraneous quotations.

Detail comments

Conclusions require deep revision and should not include questions and extraneous quotations.

Authors’ answer.

Dear reviewer,

We appreciate your taking the time to read our manuscript and provide feedback for its improvement. As a context to our reply, we note that we submitted a critical scoping review of close to 7,780 words, for a total of 15,367words including abstract, 149 academic references, PRISMA flowchart and checklist, and 23 pages of tables (three tables total). This level of detail is not unusual for a scoping review conducted by four investigators who carefully appraised and synthesized a wide range of sources on a complex, challenging, and controversial topic. For these reasons, in order to understand and address your request, we need some detail about what exactly you would like us to do, which unfortunately, we are unable to identify in your request. 

For example, you are asking for a “deep revision” of our conclusions, only 373 word long, yet we do not know what you mean by “deep revision”. We assume this is large, but what about these 373 words are you objecting to? What exactly in your judgment does not work, and why?

You are also requesting that we not include “questions and extraneous quotations”. In order to address this request adequately, we to know your reasons, because we obviously do not agree that either the one single question we have included at the beginning of the conclusions, or the scant quotations, are inappropriate, as otherwise we would not have included them. Allow me to explain why we have: first, questions are a very common stylistic feature of argumentative rhetoric, used to engage readers in the authors’ thought process towards a conclusion.

Nor do we believe that the use of quotations is inappropriate, quite the contrary, we believe it is required when quoting verbatim important expressions. This is what we do when, for instance, we quote Irving Zola, a major thinker in sociology, likely among the first, who critiqued the practice of medicalization, namely, a sociological mechanism whereby social issues are framed as falling under the jurisdiction of the medical sciences, oftentimes dispensing claims makers from providing any evidence or reasons and relying largely or exclusively on the social authority of the institution of medicine. This practice often leads to seeming “scientific”, medical interventions implemented “for our own good” – Zola’s words – when in fact they are implemented for the sake of special interests, thus the importance of giving credit to Zola for this otherwise rather trivial expression.

We hope you take these comments into consideration. We await your response so that we can provide a proper answer and revision.

Thank you.

The Authors

Author Response File: Author Response.pdf

Reviewer 2 Report

Authors should specify their lit review method clearly in the introduction .

the current body of literature that they gathered is limited , is there any reason ? That must be justified 

Paper needs clarification on what exact methodology they have used in the beginning , what question they are trying to address 

reason for this query and why results are narrowed when this subject is huge 

the authors can also discus the biases of data selectors and how they address they 

table with authors names and country , what is reason for this , is it a practise ? Not much familiar , and reason must be told 

Author Response

Reviewer 2 (comments copied verbatim)

Major comments

Authors should specify their lit review method clearly in the introduction .

the current body of literature that they gathered is limited , is there any reason ? That must be justified

Detail (sic) comments

Paper needs clarification on what exact methodology they have used in the beginning , what question they are trying to address reason for this query and why results are narrowed when this subject is huge

the authors can also discus the biases of data selectors and how they address they

table with authors names and country , what is reason for this , is it a practise ? Not much familiar , and reason must be told

Authors’ answer (organized as major / detailed, and portions within these) 

Major comments

1. Authors should specify their lit review method clearly in the introduction .

Dear reviewer,

Thank you for your comment.

Our introduction offers readers a succinct description of what we have done, namely “a critical scoping review of the expert literature in the medical and social sciences” (lines 82 - 84).

As we subsequently explain in the methodology section, by “expert literature” we mean literature vetted by the scientific community as per the peer review system. Exactly how the data was selected is carefully described in the “3.3. Data selection, charting, and analysis” section (lines 255 – 278)

2. The current body of literature that they gathered is limited, is there any reason ? That must be justified

Dear reviewer,

Thank you for your comment.

We are surprised that you would consider the body of literature we reviewed “limited”. If you conduct, for instance, a PubMed search of the medical literature with the word “misinformation” as a simple search term, 756,328 results will appear (as of July 10, 2024), obviously an impossible number of papers to screen, most of which are likely to be irrelevant to any specific research question. We therefore use medical subject headings, and misinformation is not one of them, although disinformation appears under the medical subject heading “communication”. When you apply “disinformation” to conduct the search as medical subject heading, only 84 articles appear. Because our scoping review intended to include not only medical but also social scientific peer-reviewed literature, it required judgment on our part to select and manageable amount of data, thus our choice of major, leading medical journals for the medical data set.

Similarly, we selected sources that represent alleged expertise in the field of “misinformation studies” among social scientists drawing from our own expertise in the field. After months of carefully screening the literature we selected a total of 68 articles for charting and analysis.

As background, we screened reports on the topic for major social institutions, namely, the World Health Organization, the US Surgeon General, the US Department of Homeland Security, Johns Hopkins University, and so on, all institutions that appear to have much to say about the topic of misinformation - or misinformation/disinformation/malinformation, which we chose to use following the practice of the US Department of Homeland Security.

We would like to better understand your request because in our judgment the body of literature we screened and gathered is not “limited” but actually very exhaustive.

Detail (sic) comments

Paper needs clarification on what exact methodology they have used in the beginning , what question they are trying to address reason for this query and why results are narrowed when this subject is huge

the authors can also discus the biases of data selectors and how they address they

table with authors names and country , what is reason for this , is it a practise ? Not much familiar , and reason must be told

Dear reviewer,

Thank you for your comment. We are breaking it up in sections to address each one in detail

1. Paper needs clarification on what exact methodology they have used in the beginning
   
   We believe we state the methodology clearly at the beginning, where we write that we are conducting a “critical scoping review”. Later under methods, we describe exactly whose framework for scoping review we use and why (there are more than one and we chose Arksey and O’Malley, a classic in the field), and what we mean by “critical”, as well as the reasons why we chose this approach, with relevant citations of the authors that inspire our work - primarily Carol Bacchi, but also Mark Saunders et al., who provide a framework justifying his call for reviews of any sort to be more “critical”. “On the Attributes of a Critical Literature Review.” Coaching: An International Journal of Theory, Research and Practice 4 (2): 156–62. https://doi.org/10.1080/17521882.2011.59648.
   
   
2. what question they are trying to address reason for this query

We have slightly rephrased lines 83 – 85 to make the goal of our review clearer. It now reads: “Our goal was to identify, summarize, and appraise that this literature means by the family of concepts coalescing on the notion of “misinformation” as it applies to Covid-19” (highlighted). The specific questions are described in detail, per usual practice, under Material and Methods, / Goal of the review and review questions (lines 244 – 253).

3. why results are narrowed when this subject is huge
   
   We could not agree more with you that the topic is huge, but we trust you would agree with us that so are most, if not all topics, in the medical world, and likely in other disciplines. For example, when the first author practiced medicine, she researched diabetes. Today, a PubMed search using “diabetes” as a simple search term yields 995,577 results, no less huge as misinformation, as we will soon argue. It would be very hard to conduct a review if you would have to screen them all.
   
   Similarly, if you conduct a PubMed search of the medical literature with the word “misinformation” as a simple search term, 756,328 results appear at the time of this writing (July 10, 2024), obviously an impossible number of papers to screen, most of which are likely to be irrelevant to any specific research question.

We therefore used medical subject headings, but “misinformation” is not one of them, although many medical journals talk about it, and “disinformation”, a related term, appears under the medical subject heading “communication”. When you apply “disinformation” to conduct the search as medical subject heading, only 84 articles appear. Because our scoping review intended to include not only medical but also social scientific peer-reviewed literature, it required judgment on our part to select and manageable amount of data, thus our choice of major, leading medical journals for the medical data set. Similarly, we selected using our professional judgment the sources that represent alleged expertise in the field of “misinformation studies” among social scientists. After months of carefully screening the literature we selected a total of 68 articles for charting and analysis. As a comparable example, Want et al included only 57, in their own publication on misinformation in the prestigious journal Social Science & Medicine (1).

Further, to provide readers a thorough background to our phenomenon of interest, i.e., the question of misinformation, we screened reports on the topic for major social institutions, namely, the World Health Organization, the US Surgeon General, the US Department of Homeland Security, Johns Hopkins University, and so on, all institutions that appear to have much to say about the topic of misinformation – or misinformation / disinformation / malinformation, or MDM, an acronym that we adopted, following the practice of the US Department of Homeland Security.

Qualitative research, in reality all research, requires professional/intellectual judgment to make a selection and we believe we have provided a detailed description of our process of selection, including combination of search terms, as well as relevance to our research questions, to make this process transparent. We can confidently communicate that we believe that this selection represents well the dominant view on our phenomenon of interest, which we subsequently critique.

3. the authors can also discus the biases of data selectors and how they address they

Thank you for this very useful suggestion. We had described the biases but we agree that our description was brief, so thank you for this observation. We have extended this section substantially. Please refer to pages 644 – 672 (highlighted)

4. table with authors names and country , what is reason for this , is it a practise ? Not much familiar , and reason must be told

Thank you for your comment. We have included a brief sentence clarifying the purpose of this inclusion in our revised section on our study limitations. By way of a more detailed explanation, it is indeed our usual practice, the reason being the complexity of the topic and the nature of the research questions, which is not too common in traditional scoping reviews. Another reason is that our review is “critical”, as explained in our methods section, thus our attempt to provide readers a detailed description of our data, listing salient features (e.g., country affiliation or disciplinary background of the authors) that may be relevant to our findings. A remarkable feature of our findings is that the dominant messaging around the “problem” of misinformation was no different whether authors were located in the USA (most were), Portugal, Taiwan, or Singapore, which speaks to the dominance of the identified framing. Experienced qualitative researchers offer generic advice on how to increase transparency, but do not provide “recipes” for doing so (2). Providing these tables, we believe, enhances the transparency of our work, and we have done so in many publications (3–5).

We hope you find our answers satisfactory and thank you once again for your support.

The Authors

References

1. Wang Y, McKee M, Torbica A, Stuckler D. Systematic Literature Review on the Spread of Health-related Misinformation on Social Media. Social Science & Medicine [Internet]. 2019 Nov [cited 2021 Nov 5];240:112552. Available from: https://www.sciencedirect.com/science/article/pii/S0277953619305465
2. Tracy SJ. Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research. Qualitative Inquiry [Internet]. 2010 Dec [cited 2020 Apr 18];16(10):837–51. Available from: http://journals.sagepub.com/doi/10.1177/1077800410383121
3. Chaufan C, Hemsing N. Is resistance to Covid-19 vaccination a “problem”? A critical policy inquiry of vaccine mandates for healthcare workers. AIMSPH [Internet]. 2024 [cited 2024 Jun 14];11(3):688–714. Available from: http://www.aimspress.com/article/doi/10.3934/publichealth.2024035
4. Chaufan C, Hemsing N, McDonald J, Heredia C. The Risk-Benefit Balance in the COVID-19 “Vaccine Hesitancy” Literature: An Umbrella Review Protocol. International Journal of Vaccine Theory, Practice, and Research [Internet]. 2022 Nov 9 [cited 2022 Nov 9];2(2):652–69. Available from: https://ijvtpr.com/index.php/IJVTPR/article/view/62
5. Chaufan C. Is Covid-19 “vaccine uptake” in postsecondary education a “problem”? A critical policy inquiry. Health (London) [Internet]. 2023 Nov 15 [cited 2024 Feb 17];13634593231204169. Available from: https://doi.org/10.1177/13634593231204169

Author Response File: Author Response.pdf