Tackling Fuzziness in CSR Communication Research on Social Media: Pathways to More Rigor and Replicability

Round 1

Reviewer 1 Report

Thank you for the opportunity to review this work. The paper addresses a very interesting topic, that is, antecedents and effects of CSR communication on social media based on real-world data, in addition, the author develop a guideline for identifying CSR communication on social media that addresses common pitfalls in existing research designs. 

The paper’s argument is built on an appropriate base of theory, and the research is well designed.  In this case the statement of objectives such as providing an overview of existing approaches to conceptualizing and operationalizing CSR communication on social media , between others, are correct for the resolution of the hypothesis. However, to improve clarity and understanding of the manuscript some changes are required.

The introduction covers a lot of background material, it demonstrates an adequate understanding of the relevant literature in the field and cite an appropriate range of literature sources. The first paragraphs point out a very important idea, the possibility that stakeholders perceive CSR communication as "greenwashing" in social media. It is well argued but it is recommended to include a methodology section after the "Introduction" section, in which it is made clear how the research was conducted.

From my point of view, the rest of the headings of the paper are well structured, with a clear and concise content. The guidelines will be very useful for future research on the subject, however, I would like to see a section prior to the conclusions dedicated more to "discussion of the results", in which the author summarizes the key findings, places the findings in context and discusses any unexpected results.

 As a final assessment, the article is written correctly from the methodical point of view. The formal and substantive assessment is positive, although it would be good to address all the comments submitted. After taking them into account, the article will be in perfect condition for publication.

 

Author Response

Thank you very much for your valuable feedback and suggestions for improvement. I addressed your comments in the following way:

It is well argued but it is recommended to include a methodology section after the "Introduction" section, in which it is made clear how the research was conducted.

The author guidelines for this journal suggest to include a "Materials and Methods" section in empirical research articles right after the introduction. However, for review articles (like this one) it states that "[...] It is not necessary to follow the remaining structure". Therefore, I took the freedom to diverge slightly from the template. However, you have a valid point in that the literature review methodology (i.e., article search and selection) should be more visible in the structure of the paper. Therefore, I added a subsection "review methodology" within the main literature review chapter and extended the explanation of the deployed methodology.

I would like to see a section prior to the conclusions dedicated more to "discussion of the results", in which the author summarizes the key findings, places the findings in context and discusses any unexpected results.

Analogous to my previous point I diverged slightly from the typical structure of empirical research articles to account for the review character of my paper. The development of guidelines for future research is akin to design science research (DSR) where this type of structure is typical. However, you are right that an explicit discussion of the results should be included in the paper. To address your point I renamed chapter 4 to "Discussion and Development of Guidelines for Future Research" and extended the discussion of results contained therein.

Reviewer 2 Report

First of all, the main message the introduction is trying to convey is not clear. The introduction does not clearly explain what, why and how the Authors are trying to study. In particular, the Authors do not explain the motivation behind the choice of the research topic. Why should it be studied in the first place? Why is it important? The main goal of the paper and research hypotheses are not clearly described. Therefore, I would propose to restructure the introduction according to the following more usual format:

- Overview and motivation

- Problem Statement

- Objectives of the Study

- Research Contribution

- Research Questions and the Methodology the study applies

- Key Findings of the Study

- The structure of the paper

 The paper has a proper literature review section that contains references to many papers but some of them are not very new. Moreover, it is not clear how these papers are related to each other and to the current work.

The proper section with the theoretical background is missing which makes the interpretation of empirical results difficult. The research hypotheses are not derived directly from a formal theoretical framework and are not properly described. The specific mechanisms underlying the postulated relationships are well not explained. The postulated empirical relationships need to be carefully explained by referring to specific theories or at least previous empirical studies that made references to the theory. 

The main concern regards the empirical analysis which does not move behind the simple characterization of facts. The paper is in fact about data description and there is no real attempt to test or discriminate between alternative theories or to test hypotheses derived directly from the theory.

Author Response

Thank you very much for your feedback and constructive criticism. Your comments helped me a great deal to improve the paper further and I hope my revisions can convince you of the value and contribution of my article.

Before I address each of your comments individually, I would like to clarify that this is not primarily an empirical research article but a review of the existing literature and a critique of conceptual and methodological shortcomings and inconsistencies (in short, "fuzziness") therein. It thus adopts a meta-perspective on current research. Your comments seem to convey a large degree of scepticism towards this type of research in general. However, I propose that fuzziness poses a significant problem to the research domain and hinders its further progression. Well-defined concepts and rigorous and replicable methodologies are the foundation of sound theories in any field. Therefore, this issue needs to be addressed.

To your specific points:

The main message the introduction is trying to convey is not clear. The introduction does not clearly explain what, why and how the Authors are trying to study. In particular, the Authors do not explain the motivation behind the choice of the research topic. Why should it be studied in the first place? Why is it important?

As I am not clear whether this comment addresses the importance of CSR communication research in general or the importance of my study in particular, I would like to comment on both points. The motivation for better understanding antecedents and effects of CSR communication in general from a company perspective is most commonly traced back to the issue of legitimacy pressure which ensues when there are actual or perceived divergences between firm behavior and the generally accepted social norms, values, and beliefs (Luo et al., 2022). Organizational legitimacy has been demonstrated to be positively correlated to financial performance and “illegitimacy” can lead to economic, legal and social sanctions (Dowling & Pfeffer, 1975; Lindblom, 1993; Lodhia et al., 2020). CSR communication is one of the most widely discussed tools to manage legitimacy pressure and thus it is important for companies to understand its effects in different settings and under various conditions.

However, our understanding of how CSR communication can be used to mitigate legitimacy pressure can only be improved through scientific research when there is conceptual clarity. In this paper we demonstrate that this is not the case in this domain and make suggestions on how this problem can be fixed. This paper is important because without it there is a large risk that future research will continue discussing different phenomena under the same label, thus creating more confusion than clarity.

Apparently I did not manage to convey these points effectively in the original version of my paper. Therefore, to accommodate your criticism, I rewrote the introduction using the structure that you proposed below and adapted the line of argumentation to the points made above.

The main goal of the paper and research hypotheses are not clearly described.

Indeed, the paper does not state any hypotheses. This is not customary for a literature review. Much rather, literature reviews are the foundation for the development of hypotheses in theory-testing empirical research papers.

Concerning the goals I disagree that they are not clearly described. The introduction states the following: "The goal of this article is explicitly not to develop a new method for content classification as the literature for both manual and automated content classification approaches such as qualitative and quantitative content analysis, text mining, and topic modeling is abundant. Instead, we aim to provide an overview of existing approaches to conceptualizing and operationalizing CSR communication on social media including typical challenges and obstacles in executing and documenting the respective methodologies. We shall illustrate these challenges by describing our method of identifying CSR communication in a large sample of Instagram posts and the most difficult coding decisions involved. Finally, we will develop a set of guidelines which shall help future researchers in this field to make their research more rigorous and their results more replicable."

You may disagree whether these goals are worth pursuing but as stated above I consider them very relevant and important based on the problems posed by fuzziness in this field.

Therefore, I would propose to restructure the introduction according to the following more usual format [...]

Thanks for your suggestion. I rewrote the introduction in the structure you proposed.

The paper has a proper literature review section that contains references to many papers but some of them are not very new.

71.4% of the articles discussed in the main literature review are from 2019 or newer. The oldest article is from 2014. Where older articles are referenced they are used to introduce original theories. Based on these numbers, I consider the recency of discussed articles appropriate.

Moreover, it is not clear how these papers are related to each other and to the current work.

It is customary for a literature review to have a specific focus (see vom Brocke et al., 2009). In this case the focus of the review is on conceptualisations and operationalisations of CSR communication for the reasons stated above. While I fully concur that a comprehensive discussion of how the papers are related to each other would be an interesting analysis, this is not the goal of this paper and would be out of scope.

The proper section with the theoretical background is missing which makes the interpretation of empirical results difficult. The research hypotheses are not derived directly from a formal theoretical framework and are not properly described. The specific mechanisms underlying the postulated relationships are well not explained. The postulated empirical relationships need to be carefully explained by referring to specific theories or at least previous empirical studies that made references to the theory. 

In my understanding this comment is based on a misconception of what the article aims to accomplish. As discussed above, this is not an empirical hypothesis-testing article but a literature review and methodological critique. It does not postulate any new empirical relationships. Rather, it addresses problems at the conceptual foundation of the field which, I propose, is an equally valuable contribution. In this type of research the formulation of hypotheses is not customary or expedient.

The main concern regards the empirical analysis which does not move behind the simple characterization of facts. The paper is in fact about data description and there is no real attempt to test or discriminate between alternative theories or to test hypotheses derived directly from the theory.

Again, there seems to be a misconception regarding the purpose of this paper. The empirical part is not designed to test any specific theory or hypothesis (as explained above). Rather, it is used as an illustration of the problem of fuzziness which was discovered through the literature review. Against this background, I consider the design of the empirical part of the paper appropriate and conducive to the achievement of the research goals.

To summarise my response to your feedback, I propose that theory-testing empirical research is not the only way to advance knowledge in a research domain and that  reviewing the current state of research in a field and - if necessary - suggesting path corrections is an equally valuable contribution.

 

References cited in this response letter:

Dowling, J., & Pfeffer, J. (1975). Organizational legitimacy: Social values and organizational behavior. Sociological Perspectives, 18(1), 122–136. https://doi.org/10.2307/1388226

Fernández, P., Hartmann, P., & Apaolaza, V. (2021). What drives CSR communication effectiveness on social media? A process-based theoretical framework and research agenda. International Journal of Advertising, 1–29.

Lindblom, C. K. (1993). The Implications of Organisational Legitimacy for Corporate Social Performance and Disclosure. Critical Perspectives on Accounting Conference.

Lodhia, S., Kaur, A., & Stone, G. (2020). The use of social media as a legitimation tool for sustainability reporting A study of the top 50 Australian Stock Exchange (ASX) listed companies. In Meditari Accountancy Research (Vol. 28, Issue 4, pp. 613–632). EMERALD GROUP PUBLISHING LTD. https://doi.org/10.1108/MEDAR-09-2019-0566 

Luo, X., Zhang, R., & Liu, W. (2022). Environmental legitimacy pressure, political connection and impression management of carbon information disclosure. Carbon Management, 13(1), 90–104. https://doi.org/10.1080/17583004.2021.2022537

vom Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., Cleven, A., Brocke, J. V., & Reimer, K. (2009). Reconstructing the Giant: On the Importance of Rigour in Documenting the Literature Search Process. ECIS 2009, 2206–2217. https://doi.org/10.1108/09600031211269721 

Reviewer 3 Report

I suggest explaining better the policy implications of this research from the various stakeholders. Also, the further studies should be developed. Moreover, the contributions should be better clarified and explained.

Author Response

Thank you very much for your valuable feedback. I appreciate your comments and addressed them in the following way:

I suggest explaining better the policy implications of this research from the various stakeholders. 

As this paper is mainly a critique of conceptual and methodological shortcomings in the field of social media CSR communication research, the principal stakeholder addressed through this research is the research community in this domain. The "policy implication" for this stakeholder group is clearly stated in the guidelines in Table 5. However, you are right in that the paper could expand further on the implications that CSR communication research has for the communicating brands or for regulators. To address this point, I added some comments on this to the conclusion.

Also, the further studies should be developed.

In my understanding this comments asks for more explicit suggestions for future research. Two such suggestions are already included in the last paragraph of the paper and the main artefact developed in the paper (i.e., the guidelines) is designed to guide future research. However, to address your point I extended the paragraph on future research and referenced the research agenda developed by Fernandez et al. (2022).

Moreover, the contributions should be better clarified and explained.

This point has been addressed by multiple reviewers. Therefore, I rewrote the section within the conclusion stating the contributions of the paper to make them more explicit.

Reviewer 4 Report

Looks like the author did a great job by developing a guideline for identifying CSR communication on social media that addresses common pitfalls in existing research designs. 

The literature review is extensive and there is a really interesting overview of existing approaches to conceptualizing and operationalizing CSR communication on social media including typical challenges and obstacles in executing and documenting the respective methodologies.

Guidelines developed in this paper help to guide future research towards more rigorous and replicable research methods, and to mitigate the fuzziness that pervades the domain.

I trully believe it is an excellent work.

Author Response

Thank you very much for your positive feedback. It is much appreciated.

Round 2

Reviewer 2 Report

The paper has been substantially improved. However, the number of unaddressed issues still remains. In particular, the goal of the paper is still not well explained. In the letter of response the Author has explained that the main goal of the paper is to provide a literature review. However, this is still not clearly stated in the introduction. Moreover, if the paper is about the literature review still some methodological issues should be more clearly discussed. In particular, if the paper is based on the meta-analysis it should be better explained how this meta-analysis was done. In particular, how the papers were selected and which specific criteria were used for the literature review. Finally, it is not clear what the purpose of Section 3 is. This section goes beyond the standard literature review and it is not clear what is wants to communicate. In its present form it simply does not fit where it is currently located. Therefore, I would suggest either rewriting it or shortening and moving to the appendix. In the revised concluding section I do not see any references to sustainable development. Therefore, I am not convinced that this journal is the right place for this paper.   

Author Response

Thank you very much for reviewing my revisions and your appreciation of the improvements. Please find my answers to your remaining concerns below.

In particular, the goal of the paper is still not well explained. In the letter of response the Author has explained that the main goal of the paper is to provide a literature review. However, this is still not clearly stated in the introduction.

The introduction states: "At the core of our study is an extensive literature review which includes a meta-analysis of the share of social media content that was identified as CSR-related, as well as a review of the conceptualizations and operationalizations of social media CSR communication in the existing literature."

In addition it lists 4 specific objectives which are enumerated in the text. Therefore, I disagree that the goals are not clearly stated.

Moreover, if the paper is about the literature review still some methodological issues should be more clearly discussed. In particular, if the paper is based on the meta-analysis it should be better explained how this meta-analysis was done.

Chapter 2.1 explains the methodology of the literature review and the meta-analysis. However, based on your concerns I added additional detail on the calculation of metrics in the meta-analysis in order to make it more precise.

In particular, how the papers were selected and which specific criteria were used for the literature review.

I added additional detail to this section (chapter 2.1) to make the literature selection process more explicit. The search criteria are now more clearly stated in the text: "we queried four academic search engines (Web of Science, Scopus, Ebsco, and Google Scholar) for journal articles containing any of the keywords “corporate social responsibility” or “CSR” in combination with the term “social media”. Each search engine was queried multiple times with different sorting options (by date, by relevance, and by citations) and a limit of 200 articles. This yielded 715 unique papers." The section also states two exclusion criteria and the inclusion criterion of being "empirical studies using social media analytics as their main methodology" which helped to reduce this sample from 715 to 64 relevant papers.

"Finally, it is not clear what the purpose of Section 3 is. This section goes beyond the standard literature review and it is not clear what is wants to communicate. In its present form it simply does not fit where it is currently located. Therefore, I would suggest either rewriting it or shortening and moving to the appendix.

I agree that this section goes beyond the standard literature review. However, I suggest that this is "a feature, not a bug". Section 3 is designed to provide an illustrative example of the problems previously identified in the literature review. In fact, the combination of a review and illustrative examples is relatively common in papers developing guidelines for a specific research discipline. Please refer to the following two A-journal articles as examples:

Rand, W. & Rust, R. T. (2011). Agent-based modeling in marketing: Guidelines for rigor. International Journal of Research in Marketing, 28(3), 181-193.

Venkatesh, V.,  Brown S. A. and Bala, H. (2013). Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems. MIS Quarterly, 37(1), 21-54.

In the revised concluding section I do not see any references to sustainable development. Therefore, I am not convinced that this journal is the right place for this paper.

While I do not use the term sustainable development in this section, the term CSR (corporate social responsibility) is used repeatedly and consistently. While the terms are not equivalent, CSR is clearly an essential driver of sustainable development. 

The reason why I selected this journal and still think that it is the perfect place for my paper is that Sustainability is leading the discussion on CSR communication that I aim to contribute to. 10 articles from Sustainability are explicitly referenced and discussed in my paper which indicates that the journal is a very good fit. 

A declared aim of the journal is to “promote scientific and other understanding […] related to sustainability”. A consistent terminology and conceptual clarity are essential preconditions for the advancement of our understanding of the business impact and the performative role of CSR communication. Thus, the call for more rigor articulated in my paper is aligned with this aim.

The journal also “strives to support the 2030 Agenda for Sustainable Development”. In my paper, the SDGs are used as a reference framework for the classification of CSR-related social media content. The article thus illustrates how the SDGs can not only provide guidance to practitioners but can also serve as a useful theoretical foundation in this field.

Finally, the journal explicitly encourages the disclosure of full experimental and methodological details. This call for reproducibility is also an essential theme of the guidelines developed in my paper. Based on these reasons I maintain that Sustainability is the ideal outlet for my paper and feel reassured in this position by the comments of the other reviewers.

Round 3

Reviewer 2 Report

The Author has provided appropriate explanations and addressed all comments. The paper can be published in the present form.