The evidence presented in the previous section suggests that sociology suffers from problems common across other social science disciplines, problems that lead to unreliability. The next logical question is whether sociology, as a discipline, has a commitment to being reliable. A broad concept of sociology includes studies that are ethnographic, grounded, anthropological, ethnomethodological, self-as-case-study, and critical or oriented at power structures, and most of these are scientific in the sense that they are committed to rigorous method, logical argument, and observation. However, some are explicitly not ‘science’ or reject the entire notion of science and reliability. The authors of such work, therefore, have no interest in replicability or reliability, if they have data at all. This paper has no position on such sociologists, and I have no intention to define whether they are or are not sociologists. Their work is not for policymaking, knowledge transmission or for building a credible and transparent resource for others to follow, and that is ok and may have artistic and philosophical value. For the rest of sociology, I argue we need open science because (a) sociology is not unique and (b) sociology according to some of its most important voices should follow open science principles.
3.1. Sociology Is Not Unique
The largest sociological associations over the last century have been in Germany, the United States, and Japan. Next century will be India and China, but I focus here on these older sociological associations as core institutions of the global development of the sociology discipline. Each of the three has clear guidelines about sharing data, ethical practices, and reproducibility, similar to the other social science disciplines, see
Table 1.
An estimated 78% of the major sociology journals have long-standing transparency policies that mirror those in
Table 1 [
36]. Unfortunately, these policies are mostly artifacts on paper without much enforcement. For example, only 37% of sociology articles published in the mainstream journals in 2012–2014 include shared data and/or materials [
36]. In 2015, a small group of sociologists tried to obtain materials from the authors of 53 prominent sociological studies. They obtained these from just 19%, and only 20% of all the authors they contacted bothered to respond at all despite several requests [
37]. These are unambiguously studies that used quantitative methods, but the problem appears to cut across methodological approaches. For example, a review of 55 qualitative interview-based studies from a top management research journal found that none of them shared sufficient information to allow another scholar to engage in a “conceptual replication”, as in repeat the study with the same methods [
38]. This evidence suggests sociologists are free to hide the data and materials that led to their findings without recourse, despite guidelines.
Many associations and publishers are addressing this problem through the adoption of the
Transparency and Openness Promotion Guidelines (TOP) [
39]. The TOP guidelines with help of the Center for Open Science support journals to improve science. Journals can become signatories of TOP, and in doing so they either adopt and enforce new transparency guidelines or certify that they already meet certain transparency standards. Most of the well-known psychology journals and several political science journals signed on. Other major journals such as the
Journal of Applied Econometrics and later the
American Economic Review adopted their own analogous and enforced transparency guidelines.
Until 2017, the only higher-ranking sociology journals that signed TOP were
Sociological Methods and Research and
American Journal of Cultural Sociology. In 2017, Elsevier (Amsterdam, The Netherlands) dictated that all its journals adopt guidelines, and this added
Social Science Research to the list (more about Elsevier in
Appendix E). At the time of writing this, the flagship journals
American Journal of Sociology and
American Sociological Review neither signed TOP nor enforce their own guidelines. Of top German sociology journals, the
Kölner Zeitschrift für Soziologie und Sozialpsychologie is the only signatory, and no Japanese sociology journals signed.
If intransparency is pervasive in sociology, then research cannot be (a) checked for errors, (b) reproduced, or (c) simply critiqued. Even when exact reproducibility is not the goal, as often is the case with context-specific interpretive research, readers must take a giant leap to trust what others report when methods remain shrouded in mystery. Part of the problem is that sociologists express little interest in reproduction or checking others’ works. There are few replications in the history of sociology, and if anything, they decreased over time until recently [
40]. For example, searching the articles in the
American Journal of Sociology and
American Sociological Review reveals an average of 0.73 replications published per year (22 total) from 1950–1980, only 0.27 per year (8 total) from 1981–2010, and then 0.89 per year from 2011–2020 (see the ReplicationWiki for a list
https://replication.uni-goettingen.de/wiki/index.php).
Looking at the popular usage of the terms
sociology and
science from a linguistic perspective suggests that sociology is a science. It is the task of philosophers and methodologists to argue at length about things like, ‘what is science’; but dictionaries and the usage of language on a day-to-day basis offer an insight into sociology’s public ontologies, and these are of the highest interest here. Major dictionaries’ definitions in English, German, and Japanese appear in
Table 2 and
Table 3 and suggest sociology is a science in popular denotation (Online dictionaries Merriam-Webster and Oxford (English), Duden (German), and Weblio (Japanese) accessed 30 August 2019.
In a 1983 speech, the economist George J. Stigler explained why disciplines such as sociology do not have a Nobel Prize. His reasoning was, “that they already had a Nobel Prize in literature” [
41], p. 25. Such stereotypes reflect deep misunderstandings of what sociology and science are (see
Table 1 and
Table 2), but at the same time indicate a poor reputation of sociology. Despite common stereotypes, economics is not more scientific than sociology. Economists use socio-economic data to test and develop theories about the world just like the other social science disciplines. In fact, their over-reliance on formal equations, like an over-reliance on p-values, may lead to unreliable results [
42]. The problems are not entirely from outside sociology. Internally, some sociologists look down on others, and attempt to exclude them from sociology or sociological funding. This often can be reduced to some form of naturalism versus interpretivism, and resentments that emerged over recent trends offering greater funding for quantitative research. Any internal conflicts surrounding these inequities or divisions can only further damage the perception of sociology among other disciplines and the public.
The “-ology” in the word sociology derives from Latin (“logos”) to denote
the study of something. Sociology is the study of the social, or
the science of the social consistent with all three definitions in
Table 2. Sociology is a science just like political science, psychology, social psychology, and economics. If sociology is considered science from an ontological and popular linguistic standpoint, the next question is what is science in the same popular definition approach. Looking at such definitions in
Table 3 suggests
science is a systematic, practical, logical, and reliable study of things in the public, common uses of the word across the English, German and Japanese languages.
3.2. Open Science Is Sociology
A movement is underway across science to correct the problems identified thus far. Various academics, associations, and funding agencies are now pushing to “open” science. To make it more transparent, reproducible, accessible, and less biased. The various efforts fueling this process can collectively be labeled the Open Science Movement (OS Movement) and suggests individual and institutionally sanctioned ethical practices, most often the sharing of all research materials, preregistering research plans, open access (to results and data), eliminating publication bias and ‘hacking’ to get results, and a renewed interest in replication. This movement arrived in the social sciences only in the last decade or so. In a short period, the OS Movement led to rapid changes in psychology, and initiated incremental changes in other areas such as international relations, economics, communications, political science, and social psychology.
The basic ideas of open science are consistent with the ideals of three of sociology’s key historical contributors (again see a brief review of what open science is in
Appendix D). Robert K. Merton repeatedly argued for openness in science, including communal research for the public good. In fact, he is often cited in disciplines outside of sociology because he studied science so extensively; but he is an obvious choice to discuss here. Max Weber and Jürgen Habermas have linkages to open science ideals that are perhaps less well known or apparent on the surface, and whose different takes on social science between a more positivist versus critical and interpretivist epistemology are striking. I review some of these three scholars’ ideas here in an attempt to locate open science as a part of sociology based on critical self-definitions.
Merton’s ideas provide a foundation for open science as sociology. According to his norm of
organized skepticism sociologists must consistently certify the knowledge they produce. He argued this should follow communalism, where they must do
communalism they must do this in a community of sociologists (Merton used “communism” in the original text, but this word has a different connotation today that may cause confusion). This definition demands that all scientists have access to the same knowledge or materials of knowledge construction and have the opportunity to participate in scientific exchanges without these exchanges taking place in secret, following his concept of
universalism. All together, these norms call for open access, open data and transparency (communalism and universalism), and reproducibility (communalism and organized skepticism) identical with most definitions of open science and the OS Movement. Merton proposed these norms as a paradigm shift during his time, and the OS Movement proposes similar norms today calling for a similar paradigm shift (not only in sociology but across all science, see for example Chubin [
43]).
Some decades before Merton, Weber stated that the task of a sociologist is to provide facts while engaging in critical self-clarification [
44], p. 505. He was careful to define “facts” as both objective and subjective—also evidence that Weber was not purely a positivist as many might believe. Weber argued that the effective sociological teacher avoids imposing subjective ideology (e.g., political or cultural preferences) when presenting facts to students or colleagues (
Werturteilsfreiheit: ‘value-free’/‘judgement-free’). Each sociologist needs to have in mind their own subjective interests and goals and how these relate to the facts they present in order to do this. Weber proposed therefore that sociologists cannot separate the object of inquiry from the method of inquiry, namely the sociologists themselves cannot be entirely exogenous to the social world they observe (
verstehendes Erklären; something like ‘self-awareness’ and ‘clarification’ in the research process [
45], p. 495). To maintain this self-awareness as a researcher in the process of research, a community of researchers is necessary to provide insights that lead to self-awareness through feedback. The impact of a community is limited to transparent and continual critique (as Merton also argued). For example, the hoax papers by Lindsay, Boghossian, and Pluckrose were so attractive subjectively, that editors and reviewers rarely bothered to question their methods or data.
Perhaps surprisingly to some, I believe that Habermas makes claims that are consistent with the OS Movement. He continuously insisted on a normative change at the level of communication. He advocated for a rationalization of communication that would open up all communication for consumption. This would reign in individuals’ intentions to perform communication to serve their own interests and promote instead a goal of understanding between individuals [
48], pp. 94–101. When the public sphere is commodified and controlled by private interests, there is asymmetric production of communication (those who control the communication channels control the content) and asymmetric consumption (higher quality communications become more expensive) [
49]. This distorts communicative action. If I switch the word “communication” with “science” or “sociology” then open and transparent communication would reshape science in the same ideal way that Habermas imagined for society. After all, science is part of society, so the OS Movement could be seen as simply a sub-section of a larger Habermasian movement in all social spheres. When the common values of science shift toward more openness it can solve the ‘pathologies’ of science, namely profiting from closed knowledge communications and the rigid norm of publish-or-perish.
These three sociologists are monumentally influential, but worlds apart in their philosophies, arguments, and research; Habermas was far more than a sociologist and is cited across many disciplines, not only sociology. Merton was a positivist, and Weber was mostly a positivist, describing the world as he encountered it; whereas Habermas was largely an interpretivist and described a world with hidden meaning and power structures. However, they three similarly expressed concern about scientific misconduct and the influence of private interests. A simple reading of their basic claims suggests a more responsible science if not responsible social action on the whole. Weber, like Merton, asserted that science as a vocation can be problematic. Individuals may place vocational goals ahead of scientific goals, and this can impact the reliability of their results. Habermas, like Merton, argued that morality and social utility are necessary scientific research norms.
All three theorists set forth ideal sociological practices that are still unrealized by the discipline today. Each in their own way argues that without meeting these ideals, sociological knowledge is not useful. In my view, production of useful knowledge is a necessary condition for practicing sociology and consuming public funds in this process. Of course, sociology will never be perfect, but if we choose not to decide to pursue open science, we still have made a choice—an inaction whose implications may be harmful in ways we cannot imagine further into the future.
Weber called on scientists and teachers to expose inconvenient facts without promoting one or another position. In this essay, I tried to present what I believe to be an objectively inconvenient fact in sociology: Our rich universe of theory and critique is unsubstantiated, because we only rarely provide the materials for, and engage in, this substantiation process. Sociology is not special in this regard; however, other neighbor disciplines have taken the lead in reforms. This risks putting sociology at a (further) disadvantage in terms of public trust, funding, and contributions to science as a whole. Therefore, I see my ‘position’ in favor of open science as a practical response to the status quo in sociology.
To me this position is obvious given the reality of sociology. When papers are not scrutinized then they are not reliable. They do not consist of findings but illusion and possibly sophistry. This is not a useful knowledge. We cannot discern whether this is just but a bunch of academics playing a game to win notoriety and citations, push particular private agendas, or actually science.
The prestige one can gain through academic work is theoretically unlimited. This potential for prestige accumulation produces “ego-maniacs” [
13], p. 1358. Bruno Frey calls the academic institution a system of “Publishing as Prostitution”, because we face a monumental tradeoff between pursuing our “Own Ideas and Academic Success” [
46]. Perhaps in an obfuscated act of irony that motivates open science (or simply evidence of how easy it is to be unethical without really trying), Frey got into trouble after he published a similar paper in four different journals [
47]. If we as sociologists, or any scientists for that matter, cannot provide useful knowledge (‘ideas’) to the social world, then taking the OS Movement seriously would lead to a call on public and private funding agencies to discontinue supporting us sociological researchers. This is not a radical or self-defeating position, unless our only reason for practicing sociology is to have a comfortable job with research funding. By supporting this position, just like being transparent before and after we conduct our research, we ensure our goals are ethical and target useful information as opposed to private status accumulation, in line with the sociology Weber and Merton argued for.