Publish-and-Flourish: Using Blockchain Platform to Enable Cooperative Scholarly Communication

Round  1

Reviewer 1 Report

This paper, “Publish and flourish” presents a proposal for a scholarly communication system based on blockchain technologies. The specific focus is on defining a system in which scholarly paper submissions, reviews, and revisions are documented and managed via a blockchain ledger.

The paper is clearly written and straightforward to read. The proposed model is thought-provoking for anybody who is interested in how blockchain technologies might be applied within scholarly communication institutions and infrastructures. If widely adopted, the approach described here could present opportunities described in the paper for new kinds of incentives and trust models within the scholarly communication system. In my reading, however, there is a very big “IF” component to this work.  This paper, as currently written, is based on some problematic assumptions, and does not reflect enough on the reasons why the current state of journal-centric scholarship has maintained its dominance. The paper is also very light on practical implementation details. It presents a techno-centric solution to very complex social and institutional problems. Below I provide suggestions and pointers related to these points that I believe could strengthen the overall paper, and improve its utility to potential readers.

CONCEPTUAL COMMENTS

- Pg. 2, the first paragraph talks about how science is “logically centralized (scientific community must agree on one state – scientific truth)”, and is a process that leads to a “broadly shared consensus” that is reached “through scholarly communication”.  This is extremely problematic. What does “broadly shared consensus” mean? Does any scientific community actually agree on one state for any research topic? A foundational idea in the sociology of science is that any “consensus” or “one state” is inevitably temporary (Kuhn, 1962). Philosophers of science also describe how securing reliable research findings is a process of explanation, iteration, triangulation, and convergence within and across multiple social institutions (Cleland, 2011; Leonelli, 2018). So the goal of any scholarly communication system should not be to somehow try to achieve a consensus, because this is not how science works.

- Pg. 2 – the fourth paragraph presents three models of academic publishing that exist today. While these are accurate on a high-level, significant blending occurs within these models. Some journals are paywalled for only a period of time, for example, before being made open access. It might be more useful to think of journals as “club goods”, as discussed by Potts et al (2017). This idea helps move the discussion about scholarly communication systems beyond the open/closed access distinction. Potts et al also talk briefly about applying blockchain technologies to scholarly communication, in some ways that might inform the proposed approach in this paper.

- pg. 3 – The claim is made in the second paragraph that the solution to the problems in scholarly communication is “a change from unmodifiable to modifiable published scholarly content that by definition demands open and continuous review process.” At this point, it might be worth considering why journals and the current scholarly communication system is dominant? The problems with the current scholarly publishing model are significant, and do limit the kinds of projects and products that scholars are free to produce. This paper does not, however, reflect in any length on the flip side of this issue, namely, with all of these problems, why are traditionally structured journals still dominant? Proposals to radically (or modestly) reinvent journal publishing models have been promulgated since the 1990s, along with predictions of the death of traditional journals (see Odlyzko 1995). Reviews of this literature can be found in a few places (Kling & Callahan 2003; Borgman 2000, chap 4). Journals have switched largely to electronic form, but otherwise, despite these many studies and proposals, traditional journals still remain the primary mode for the publication of scholarly work. It is important to reflect on why this is. In particular, some of the aforementioned works have also discussed how the current journal publication model provides significant benefits, and serves to fulfill several key functions necessary for scholarship to thrive, such as registration, certification, legitimization, dissemination, access, preservation, and curation (Borgman 2007, chap 4; Van de Sompel et al 2004).  I think the paper’s proposal would be much stronger if it directly addressed these core functions of scholarly communication. Would the I8X be different if it was designed to support these key functions (legitimization, etc), instead of the ones listed on pg 6 line 210: “transparency, integrity, and engaging the members of the community”? It is not clear, for example, how preservation or curation fits into the proposed system. Who is to be doing the curation of content and reviews, as shown in Figure 3?

- On “transparency” – since transparency is described as a key motivation and benefit of the blockchain approach, the paper should engage with the recent paper by Hofman and Novin (2018) that unpacks the notion of transparency as related to blockchain technologies. As they describe, using the blockchain facilitates achieving some aspects of what “transparency” might entail, but not all.

- pg. 3, line 117, a reference is made to the “tragedy of the commons”. Shortly after, a mention is made of the need to address these issues via “information, identify, institutions and incentives.” The paper does discuss how the I8X approach would potentially address information, identity, and incentives issues, but does not come back to discuss how this would address the “institutions” component. This is an important omission. It is notable that Elinor Ostrom, who won the Nobel Prize for her analysis of commons tragedies, primarily focused on the role of institutions in the failure and amelioration of such situations. See Ostrom 2005 for relevant examples using game theory and field work. My basic point here (informed by Ostrom, see also Agre 2000) is that any technical innovation in the scholarly communication space has to be coupled with institutional innovation/evolution. If the institutional components are ignored, technical innovations will get nowhere.

- pg. 4, lines 128 states that “pseudonymous peer-to-peer communication, with a choice for a user to opt for proof of identity to others, induces a strong sense of community”. This is very counterintuitive. How does pseudonymous communication with an optional proof of identity induce community? This needs to be explained further.

- pg. 5, lines 191-192 state that increasing trust requires that papers and reviews “must not be static objects.” This statement is not well motivated. I refer back to my above comments about the longevity of the current scholarly communication system, which has been successful in establishing trustworthy communications for over 300 years using largely static documents. Perhaps trust can be established via non-static objects, but it clearly can be done with static objects.

- pg. 5, lines 194-196 states that “a research paper is, in a sense, a contract between author(s) and the community”. This is an unfounded and extremely problematic assertion for which no evidence or backing is provided. In what ways is a research paper like a contract? I simply do not see how this works as a direct equivalence or even an analogy. A research paper is a statement, a claim perhaps, made as part of a (usually) long-running set of conversations and arguments about particular topics. There is nothing about this process that is like a contract.

- How would this new model work? The paper is very short on practical details of how to institute this approach. I realize that this is not the goal of the paper, but without more detail on how this would work in practice, it is hard to take the proposed model too seriously. I think the approach makes the most sense when applied very concretely, e.g. to the functioning of specific journals. If someone was creating a new journal now (or had tight control over a specific existing journal), it would be very possible for them to implement this model. It is much harder for me to imagine how any of the existing mega (or even moderately sized) publishers would move to this model. The practical challenges are huge. 

- One specific example of the need for more practical implementation details and attention to institutional factors is the discussion of tokens. From Pg. 8, line 255, “The reputation of the user on I8X is a function of number of tokens the users hold at each instant of time.” As noted in the paper, reputation is a key component of academic work. It is very hard to imagine, however, any realistic scenario in which academic reputation would get formalized into technically instantiated tokens. Unpacking realistic situations in which such tokens could replace existing (largely informal) reputation mechanisms would be very helpful to understanding how this model could actually work.

MINOR COMMENTS

 - pg. 3, line 105, the game theory section is based on an assumption that the games are played by “rational actors”. I know this is common in game theory, but this is really problematic when applied to anything academics do.

- The definitions of C and D seem to shift somewhat through the game theory section. When introduced, C is described as “contributing reviews”, and D is implied to be the opposite, namely not doing reviews. Later on page 5 (line 161), C is described as “honestly reporting their research” and D is to “hype up the paper…”. This inconsistency makes the section somewhat more difficult to follow.

 - pg. 7, line 220 says that the I8X database has 150 million metadata points. Where does all of this metadata come from? Is the system already in use by some journals?

- The code snippets on page 9 and 10 do not add value to the paper. I suggest replacing these with textual (or diagrammatic) descriptions of any important features of the code, and pointing readers to the actual code, if it is accessible somewhere online.

WORKS CITED

Agre, P.E. (2000). Commodity and Community: Institutional Design for the Networked University. Planning for Higher Education, 29(2): 5-14. http://dlc.dlib.indiana.edu/dlc/bitstream/handle/10535/4553/agre.pdf?sequence=1

Borgman, C.L. (2000). From Gutenberg to the Global Information Infrastructure. Cambridge MA: MIT Press.

Borgman, C.L. (2007). Scholarship in the Digital Age. Cambridge MA: MIT Press.

Cleland, C. E. (2011). Prediction and Explanation in Historical Natural Science. The British Journal for the Philosophy of Science, 62(3), 551–582. https://doi.org/10.1093/bjps/axq024

Hofman, D. & Novin, A. (2018). Blocked and chained: Blockchain and the problems of transparency. 2018 ASIS&T Annual Meeting (pg. 171-178). https://www.asist.org/publications/annual-meeting-proceedings/

Kling, R. and E. Callahan. (2003). Electronic journals, the internet, and scholarly communication. Annual Review of Information Science and Technology. 37: 127-177. https://doi.org/10.1002/aris.1440370105

Kuhn, T. (1962). The Structure of Scientific Revolutions. University of Chicago Press. 

Leonelli, Sabina (2018) Re-Thinking Reproducibility as a Criterion for Research Quality. [Preprint]. http://philsci-archive.pitt.edu/14352/

Odlyzko, A. M. (1995). Tragic loss or good riddance? The impending demise of traditional scholarly journals. International Journal of Human-Computer Studies, 42(1), 71–122. https://doi.org/10.1006/ijhc.1995.1004

Ostrom, E. (2005). Understanding Institutional Diversity. Princeton, NJ: Princeton University Press.

Potts, J., Hartley, J., Montgomery, L., Neylon, C., & Rennie, E. (2017). A journal is a club: a new economic model for scholarly publishing. Prometheus, 35(1), 75–92. https://doi.org/10.1080/08109028.2017.1386949 

Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., & Warner, S. (2004). Rethinking scholarly communication. D-Lib Magazine, 10(9). https://doi.org/10.1045/september2004-vandesompel

Author Response

We are very grateful to this reviewer for providing us with a very deep and thoughtful review of our manuscript, along with many valuable pointers on how to improve the text. Having considered all points raised by the reviewer, we provide here our responses, comments and views, and we explain how we address these in the manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper describes a blockchain based platform for decentralized scholarly communication. I would strongly advise the authors of this paper to rewrite their introduction, namely regarding peer review since the proposal intends to present an open science platform which would allow open and continuous peer review. 

The paper would benefit from the revision of:

some of the keywords chosen (e.g. decentralisation; smart contract; publish-and-flourish);

title (the manuscript is about a blockchain based platform that does not appear in the title), and

abstract (the abstract should describe the content of the manuscript).

Author Response

We are thankful to this reviewer for the comments on how to improve our manuscript. 

Following reviewer's advice, we have changed the title of the manuscript to better reflect its content, corrected the keywords, and revised the text.

Round  2

Reviewer 1 Report

This paper, "Publish-and-Flourish: using blockchain platform to enable cooperative scholarly communication" has been revised to address my review of the previously submitted draft. I thank the authors for their detailed response, and substantive changes to the text. The changes in this draft address most of my comments, by adding additional context and adjusting problematic claims. Although the practical challenges in implementing the proposed model are significant, it is important to have detailed proposals like this out in the open literature to stimulate rigorous discussion, evaluation, and evolution of these ideas.
I have one remaining issue that I think was not addressed fully in the revision, specifically the discussion of the functions of scholarly communication on page 8 (lines 271-283). I appreciate the adjustment of this point based on my previous comment, but the new text mis-interprets these functions, specifically registration, certification, and legitimization in a narrowly technical way. Here is a brief passage from Borgman's book for context:
[Scholars have distinguished] "between 'certification' as the 'quality stamp' given by the community and 'registration,' or the process of recording the results on behalf of authors. Establishing priority also is part of the legitimization function. ...If disputes arise over who can claim priority for a finding, discovery, or interpretation, these registration and publication dates can have legal significance." (Borgman, 2007, pg 66)
"Registration" as described in this quote, is thus focused on the results (e.g. the findings) of the research, not the ability to sign up for a system, which is how it was interpreted on line 274. Likewise, the "certification" function is depicted by Borgman as a 'quality stamp' from a community. In other words, by going through the process of publishing a journal paper (submission, review, response, 2nd review, editing, etc), the community ensures a sense of quality to published papers. As flawed as that process might be (well described in the present submission), this 'certification' of quality is a broader social function that cannot be subsumed by writing digital hashes for papers and associated metadata.
In the authors' review response on this point, they state, "For the most part, the persistence of the traditional journal system is due to the evaluation processes and methods used in recruiting for university positions and grant awarding" (pg 2). Evaluation is a critical factor, but in my view this is an oversimplification. Journal articles have evolved over 300+ years to be an extremely efficient genre of communication for scholarly research. Bazerman (1988, citation at bottom) wrote a book about the evolution of the genre. Think of how many articles we look at every day/week. We know how to examine them very quickly to determine whether they are worth reading more closely. It is then usually possible to skim an article in a very short time (less than 5 minutes) to get the main arguments/findings. The research literature simply could not be as voluminous as it is were the main vehicle (journal articles) not so efficient to digest. Articles also then provide a stable genre for the other functions discussed above (registration of research claims for purposes of priority, certification of 'quality' indicators for the article, curation of papers, etc). This was the intended broader point of the remarks in my previous review. Static journal articles are problematic, yes, but they are also highly evolved to serve the communication needs of research-focused scholars. Any proposal to move away from static journals has to account for what could be lost in such a transition.
For the immediate article, I think that the authors can engage with these functions of scholarly communication in more nuance. The below article by Van de Sompel, et al is also a good distillation of these functions.
Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., & Warner, S. (2004). Rethinking scholarly communication. D-Lib Magazine, 10(9). https://doi.org/10.1045/september2004-vandesompel
Bazerman, C. 1988. Shaping written knowledge: the genre and activity of the experimental article in science. Madison, WI: University of Wisconsin Press. http://wac.colostate.edu/books/bazerman_shaping/

Author Response

We thank this reviewer again for recognizing the value in our manuscript and for providing us with further detailed comments and recommendations on how to improve the text. We have revised and additionally referenced the manuscript following the reviewer's suggestions and have addressed the comments in the attached pdf.

Dean Korošak

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors solved the problems identified in the manuscript.

Author Response

We would like to thank this reviewer again for the recommendations and suggestions that

helped improve our manuscript.

Dean Korošak