Review Reports - Hybrid Gold Open Access Citation Advantage in Clinical Medicine: Analysis of Hybrid Journals in the Web of Science

Round 1

Reviewer 1 Report

Language editting requiired

Author Response

Reviewer 1’s comment:

Language editing required.

Response:

Thank you very much. We have carefully checked the language again and edited it as necessary.

Reviewer 2 Report

The characteristics of journals used in the analysis were as follows: (1) indexed in the 87 Science Citation Index Expanded (SCIE), Social Science Citation Index (SSCI), or Arts & 88 Humanities Citation Index (AHCI), (2) were hybrid journals with a percentage of hybrid 89 Gold open access articles between 10 and 90, and (3) had hybrid Gold open access articles 90 of at least 20 articles. Fully Gold open access journals or those indexed in Emerging 91 Sources Citation Index (ESCI) were excluded.

Reviewer comment: Why weren’t journal indexed in Medline/Pubmed sought?

In Table 1, it would be more appropriate to refer to all publishers by their formal name, ex, Oxford should be Oxford University Press and Sage should be SAGE (in all caps) Publishing, etc.

An increase in cita-147 tions may be attributable to several factors, including but not limited to immediate access 148 to the article [41,42].

It may be useful to expand upon this. For example, most if not all publishers allow articles to publish online in advance of print at which point the article receives a DOI and is immediately readable and citable. Some publishers have better metrics than others and allow the “raw” (non-edited, accepted ms) to publish in advance, and quickly, which could cause a further skew of the data/results in this regard.

Fourth, subscription articles 188 might have been openly available through other modes [45], none of which were taken 189 into account in our analysis.

I appreciate the inclusion of this statement because it is a limitation of the results.

Hybrid Gold open access articles are 207 more likely to be cited by peers than subscription articles published in the same journal 208 i_n_ _t_h_e_ _‘C_l_i_n_i_c_a_l_ _M_e_d_i_c_i_n_e_’ _d_i_s_c_i_p_l_i_n_e_._ _

While the authors present good work and make a good case for the conclusions, does this offer anything new to the literature or is this conclusion “common sense”? It seems rather obvious that OA journals (whether gold or otherwise) which are freely available would receive more citations than locked-down subscription journals due to the nature of the firewall and inability of the masses to read them freely.

Author Response

Reviewer 2’s comment:

Reviewer comment: Why weren’t journal indexed in Medline/Pubmed sought?

Response:

We sought the journals in Clarivate’s Journal Citation Reports (JCR) to obtain relevant parameters used for analysis. Some journals indexed only in Medline/Pubmed (but are not included in JCR) have no relevant parameters for our analysis. That is the reason why we did not include journals solely indexed in Medline/Pubmed in this study.

Reviewer 2’s comment:

In Table 1, it would be more appropriate to refer to all publishers by their formal name, ex, Oxford should be Oxford University Press and Sage should be SAGE (in all caps) Publishing, etc.

Response:

Thank you for your suggestion. We have revised Table 1 according to the reviewer’s suggestion.

Reviewer 2’s comment:

An increase in citations may be attributable to several factors, including but not limited to immediate access to the article [41,42]. It may be useful to expand upon this. For example, most if not all publishers allow articles to publish online in advance of print at which point the article receives a DOI and is immediately readable and citable. Some publishers have better metrics than others and allow the “raw” (non-edited, accepted ms) to publish in advance, and quickly, which could cause a further skew of the data/results in this regard.

Response:

Thank you for pointing this out. The present study was designed to determine a citation advantage in hybrid journals, aiming to minimize confounding factors arising from editorial policies (e.g., publication lag or early view effects). We discuss this point in the manuscript (Lines 157-160).

Reviewer 2’s comment:

Fourth, subscription articles might have been openly available through other modes [45], none of which were taken into account in our analysis. I appreciate the inclusion of this statement because it is a limitation of the results.

Response:

Thank you for your comment.

Reviewer 2’s comment:

Hybrid Gold open access articles are more likely to be cited by peers than subscription articles published in the same journal in the ‘Clinical Medicine’ discipline. While the authors present good work and make a good case for the conclusions, does this offer anything new to the literature or is this conclusion “common sense”? It seems rather obvious that OA journals (whether gold or otherwise) which are freely available would receive more citations than locked-down subscription journals due to the nature of the firewall and inability of the masses to read them freely.

Response:

Thank you for your comment. Although Gold open access articles seem to receive more citations than subscription articles, it is not necessary to be the case. In the Introduction section, we bring some evidence to justify our research question of whether Gold open access would translate to greater recognition and in turn article citations. In the manuscript (Lines 52-58), it reads “Several studies support the existence of an open access citation advantage – increased citations of articles made available open access irrespective of the article’s quality or a trending topic of attention; however, this claim has been refuted by many other studies. It is undeniable that hybrid Gold open access articles can reach more readers than subscription ones, but it is uncertain if this translates to greater recognition and in turn article citations. Hitherto, an immediate hybrid Gold open access citation advantage has been the topic of much discussion.” Therefore, we consider that the conclusion of the present study is not common sense, rather it is valuable and useful, for example, for institutional policies or funding agencies.

Reviewer 3 Report

- This is a sound (if hardly original) piece of bibliometrics research. The paper is well structured and presented, and the arguments are solid. However, bearing in mind that this manuscript has been submitted to a special issue on "Measuring Open Access Uptake: Databases, Metrics, and International Comparisons" some further analysis on the meaning of the results could be expected. This particularly applies to the impact of the ever more widespread Transformative Agreements on the percentage of articles published in hybrid titles that are published Gold OA (i.e. will the competitive advantage remain when the number of Gold OA papers significantly increases?) It is of course too early to analyse the impact of these TAs, but some comment along the lines of "further analysis needed" should definitely be there.

- It could make sense to try to clearly state what makes this study different from previous ones that have looked into citation advantages (plenty of which are listed in a rather long bibliography). Is "clinical medicine" as a domain more relevant than its subdomains some of the references in the bibliography seem to have already previously examined for citation advantages?

- Clinical medicine is a fairly wide domain, as shown in the list of 59 categories on p. 2. Could it be possible to examine differences across these subcategories, i.e. whether specific subdomains of clinical medicine have a higher-than-average citation advantage?

- Factors like the bias towards higher-quality-than-average papers created by the need to make an APC payment are adequately described but there's no mention to the APC prices on the paper. This is of course beyond the analysis proposed in the text but it could be worth exploring a possible correlation between APC prices and citation advantages

- Bibliographic references are often assigned "in stacks" without really comparing previous findings to current ones

- The sentence on row 161 "However, it is to be noted that some funders may adopt the Plan S principles and do not financially support ‘hybrid’ open access publication fees in subscription venues [49]" is not correct. An additional "unless the publisher has signed a Transitional Agreement with the institution or consortium thereof" clause is missing to accurately describe the cOAlition S member funders' policy

- English style: there's a strange-sounding sentence in the abstract
"This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles from hybrid journals against ascertaining if hybrid Gold open access publications yield an advantage in terms of citations".
It's the "against" bit that does not fit. A possible alternative could be:
"This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles published in hybrid journals. The study aims to ascertain if hybrid Gold open access publications yield an advantage in terms of citations"

Author Response

Reviewer 3’s comment:

This is a sound (if hardly original) piece of bibliometrics research. The paper is well structured and presented, and the arguments are solid. However, bearing in mind that this manuscript has been submitted to a special issue on “Measuring Open Access Uptake: Databases, Metrics, and International Comparisons” some further analysis on the meaning of the results could be expected. This particularly applies to the impact of the ever more widespread Transformative Agreements on the percentage of articles published in hybrid titles that are published Gold OA (i.e. will the competitive advantage remain when the number of Gold OA papers significantly increases?) It is of course too early to analyse the impact of these TAs, but some comment along the lines of "further analysis needed" should definitely be there.

Response:

Thank you very much for such a valuable comment. We have added your suggestion to the revised manuscript. In the revised manuscript (Lines 194-197), it now reads “Since the Transformative Agreement is now more widely adopted, it is uncertain if the citation advantage will continue to hold true in the near future when the number of hybrid Gold open access articles dramatically increases.”

Reviewer 3’s comment:

It could make sense to try to clearly state what makes this study different from previous ones that have looked into citation advantages (plenty of which are listed in a rather long bibliography). Is “clinical medicine” as a domain more relevant than its subdomains some of the references in the bibliography seem to have already previously examined for citation advantages?

Response:

Thank you for your suggestion. We all know that one of the main drawbacks of hybrid Gold open access is the article processing charge which the authors are required to pay and many of them could not afford it (Lines 44-47). The present study provides information that may aid institutions, particularly those in the
“Clinical Medicine” discipline, in developing policies that will support the article processing charge, among other things.

Reviewer 3’s comment:

Clinical medicine is a fairly wide domain, as shown in the list of 59 categories on p. 2. Could it be possible to examine differences across these subcategories, i.e. whether specific subdomains of clinical medicine have a higher-than-average citation advantage?

Response:

Thank you for your suggestion. Such analysis is possible but may require a larger dataset. We will consider doing so in future studies.

Reviewer 3’s comment:

Factors like the bias towards higher-quality-than-average papers created by the need to make an APC payment are adequately described but there’s no mention to the APC prices on the paper. This is of course beyond the analysis proposed in the text but it could be worth exploring a possible correlation between APC prices and citation advantages.

Response:

Thank you for your mention. This is another interesting point that may be explored in future studies.

Reviewer 3’s comment:

Bibliographic references are often assigned “in stacks” without really comparing previous findings to current ones.

Response:

Thank you for your remark. We reconsider our text again and would like to give some explanations. Bibliographic references which are assigned “in stacks” are mainly cited in the Introduction section, and they should not be compared with the current finding accordingly. Bibliographic references in the Discussion section are not assigned “in stacks”; rather, they are cited to support our messages.

Reviewer 3’s comment:

The sentence on row 161 “However, it is to be noted that some funders may adopt the Plan S principles and do not financially support ‘hybrid’ open access publication fees in subscription venues [49]” is not correct. An additional “unless the publisher has signed a Transitional Agreement with the institution or consortium thereof” clause is missing to accurately describe the cOAlition S member funders’ policy.

Response:

Thank you very much for your careful consideration and suggested correction. In the revised manuscript (Lines 165-168), it now reads “However, it is to be noted that some funders may adopt the Plan S principles and do not financially support ‘hybrid’ open access publication fees in subscription venues unless the publisher has signed a Transitional Agreement with the institution or consortium thereof.”

Reviewer 3’s comment:

English style: there’s a strange-sounding sentence in the abstract
“This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles from hybrid journals against ascertaining if hybrid Gold open access publications yield an advantage in terms of citations”. It’s the “against” bit that does not fit. A possible alternative could be:
“This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles published in hybrid journals. The study aims to ascertain if hybrid Gold open access publications yield an advantage in terms of citations.”

Response:

Thank you for your suggestion. We have revised the sentence according to your suggestion. In the revised manuscript (Lines 15-18), it now reads “This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles published in hybrid journals”. The study aims to ascertain if hybrid Gold open access publications yield an advantage in terms of citations.”

Reviewer 4 Report

Comments for author File: Comments.pdf

Author Response

Reviewer 4’s comment:

The article is interesting not by the subject itself but for the very good discussions section which develops the study limitations and is frankly discussing numerous confounding factors that make journal citation studies so difficult.

Some suggestions:

1. Perhaps you should discuss in a paragraph whether hybrid open access is/is not a business model used by the publishers (speculating on the pressure to which researchers and universities are exposed in the current research environment).

Response:

Thank you for your suggestion. This issue is very interesting and can be discussed extensively. Some critics argue that hybrid Gold open access is a business model for publishers because the research community needs to pay twice or double dipping (i.e., the article processing charge in addition to the subscription fee) to the publishers. However, we consider that this issue, albeit interesting, may be beyond the scope of the present study, which primarily aimed to determine the hybrid Gold open access citation advantage in Clinical Medicine. Therefore, we have added some remarks regarding this issue and referred to additional references. In the revised manuscript (Lines 212-214), it now reads “Furthermore, whether the hybrid model for open access publication of articles is appropriate or not is also beyond the scope of the present study and it is extensively discussed elsewhere.”

Reviewer 4’s comment:

2. Including Rstudio in the citation list would be nice.

Response:

We have included RStudio in the citation list.

Reviewer 4’s comment:

3. A comment/explanation on why “clinical medicine” was chosen to study hybrid gold model impact on citations. Is this a more competitive field? Is it more dynamic than other research domains?

Response:

We chose to study the “Clinical Medicine” discipline because this is our field of interest. To be honest, we have limited information to determine whether or not “Clinical Medicine” is more competitive or more dynamic than other research domains. However, with our scope and focus under the “Clinical Medicine” discipline, the present study provides information that may aid institutions, particularly those in the “Clinical Medicine” discipline, in developing policies that will support the article processing charge, among other things.

Reviewer 4’s comment:

4. Do the authors have any opinion on the continuously increasing number of journals using the hybrid gold model?

Response:

Thank you for pointing this out. This observation is interesting and consistent with the article written by Bjork B.C. (2017) (Titled, “Growth of hybrid open access, 2009-2016”), which extensively discusses this issue. We, therefore, cited this article in the revised manuscript.

Reviewer 4’s comment:

5. The enumeration of all clinical subdomains does not seem beneficial to the reader (rows 67-86). Almost everybody knows what clinical medicine is.

Response:

Thank you for your remark. This is to help readers understand what clinical medicine is. Some readers may not be familiar with the ‘Clinical Medicine’ group in Clarivate’s Journal Citation Reports.

Reviewer 4’s comment:

6. A large number of variables were extracted from the JCR database but finally very few of them are discussed and therefore do not add value to the study; the reader should be explained what’s the purpose/rationale of these extracted variables.

Response:

Thank you for your mention. Some variables (e.g., journal metrics) were extracted to provide general characteristics of the included journals.

Reviewer 4’s comment:

7. A more schematic approach to the “materials and methods” section would be helpful to the reader:

- Inclusion criteria for articles and journals

- Exclusion criteria for articles and journals

- How did the process of item selection work?

- Etc...

Response:

Thank you for your suggestion. We have added subsections under the Materials and Methods section.

Reviewer 4’s comment:

8. There is a mention of BMC in the introduction; they are part of the Springer Nature group.

Response:

Thank you for your remark. In the revised manuscript, we have deleted “BMC” in that sentence.