Editorial Board Membership, Time to Accept, and the Effect on the Citation Counts of Journal Articles

Abstract

In this paper we report on a study of 1541 articles from three different journals (Journal of Informetrics, Information Processing and Management, and Computers and Electrical Engineering) from the period 2007–2014. We analyzed their dates of submission and of final decision to accept and investigated whether the difference between these two dates (the so-called “time to accept”) is smaller for the articles authored by the corresponding journal’s editorial board members and whether longer times to accept yield higher citation counts. The main results are that we found significantly shorter times to accept editorial board member’s articles only in Journal of Informetrics and not in the other two journals, and that articles in any of these journals that took longer to be accepted did not receive markedly more citations.

1. Introduction

The time elapsed since the submission of a research manuscript by its authors until the final editorial decision to accept it for publication in a journal can be called differently: Reviewing time, editorial delay, or simply the time to accept a paper. Since reviewing time may be understood by some as only the time period used by referees to review the paper and editorial delay is unclear as to the processes involved (on the part of authors, editors, referees, or all), we prefer using the term “time to accept” as the whole time period from the submission to the final decision. This period may possibly involve several rounds of reviews and revisions, plus any editorial delays, and it will be given in days because, in most cases, they are the smallest time unit that can be recorded in the editorial process. The time to accept can vary significantly for articles from different journals for obvious reasons: Diverse scientific fields, availability of referees in a specific domain, presence/absence of an electronic editorial management system, and other factors can all heavily influence the duration of the time to accept. Additionally, within one journal, articles may take longer or shorter to be accepted depending on their quality, topic or complexity, document type (original research article or a review article), length (full-length article or a short communication), response time of reviewers, timely revisions of authors, etc. It is also true that some research fields prefer conferences to journals as their publication venues. However, unlike journals, conferences have a precise schedule with a fixed “final decision” date, and manuscripts submitted between the publication of the call for papers and a “papers due” date, which makes the times to accept of conference papers much more homogeneous and far less interesting to study. Therefore, this short paper is concerned solely with journal articles.

Thus far, there have been few studies dealing with editorial and publication dates of journal articles. As late as in the 1990s, the so-called “publication delays” (the time from submission to publication) of journal articles were inspected [1,2] and only after 2000, with the widespread online availability of editorial and publication dates, researchers investigated “editorial delays” (time to accept) of papers in selected journals [3,4]. Some analyses have also been concerned with the editorial (submission, revision, acceptance) dates as such, and not with the time periods formed by them [5,6]. Additionally, only very recently, the effect of the time to accept on the citedness of a paper has been found to be weakly negative in Reference [7], as well as in Reference [8], for articles in a few top-tier journals; however, to the best of our knowledge, the influence of editorial board membership on citation counts has never been explored.

In the present study, we analyze a total of 1541 articles from three different journals (Journal of Informetrics, Information Processing and Management, and Computers and Electrical Engineering) from the period 2007–2014, which are all published by Elsevier using the same electronic submission system, and report on computer science and information science research. We examined the articles in our data set and asked the two following questions: (a) Do articles authored by members of the editorial board of a journal have shorter times to accept than other articles in that respective journal? (b) Are articles with longer times to accept rewarded with more frequent citations than those with shorter times to accept? In the rest of the paper, after testing the relevant hypotheses, we will show that the answer to question a) is “Yes” for Journal of Informetrics only, and the answer to question b) is “No” for all of the journals.

2. Data and Methods

We chose articles from three journals to be included in our analysis: Journal of Informetrics (JoI, ISSN 1751-1577), Information Processing and Management (IPM, ISSN 0306-4573), and Computers and Electrical Engineering (CEE, ISSN 0045-7906). Journal of Informetrics can be considered a “new” journal because it was founded in 2007, while the other two journals can be considered as “old” (or established) journals, both published since the 1970s. All three journals are published by Elsevier, their articles are provided with the necessary data we need for our analysis (see below), and their current editorial boards are regularly announced in journal issues. JoI was of primary interest to us since we were familiar with its focus and contents from a readers’, as well as authors’, point of view, and at the time of the commencement of collecting data for this study (April 2015), it had a relatively reasonable publishing period of 2007–2014. We also knew that this journal systematically publicized the dates of submission (date received) and of final decision (date accepted) of the papers published. Therefore, we manually inspected 471 JoI articles from 2007 to 2014 using Elsevier’s Science Direct online publishing platform, deliberately skipping editorial notes or letters or articles with incomplete editorial data.

Similarly, we collected publication data for 601 IPM articles and 469 CEE articles from the same period (2007–2014) in August 2015 and April 2016, respectively. The latter journal contains several special issues or special issue sections with articles reprinted from conference proceedings. Even though some of these reprinted articles had editorial data associated with them, we basically omitted whole special issues or special issue sections because most of their articles had no, or incomplete, editorial data or their receive and accept dates were quite restricted due to obvious reasons (conference deadlines). However, we considered individual articles, even from special issues or special issue sections, when it was possible to do so. Thus, we obtained data for 1541 articles from JoI, IPM, and CEE in total, and we gathered their citation counts from Scopus in May 2016.

3. Results and Discussion

In addition to the received and accepted dates, we also manually checked the authors of the 1541 journal articles against the members of the current editorial board of a particular journal valid in the issue of publication of an article. A simple name and institution matching procedure was used. The editorial board of JoI (throughout the years under study) included 41 distinct members and the boards of IPM and CEE had 76 members each. In this manner, we were able to produce three series of numbers (days to accept an article) for each journal: All articles (All), articles by authors from which one at least is on the editorial board of that journal (On), and articles by authors, none of which are on the editorial board of that journal (Off).

Table 1. Descriptive statistics of three journal data sets and of their aggregate.

	JoI			IPM			CEE			Total
	All	On	Off	All	On	Off	All	On	Off	All	On	Off
# Articles (n)	471	151	320	601	30	571	469	12	457	1541	193	1348
Mean Days to Accept	101	89	107	285	269	286	373	368	373	256	134	273
Std. Deviation of DtA	78	61	85	205	256	202	197	167	197	205	151	205
Min. Days to Accept	0	0	12	2	59	2	2	2	27	0	0	2
Max. Days to Accept	608	418	608	1301	1298	1301	1428	695	1428	1428	1298	1428

presents basic statistics of the series for all journals under consideration as well as their combined aggregate and Figure 1 accompanies this table with a graphical representation using box plots.

We can see that 471 JoI articles needed 101 days on average and JoI, as such, was a relatively “fast” journal, whereas both IPM and CEE were substantially “slower” with an average of 285 and 373 days to accept, respectively. This fact is also well documented by the different time spans between the “fastest” and “slowest” articles, ranging from 0 to 608 days to accept in JoI, from 2 to 1301 in IPM, and from 2 to 1428 in CEE. These characteristics of the data sets are visualized in Figure 1, where the lower edge of each box marks the end of the first quartile, the upper edge marks the beginning of the fourth quartile, and the inner edge (separating two differently colored sub-boxes) marks the median value, and the “whiskers” represent the minimum and maximum value of time to accept. We can immediately see that the bars of IPM and CEE are generally shifted towards a higher number of days to accept than those of JoI.

What we may further see in Table 1 is that the number of articles written by editorial board members is significant in JoI (151 or 32%), but low in both IPM and CEE (30 and 12, or 5% and 2.6%, respectively). The bulk of these articles in JoI were written by L. Bornmann, R. Rousseau, L. Leydesdorff, L. Egghe, and M. Thelwall with 25, 21, 20, 14, and 13 papers, respectively. The mean number of days to accept of the articles authored by editorial board members was 89 in JoI, 269 in IPM, and 368 in CEE. However, only the difference between the mean days to accept of “on-board” and “off-board” JoI articles is statistically significant (at the 0.01 level using a two-sample t-test assuming unequal variances). This means that we could not reject the null hypothesis of equal mean days to accept of “on-board” and “off-board” articles in both IPM and CEE. Thus, the speculation that articles by editorial board members have shorter times to accept than other articles turns out to be true only in the “new” journal (JoI), but cannot be proved for the “old” ones (IPM and CEE). Regarding the low number of “on-board” articles in both latter journals, however, this finding should be corroborated by further tests on larger data sets with more articles authored by editorial board members. Additionally, for similar reasons, it would be inappropriate to draw general conclusions from the significant difference between “on-board” and “off-board” mean days to accept of the articles from the three journals as a whole (134 vs. 273) because of the unequal distribution of “on-board” articles across the journals and the strong bias towards JoI.

We also made an attempt to focus our analysis on papers authored by editors-in-chief (being themselves considered editorial board members), but were unable to proceed properly due to the low numbers of such papers (14 in JoI, 2 in IPM, and 0 in CEE). At first sight, however, the articles in JoI had a remarkably short mean time to accept (57 days), and, in one case, one such article [9] took 0 days to be accepted. The editors-in-chief were quite stable in the journals: L. Egghe served for the whole period of 2007–2014 in JoI, T. Saracevic and F. Crestani took their responsibilities in 2007 and in 2008–2014 for IPM, and so did M. Jamshidi and M. Malek in 2007–2009 and in 2010–2014 for CEE. It would thus seem that L. Egghe took the most advantage of publishing in “his” journal, but when looking for his papers (articles, reviews, and conference papers) from 2007-2014 indexed in Scopus, we found 86 documents and, therefore, his 14 JoI articles made up only 16% of his total production. Similarly, F. Crestani published 73 papers in the respective period and his two IPM articles were, thus, less than 3% of his publication output. The other editors-in-chief above did not publish at all in “their” journals, and, thus, it appears that, based on the data we investigated, being responsible for a journal does not result in excessive publication therein. Let us also remark that a new journal (JoI) usually invites the members of its editorial board to publish articles in the coming issues, and that the rejection rate often tends to be lower in the initial volumes as mentioned in Reference [9].

Another hypothesis we wanted to test was the assumption that the articles that took longer to be accepted would be more frequently cited in the future because of their (presumably) greater attention to points made by the referees. Of course, other things, such as a lengthy search for a qualified referee or reviewer delays, may also play a role, but the basic assumption has already been used in previous research (in References [7,8] the opposite hypothesis was tested). To this end, we computed Pearson correlation coefficients between two variables, days to accept a paper and relative citedness of that paper, for all articles in the journals under study. The relative citedness of a paper was determined as the number of citations to that paper divided by the mean number of citations per paper in a specific journal and year (in fact, an instance of the mean normalized citation score—MNCS). The citation counts (including self-citations) were retrieved from Scopus, as stated earlier, and, for convenience, the relative citedness is expressed in percentage points in the text below. We found that these two variables (days to accept and relative citedness) were uncorrelated in all three journals. For JoI, see the scatter plot in Figure 2. The correlation coefficient is about −0.02 here and two articles are highlighted because of extreme values: Reference [10] with 52 days to accept and a citedness of 779% as a red rectangle, and Reference [11] with 608 days to accept and a citedness of 79% as a red triangle. The relative citedness of “on-board” articles is 124% compared to 89% of “off-board” articles. In IPM (Figure 3), with a correlation of −0.06, four articles with extreme values are highlighted: Reference [12] with 386 days to accept and a citedness of 1068% as a big red rectangle, Reference [13] with 270 days and 1015% as a small red rectangle, Reference [14] with 1298 days and 0% as a red triangle, and Reference [15] with 1301 days and 93% as a red circle. Similar to JoI, the relative citedness of “on-board” articles is 124% compared to 99% of “off-board” articles. As for CEE (Figure 4), the correlation of days to accept and relative citedness is close to 0 as well (−0.04) and the highlighted outliers are Reference [16] with 383 days to accept and a citedness of 907% as a red rectangle and Reference [17] with 1428 days and 53% as a red triangle. Unlike JoI and CEE, the relative citedness of “on-board” articles (97%) is a little smaller than that of “off-board” articles (101%), but this is certainly affected by the very low number of papers written by editorial board members. A summary scatter plot for the articles from all three journals (n = 1541, r = −0.04) is presented in Figure 5, where the days to accept are made relative by analogy with the relative citedness. We may conclude that we found no correlation between the time to accept and the citedness of the articles under investigation, but that some articles definitely had a very advantageous ratio of time to accept to citedness (such as [10,12,13,16]), unlike some others with a very poor ratio ([11,14,15,17]).

4. Conclusions

The average time from the submission of a research manuscript until the final decision of its acceptance (or rejection) can vary greatly between individual journals. Apart from the scientific disciplines and topics of the articles, their document type (original research article or review), length (full-length article or short communication), journal editorial system (electronic or not) and its effectiveness, reviewer selection mechanism, and the response time of reviewers, it is also up to the manuscript authors themselves, and their ability to react quickly and provide timely quality revisions, that can be accepted without delay. Therefore, even within a journal, the time to accept an individual paper can vary considerably. In this study we analyzed 1541 articles from three journals in related fields (computer science and information science) and from the same publisher (Elsevier): Journal of Informetrics (JoI), Information Processing and Management (IPM), and Computers and Electrical Engineering (CEE). We wanted to find out whether articles written by editorial board members of their respective journals had a shorter mean time to accept, and whether articles with a longer time to accept would be rewarded with more frequent citations in the future. The main conclusions of our analysis are: (a) we did not find any significant difference between the mean time to accept of articles authored by editorial board members and the other articles, except for the relatively new journal JoI, which had, by far, the largest amount of editorial board members’ articles (both in absolute and relative terms), and (b) we did not find any correlation between the time to accept and citedness of the articles under study. In the future, it will be useful to repeat our experiments with much larger data sets to see if the main conclusions still hold.